Oct 31 00:24:55.082019 ip-10-0-132-173 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Oct 31 00:24:55.082029 ip-10-0-132-173 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Oct 31 00:24:55.082035 ip-10-0-132-173 systemd[1]: kubelet.service: Failed with result 'resources'.
Oct 31 00:24:55.082243 ip-10-0-132-173 systemd[1]: Failed to start Kubernetes Kubelet.
Oct 31 00:25:05.318943 ip-10-0-132-173 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Oct 31 00:25:05.318958 ip-10-0-132-173 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot 251962cacb1d4df99f05f29568131ac4 --
Oct 31 00:27:24.769499 ip-10-0-132-173 systemd[1]: Starting Kubernetes Kubelet...
Oct 31 00:27:25.211348 ip-10-0-132-173 kubenswrapper[2572]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 31 00:27:25.211348 ip-10-0-132-173 kubenswrapper[2572]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Oct 31 00:27:25.211348 ip-10-0-132-173 kubenswrapper[2572]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 31 00:27:25.211348 ip-10-0-132-173 kubenswrapper[2572]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 31 00:27:25.215756 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.215557    2572 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 31 00:27:25.215756 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.215757    2572 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 31 00:27:25.215828 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.215761    2572 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 31 00:27:25.215828 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.215764    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 31 00:27:25.215828 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.215767    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 31 00:27:25.215828 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.215770    2572 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 31 00:27:25.215828 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.215773    2572 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 31 00:27:25.215828 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.215775    2572 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 31 00:27:25.215828 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.215778    2572 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 31 00:27:25.215828 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.215788    2572 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 31 00:27:25.215828 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.215791    2572 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 31 00:27:25.215828 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.215794    2572 feature_gate.go:386] unrecognized feature gate: Example
Oct 31 00:27:25.215828 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.215796    2572 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 31 00:27:25.215828 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.215800    2572 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 31 00:27:25.215828 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.215802    2572 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 31 00:27:25.215828 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.215805    2572 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 31 00:27:25.215828 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.215808    2572 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 31 00:27:25.215828 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.215811    2572 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 31 00:27:25.215828 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.215813    2572 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 31 00:27:25.215828 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.215816    2572 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 31 00:27:25.215828 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.215818    2572 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 31 00:27:25.216279 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.215821    2572 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 31 00:27:25.216279 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.215823    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 31 00:27:25.216279 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.215826    2572 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 31 00:27:25.216279 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.215828    2572 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 31 00:27:25.216279 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.215831    2572 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 31 00:27:25.216279 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.215834    2572 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 31 00:27:25.216279 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.215836    2572 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 31 00:27:25.216279 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.215839    2572 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 31 00:27:25.216279 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.215842    2572 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 31 00:27:25.216279 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.215844    2572 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 31 00:27:25.216279 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.215847    2572 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 31 00:27:25.216279 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.215849    2572 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 31 00:27:25.216279 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.215851    2572 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 31 00:27:25.216279 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.215854    2572 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 31 00:27:25.216279 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.215856    2572 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 31 00:27:25.216279 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.215859    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 31 00:27:25.216279 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.215861    2572 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 31 00:27:25.216279 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.215863    2572 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 31 00:27:25.216279 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.215867    2572 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 31 00:27:25.217357 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.215872    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 31 00:27:25.217357 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.215875    2572 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 31 00:27:25.217357 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.215878    2572 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 31 00:27:25.217357 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.215880    2572 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 31 00:27:25.217357 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.215882    2572 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 31 00:27:25.217357 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.215885    2572 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 31 00:27:25.217357 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.215887    2572 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 31 00:27:25.217357 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.215890    2572 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 31 00:27:25.217357 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.215892    2572 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 31 00:27:25.217357 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.215894    2572 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 31 00:27:25.217357 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.215897    2572 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 31 00:27:25.217357 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.215900    2572 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 31 00:27:25.217357 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.215902    2572 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 31 00:27:25.217357 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.215905    2572 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 31 00:27:25.217357 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.215908    2572 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 31 00:27:25.217357 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.215911    2572 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 31 00:27:25.217357 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.215914    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 31 00:27:25.217357 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.215916    2572 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 31 00:27:25.217357 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.215918    2572 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 31 00:27:25.217357 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.215921    2572 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 31 00:27:25.217815 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.215923    2572 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 31 00:27:25.217815 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.215926    2572 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 31 00:27:25.217815 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.215928    2572 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 31 00:27:25.217815 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.215931    2572 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 31 00:27:25.217815 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.215933    2572 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 31 00:27:25.217815 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.215935    2572 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 31 00:27:25.217815 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.215938    2572 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 31 00:27:25.217815 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.215940    2572 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 31 00:27:25.217815 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.215942    2572 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 31 00:27:25.217815 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.215945    2572 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 31 00:27:25.217815 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.215947    2572 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 31 00:27:25.217815 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.215950    2572 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 31 00:27:25.217815 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.215952    2572 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 31 00:27:25.217815 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.215954    2572 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 31 00:27:25.217815 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.215957    2572 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 31 00:27:25.217815 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.215960    2572 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 31 00:27:25.217815 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.215962    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 31 00:27:25.217815 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.215964    2572 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 31 00:27:25.217815 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.215967    2572 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 31 00:27:25.218335 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.215969    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 31 00:27:25.218335 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.215974    2572 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 31 00:27:25.218335 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.215977    2572 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 31 00:27:25.218335 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.215980    2572 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 31 00:27:25.218335 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.215983    2572 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 31 00:27:25.218335 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.215985    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 31 00:27:25.218335 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.215988    2572 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 31 00:27:25.218335 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.215993    2572 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 31 00:27:25.218335 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.215996    2572 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 31 00:27:25.218335 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.215999    2572 feature_gate.go:386] unrecognized feature gate: Example2
Oct 31 00:27:25.218335 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.216001    2572 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 31 00:27:25.218335 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.216004    2572 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 31 00:27:25.218335 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.216006    2572 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 31 00:27:25.218335 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.216009    2572 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 31 00:27:25.218335 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.216011    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 31 00:27:25.218335 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.216014    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 31 00:27:25.218335 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.216016    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 31 00:27:25.218335 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.216019    2572 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 31 00:27:25.218335 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.216021    2572 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 31 00:27:25.218777 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.216025    2572 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 31 00:27:25.218777 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.216027    2572 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 31 00:27:25.218777 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.216029    2572 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 31 00:27:25.218777 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.216032    2572 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 31 00:27:25.218777 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.216034    2572 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 31 00:27:25.218777 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.216036    2572 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 31 00:27:25.218777 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.216039    2572 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 31 00:27:25.218777 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.216041    2572 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 31 00:27:25.218777 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.216820    2572 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 31 00:27:25.218777 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.216827    2572 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 31 00:27:25.218777 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.216832    2572 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 31 00:27:25.218777 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.216835    2572 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 31 00:27:25.218777 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.216838    2572 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 31 00:27:25.218777 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.216841    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 31 00:27:25.218777 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.216844    2572 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 31 00:27:25.218777 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.216846    2572 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 31 00:27:25.218777 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.216849    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 31 00:27:25.218777 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.216852    2572 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 31 00:27:25.218777 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.216855    2572 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 31 00:27:25.218777 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.216857    2572 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 31 00:27:25.219269 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.216862    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 31 00:27:25.219269 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.216865    2572 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 31 00:27:25.219269 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.216867    2572 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 31 00:27:25.219269 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.216870    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 31 00:27:25.219269 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.216872    2572 feature_gate.go:386] unrecognized feature gate: Example2
Oct 31 00:27:25.219269 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.216875    2572 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 31 00:27:25.219269 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.216877    2572 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 31 00:27:25.219269 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.216880    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 31 00:27:25.219269 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.216883    2572 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 31 00:27:25.219269 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.216885    2572 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 31 00:27:25.219269 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.216888    2572 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 31 00:27:25.219269 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.216891    2572 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 31 00:27:25.219269 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.216893    2572 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 31 00:27:25.219269 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.216897    2572 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 31 00:27:25.219269 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.216900    2572 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 31 00:27:25.219269 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.216903    2572 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 31 00:27:25.219269 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.216906    2572 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 31 00:27:25.219269 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.216908    2572 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 31 00:27:25.219269 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.216911    2572 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 31 00:27:25.219269 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.216913    2572 feature_gate.go:386] unrecognized feature gate: Example
Oct 31 00:27:25.219753 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.216915    2572 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 31 00:27:25.219753 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.216918    2572 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 31 00:27:25.219753 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.216920    2572 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 31 00:27:25.219753 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.216923    2572 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 31 00:27:25.219753 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.216925    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 31 00:27:25.219753 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.216928    2572 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 31 00:27:25.219753 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.216930    2572 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 31 00:27:25.219753 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.216933    2572 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 31 00:27:25.219753 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.216936    2572 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 31 00:27:25.219753 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.216938    2572 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 31 00:27:25.219753 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.216941    2572 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 31 00:27:25.219753 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.216943    2572 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 31 00:27:25.219753 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.216947    2572 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 31 00:27:25.219753 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.216949    2572 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 31 00:27:25.219753 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.216952    2572 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 31 00:27:25.219753 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.216954    2572 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 31 00:27:25.219753 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.216956    2572 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 31 00:27:25.219753 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.216959    2572 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 31 00:27:25.219753 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.216961    2572 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 31 00:27:25.219753 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.216963    2572 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 31 00:27:25.220205 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.216966    2572 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 31 00:27:25.220205 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.216969    2572 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 31 00:27:25.220205 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.216971    2572 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 31 00:27:25.220205 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.216973    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 31 00:27:25.220205 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.216976    2572 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 31 00:27:25.220205 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.216978    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 31 00:27:25.220205 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.216981    2572 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 31 00:27:25.220205 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.216983    2572 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 31 00:27:25.220205 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.216986    2572 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 31 00:27:25.220205 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.216988    2572 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 31 00:27:25.220205 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.216991    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 31 00:27:25.220205 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.216993    2572 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 31 00:27:25.220205 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.216995    2572 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 31 00:27:25.220205 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.216998    2572 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 31 00:27:25.220205 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.217000    2572 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 31 00:27:25.220205 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.217003    2572 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 31 00:27:25.220205 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.217006    2572 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 31 00:27:25.220205 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.217008    2572 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 31 00:27:25.220205 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.217010    2572 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 31 00:27:25.220205 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.217013    2572 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 31 00:27:25.220726 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.217016    2572 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 31 00:27:25.220726 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.217018    2572 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 31 00:27:25.220726 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.217021    2572 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 31 00:27:25.220726 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.217023    2572 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 31 00:27:25.220726 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.217027    2572 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 31 00:27:25.220726 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.217029    2572 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 31 00:27:25.220726 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.217032    2572 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 31 00:27:25.220726 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.217034    2572 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 31 00:27:25.220726 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.217036    2572 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 31 00:27:25.220726 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.217039    2572 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 31 00:27:25.220726 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.217041    2572 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 31 00:27:25.220726 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.217043    2572 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 31 00:27:25.220726 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.217046    2572 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 31 00:27:25.220726 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.217048    2572 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 31 00:27:25.220726 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.217051    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 31 00:27:25.220726 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.217053    2572 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 31 00:27:25.220726 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.217056    2572 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 31 00:27:25.220726 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.217058    2572 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 31 00:27:25.220726 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.217061    2572 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 31 00:27:25.221160 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.217063    2572 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 31 00:27:25.221160 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.217066    2572 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 31 00:27:25.221160 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.217068    2572 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 31 00:27:25.221160 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.217071    2572 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 31 00:27:25.221160 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.217073    2572 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 31 00:27:25.221160 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.217075    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 31 00:27:25.221160 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.217078    2572 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 31 00:27:25.221160 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.217080    2572 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 31 00:27:25.221160 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.217082    2572 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 31 00:27:25.221160 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.217085    2572 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 31 00:27:25.221160 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.217087    2572 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 31 00:27:25.221160 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.217090    2572 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 31 00:27:25.221160 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.217095    2572 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 31 00:27:25.221160 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.217100    2572 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 31 00:27:25.221160 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.217102    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 31 00:27:25.221160 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.217579    2572 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 31 00:27:25.221160 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.217584    2572 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 31 00:27:25.221160 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.217589    2572 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 31 00:27:25.221160 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.217592    2572 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 31 00:27:25.221631 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.217594    2572 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 31 00:27:25.221631 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.217597    2572 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 31 00:27:25.221631 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.217599    2572 feature_gate.go:386] unrecognized feature gate: Example
Oct 31 00:27:25.221631 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.217602    2572 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 31 00:27:25.221631 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.217604    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 31 00:27:25.221631 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.217606    2572 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 31 00:27:25.221631 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.217609    2572 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 31 00:27:25.221631 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.217611    2572 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 31 00:27:25.221631 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.217614    2572 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 31 00:27:25.221631 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.217616    2572 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 31 00:27:25.221631 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.217619    2572 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 31 00:27:25.221631 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.217621    2572 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 31 00:27:25.221631 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.217624    2572 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 31 00:27:25.221631 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.217626    2572 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 31 00:27:25.221631 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.217629    2572 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 31 00:27:25.221631 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.217631    2572 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 31 00:27:25.221631 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.217633    2572 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 31 00:27:25.221631 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.217636    2572 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 31 00:27:25.221631 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.217638    2572 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 31 00:27:25.221631 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.217640    2572 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 31 00:27:25.222090 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.217643    2572 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 31 00:27:25.222090 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.217645    2572 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 31 00:27:25.222090 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.217648    2572 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 31 00:27:25.222090 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.217651    2572 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 31 00:27:25.222090 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.217654    2572 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 31 00:27:25.222090 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.217656    2572 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 31 00:27:25.222090 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.217660    2572 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 31 00:27:25.222090 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.217662    2572 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 31 00:27:25.222090 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.217664    2572 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 31 00:27:25.222090 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.217667    2572 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 31 00:27:25.222090 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.217670    2572 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 31 00:27:25.222090 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.217673    2572 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 31 00:27:25.222090 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.217675    2572 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 31 00:27:25.222090 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.217677    2572 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 31 00:27:25.222090 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.217680    2572 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 31 00:27:25.222090 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.217682    2572 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 31 00:27:25.222090 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.217684    2572 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 31 00:27:25.222090 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.217687    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 31 00:27:25.222090 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.217689    2572 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 31 00:27:25.222568 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.217691    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 31 00:27:25.222568 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.217694    2572 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 31 00:27:25.222568 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.217696    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 31 00:27:25.222568 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.217698    2572 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 31 00:27:25.222568 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.217701    2572 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 31 00:27:25.222568 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.217703    2572 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 31 00:27:25.222568 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.217705    2572 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 31 00:27:25.222568 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.217708    2572 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 31 00:27:25.222568 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.217710    2572 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 31 00:27:25.222568 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.217712    2572 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 31 00:27:25.222568 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.217715    2572 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 31 00:27:25.222568 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.217718    2572 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 31 00:27:25.222568 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.217722    2572 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 31 00:27:25.222568 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.217724    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 31 00:27:25.222568 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.217727    2572 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 31 00:27:25.222568 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.217730    2572 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 31 00:27:25.222568 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.217733    2572 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 31 00:27:25.222568 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.217736    2572 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 31 00:27:25.222568 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.217739    2572 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 31 00:27:25.222568 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.217741    2572 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 31 00:27:25.223040 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.217744    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 31 00:27:25.223040 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.217746    2572 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 31 00:27:25.223040 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.217749    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 31 00:27:25.223040 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.217752    2572 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 31 00:27:25.223040 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.217756    2572 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 31 00:27:25.223040 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.217760    2572 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 31 00:27:25.223040 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.217762    2572 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 31 00:27:25.223040 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.217765    2572 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 31 00:27:25.223040 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.217767    2572 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 31 00:27:25.223040 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.217770    2572 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 31 00:27:25.223040 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.217772    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 31 00:27:25.223040 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.217774    2572 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 31 00:27:25.223040 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.217777    2572 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 31 00:27:25.223040 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.217779    2572 feature_gate.go:386] unrecognized feature gate: Example2
Oct 31 00:27:25.223040 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.217781    2572 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 31 00:27:25.223040 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.217783    2572 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 31 00:27:25.223040 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.217786    2572 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 31 00:27:25.223040 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.217788    2572 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 31 00:27:25.223040 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.217791    2572 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 31 00:27:25.223512 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.217793    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 31 00:27:25.223512 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.217796    2572 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 31 00:27:25.223512 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.217798    2572 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 31 00:27:25.223512 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.217800    2572 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 31 00:27:25.223512 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.217803    2572 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 31 00:27:25.223512 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.217805    2572 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 31 00:27:25.223512 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.217808    2572 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 31 00:27:25.223512 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.217810    2572 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 31 00:27:25.223512 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.217812    2572 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 31 00:27:25.223512 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.217815    2572 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 31 00:27:25.223512 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.217818    2572 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 31 00:27:25.223512 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.217820    2572 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 31 00:27:25.223512 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.217823    2572 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 31 00:27:25.223512 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.217825    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 31 00:27:25.223512 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.217828    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 31 00:27:25.223512 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.217830    2572 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 31 00:27:25.223512 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.217834    2572 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 31 00:27:25.223512 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.217836    2572 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 31 00:27:25.223512 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.217839    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 31 00:27:25.223957 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.217841    2572 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 31 00:27:25.223957 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.217844    2572 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 31 00:27:25.223957 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.217846    2572 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 31 00:27:25.223957 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.217850    2572 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 31 00:27:25.223957 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.217852    2572 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 31 00:27:25.223957 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.218930    2572 flags.go:64] FLAG: --address="0.0.0.0"
Oct 31 00:27:25.223957 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.218940    2572 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Oct 31 00:27:25.223957 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.218945    2572 flags.go:64] FLAG: --anonymous-auth="true"
Oct 31 00:27:25.223957 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.218950    2572 flags.go:64] FLAG: --application-metrics-count-limit="100"
Oct 31 00:27:25.223957 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.218954    2572 flags.go:64] FLAG: --authentication-token-webhook="false"
Oct 31 00:27:25.223957 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.218958    2572 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Oct 31 00:27:25.223957 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.218962    2572 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Oct 31 00:27:25.223957 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.218966    2572 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Oct 31 00:27:25.223957 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.218969    2572 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Oct 31 00:27:25.223957 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.218972    2572 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Oct 31 00:27:25.223957 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.218976    2572 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Oct 31 00:27:25.223957 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.218979    2572 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Oct 31 00:27:25.223957 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.218982    2572 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Oct 31 00:27:25.223957 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.218985    2572 flags.go:64] FLAG: --cgroup-root=""
Oct 31 00:27:25.223957 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.218988    2572 flags.go:64] FLAG: --cgroups-per-qos="true"
Oct 31 00:27:25.223957 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.218991    2572 flags.go:64] FLAG: --client-ca-file=""
Oct 31 00:27:25.223957 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.218995    2572 flags.go:64] FLAG: --cloud-provider="external"
Oct 31 00:27:25.223957 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.218997    2572 flags.go:64] FLAG: --cluster-dns="[]"
Oct 31 00:27:25.224495 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.219002    2572 flags.go:64] FLAG: --cluster-domain=""
Oct 31 00:27:25.224495 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.219006    2572 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Oct 31 00:27:25.224495 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.219010    2572 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Oct 31 00:27:25.224495 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.219013    2572 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Oct 31 00:27:25.224495 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.219016    2572 flags.go:64] FLAG: --container-log-max-files="5"
Oct 31 00:27:25.224495 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.219020    2572 flags.go:64] FLAG: --container-log-max-size="10Mi"
Oct 31 00:27:25.224495 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.219025    2572 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Oct 31 00:27:25.224495 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.219028    2572 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Oct 31 00:27:25.224495 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.219031    2572 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Oct 31 00:27:25.224495 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.219034    2572 flags.go:64] FLAG: --contention-profiling="false"
Oct 31 00:27:25.224495 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.219037    2572 flags.go:64] FLAG: --cpu-cfs-quota="true"
Oct 31 00:27:25.224495 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.219040    2572 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Oct 31 00:27:25.224495 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.219043    2572 flags.go:64] FLAG: --cpu-manager-policy="none"
Oct 31 00:27:25.224495 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.219046    2572 flags.go:64] FLAG: --cpu-manager-policy-options=""
Oct 31 00:27:25.224495 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.219050    2572 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Oct 31 00:27:25.224495 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.219053    2572 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Oct 31 00:27:25.224495 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.219056    2572 flags.go:64] FLAG: --enable-debugging-handlers="true"
Oct 31 00:27:25.224495 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.219059    2572 flags.go:64] FLAG: --enable-load-reader="false"
Oct 31 00:27:25.224495 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.219062    2572 flags.go:64] FLAG: --enable-server="true"
Oct 31 00:27:25.224495 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.219065    2572 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Oct 31 00:27:25.224495 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.219068    2572 flags.go:64] FLAG: --event-burst="100"
Oct 31 00:27:25.224495 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.219071    2572 flags.go:64] FLAG: --event-qps="50"
Oct 31 00:27:25.224495 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.219074    2572 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Oct 31 00:27:25.224495 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.219077    2572 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Oct 31 00:27:25.225089 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.219080    2572 flags.go:64] FLAG: --eviction-hard=""
Oct 31 00:27:25.225089 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.219086    2572 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Oct 31 00:27:25.225089 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.219088    2572 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Oct 31 00:27:25.225089 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.219091    2572 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Oct 31 00:27:25.225089 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.219095    2572 flags.go:64] FLAG: --eviction-soft=""
Oct 31 00:27:25.225089 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.219098    2572 flags.go:64] FLAG: --eviction-soft-grace-period=""
Oct 31 00:27:25.225089 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.219100    2572 flags.go:64] FLAG: --exit-on-lock-contention="false"
Oct 31 00:27:25.225089 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.219103    2572 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Oct 31 00:27:25.225089 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.219106    2572 flags.go:64] FLAG: --experimental-mounter-path=""
Oct 31 00:27:25.225089 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.219109    2572 flags.go:64] FLAG: --fail-cgroupv1="true"
Oct 31 00:27:25.225089 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.219111    2572 flags.go:64] FLAG: --fail-swap-on="true"
Oct 31 00:27:25.225089 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.219115    2572 flags.go:64] FLAG: --feature-gates=""
Oct 31 00:27:25.225089 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.219119    2572 flags.go:64] FLAG: --file-check-frequency="20s"
Oct 31 00:27:25.225089 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.219122    2572 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Oct 31 00:27:25.225089 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.219126    2572 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Oct 31 00:27:25.225089 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.219129    2572 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Oct 31 00:27:25.225089 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.219132    2572 flags.go:64] FLAG: --healthz-port="10248"
Oct 31 00:27:25.225089 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.219135    2572 flags.go:64] FLAG: --help="false"
Oct 31 00:27:25.225089 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.219138    2572 flags.go:64] FLAG: --hostname-override="ip-10-0-132-173.us-west-2.compute.internal"
Oct 31 00:27:25.225089 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.219142    2572 flags.go:64] FLAG: --housekeeping-interval="10s"
Oct 31 00:27:25.225089 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.219145    2572 flags.go:64] FLAG: --http-check-frequency="20s"
Oct 31 00:27:25.225089 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.219148    2572 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Oct 31 00:27:25.225089 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.219151    2572 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Oct 31 00:27:25.225692 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.219154    2572 flags.go:64] FLAG: --image-gc-high-threshold="85"
Oct 31 00:27:25.225692 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.219158    2572 flags.go:64] FLAG: --image-gc-low-threshold="80"
Oct 31 00:27:25.225692 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.219160    2572 flags.go:64] FLAG: --image-service-endpoint=""
Oct 31 00:27:25.225692 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.219163    2572 flags.go:64] FLAG: --kernel-memcg-notification="false"
Oct 31 00:27:25.225692 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.219166    2572 flags.go:64] FLAG: --kube-api-burst="100"
Oct 31 00:27:25.225692 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.219169    2572 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Oct 31 00:27:25.225692 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.219172    2572 flags.go:64] FLAG: --kube-api-qps="50"
Oct 31 00:27:25.225692 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.219175    2572 flags.go:64] FLAG: --kube-reserved=""
Oct 31 00:27:25.225692 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.219178    2572 flags.go:64] FLAG: --kube-reserved-cgroup=""
Oct 31 00:27:25.225692 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.219181    2572 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Oct 31 00:27:25.225692 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.219186    2572 flags.go:64] FLAG: --kubelet-cgroups=""
Oct 31 00:27:25.225692 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.219189    2572 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Oct 31 00:27:25.225692 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.219192    2572 flags.go:64] FLAG: --lock-file=""
Oct 31 00:27:25.225692 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.219195    2572 flags.go:64] FLAG: --log-cadvisor-usage="false"
Oct 31 00:27:25.225692 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.219198    2572 flags.go:64] FLAG: --log-flush-frequency="5s"
Oct 31 00:27:25.225692 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.219201    2572 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Oct 31 00:27:25.225692 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.219207    2572 flags.go:64] FLAG: --log-json-split-stream="false"
Oct 31 00:27:25.225692 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.219221    2572 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Oct 31 00:27:25.225692 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.219224    2572 flags.go:64] FLAG: --log-text-split-stream="false"
Oct 31 00:27:25.225692 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.219227    2572 flags.go:64] FLAG: --logging-format="text"
Oct 31 00:27:25.225692 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.219230    2572 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Oct 31 00:27:25.225692 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.219234    2572 flags.go:64] FLAG: --make-iptables-util-chains="true"
Oct 31 00:27:25.225692 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.219237    2572 flags.go:64] FLAG: --manifest-url=""
Oct 31 00:27:25.225692 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.219240    2572 flags.go:64] FLAG: --manifest-url-header=""
Oct 31 00:27:25.225692 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.219245    2572 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Oct 31 00:27:25.226636 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.219248    2572 flags.go:64] FLAG: --max-open-files="1000000"
Oct 31 00:27:25.226636 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.219255    2572 flags.go:64] FLAG: --max-pods="110"
Oct 31 00:27:25.226636 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.219258    2572 flags.go:64] FLAG: --maximum-dead-containers="-1"
Oct 31 00:27:25.226636 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.219261    2572 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Oct 31 00:27:25.226636 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.219264    2572 flags.go:64] FLAG: --memory-manager-policy="None"
Oct 31 00:27:25.226636 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.219267    2572 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Oct 31 00:27:25.226636 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.219269    2572 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Oct 31 00:27:25.226636 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.219273    2572 flags.go:64] FLAG: --node-ip="0.0.0.0"
Oct 31 00:27:25.226636 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.219276    2572 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Oct 31 00:27:25.226636 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.219283    2572 flags.go:64] FLAG: --node-status-max-images="50"
Oct 31 00:27:25.226636 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.219286    2572 flags.go:64] FLAG: --node-status-update-frequency="10s"
Oct 31 00:27:25.226636 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.219289    2572 flags.go:64] FLAG: --oom-score-adj="-999"
Oct 31 00:27:25.226636 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.219292    2572 flags.go:64] FLAG: --pod-cidr=""
Oct 31 00:27:25.226636 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.219295    2572 flags.go:64] FLAG: --pod-manifest-path=""
Oct 31 00:27:25.226636 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.219298    2572 flags.go:64] FLAG: --pod-max-pids="-1"
Oct 31 00:27:25.226636 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.219301    2572 flags.go:64] FLAG: --pods-per-core="0"
Oct 31 00:27:25.226636 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.219304    2572 flags.go:64] FLAG: --port="10250"
Oct 31 00:27:25.226636 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.219308    2572 flags.go:64] FLAG: --protect-kernel-defaults="false"
Oct 31 00:27:25.226636 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.219311    2572 flags.go:64] FLAG: --provider-id="aws:///us-west-2a/i-00e6ca61ed28a7c40"
Oct 31 00:27:25.226636 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.219314    2572 flags.go:64] FLAG: --qos-reserved=""
Oct 31 00:27:25.226636 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.219317    2572 flags.go:64] FLAG: --read-only-port="10255"
Oct 31 00:27:25.226636 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.219320    2572 flags.go:64] FLAG: --register-node="true"
Oct 31 00:27:25.226636 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.219322    2572 flags.go:64] FLAG: --register-with-taints=""
Oct 31 00:27:25.226636 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.219326    2572 flags.go:64] FLAG: --registry-burst="10"
Oct 31 00:27:25.226636 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.219329    2572 flags.go:64] FLAG: --registry-qps="5"
Oct 31 00:27:25.226636 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.219332    2572 flags.go:64] FLAG: --reserved-cpus=""
Oct 31 00:27:25.227428 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.219334    2572 flags.go:64] FLAG: --reserved-memory=""
Oct 31 00:27:25.227428 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.219339    2572 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Oct 31 00:27:25.227428 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.219342    2572 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Oct 31 00:27:25.227428 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.219345    2572 flags.go:64] FLAG: --rotate-certificates="false"
Oct 31 00:27:25.227428 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.219348    2572 flags.go:64] FLAG: --rotate-server-certificates="false"
Oct 31 00:27:25.227428 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.219351    2572 flags.go:64] FLAG: --runonce="false"
Oct 31 00:27:25.227428 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.219354    2572 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Oct 31 00:27:25.227428 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.219357    2572 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Oct 31 00:27:25.227428 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.219360    2572 flags.go:64] FLAG: --seccomp-default="false"
Oct 31 00:27:25.227428 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.219363    2572 flags.go:64] FLAG: --serialize-image-pulls="true"
Oct 31 00:27:25.227428 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.219365    2572 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Oct 31 00:27:25.227428 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.219368    2572 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Oct 31 00:27:25.227428 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.219371    2572 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Oct 31 00:27:25.227428 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.219374    2572 flags.go:64] FLAG: --storage-driver-password="root"
Oct 31 00:27:25.227428 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.219377    2572 flags.go:64] FLAG: --storage-driver-secure="false"
Oct 31 00:27:25.227428 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.219380    2572 flags.go:64] FLAG: --storage-driver-table="stats"
Oct 31 00:27:25.227428 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.219383    2572 flags.go:64] FLAG: --storage-driver-user="root"
Oct 31 00:27:25.227428 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.219386    2572 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Oct 31 00:27:25.227428 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.219389    2572 flags.go:64] FLAG: --sync-frequency="1m0s"
Oct 31 00:27:25.227428 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.219392    2572 flags.go:64] FLAG: --system-cgroups=""
Oct 31 00:27:25.227428 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.219395    2572 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Oct 31 00:27:25.227428 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.219400    2572 flags.go:64] FLAG: --system-reserved-cgroup=""
Oct 31 00:27:25.227428 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.219403    2572 flags.go:64] FLAG: --tls-cert-file=""
Oct 31 00:27:25.227428 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.219407    2572 flags.go:64] FLAG: --tls-cipher-suites="[]"
Oct 31 00:27:25.227428 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.219423    2572 flags.go:64] FLAG: --tls-min-version=""
Oct 31 00:27:25.228124 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.219426    2572 flags.go:64] FLAG: --tls-private-key-file=""
Oct 31 00:27:25.228124 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.219429    2572 flags.go:64] FLAG: --topology-manager-policy="none"
Oct 31 00:27:25.228124 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.219432    2572 flags.go:64] FLAG: --topology-manager-policy-options=""
Oct 31 00:27:25.228124 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.219434    2572 flags.go:64] FLAG: --topology-manager-scope="container"
Oct 31 00:27:25.228124 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.219437    2572 flags.go:64] FLAG: --v="2"
Oct 31 00:27:25.228124 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.219442    2572 flags.go:64] FLAG: --version="false"
Oct 31 00:27:25.228124 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.219446    2572 flags.go:64] FLAG: --vmodule=""
Oct 31 00:27:25.228124 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.219449    2572 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Oct 31 00:27:25.228124 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.219452    2572 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Oct 31 00:27:25.228124 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.219607    2572 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 31 00:27:25.228124 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.219611    2572 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 31 00:27:25.228124 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.219616    2572 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 31 00:27:25.228124 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.219619    2572 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 31 00:27:25.228124 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.219621    2572 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 31 00:27:25.228124 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.219624    2572 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 31 00:27:25.228124 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.219626    2572 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 31 00:27:25.228124 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.219629    2572 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 31 00:27:25.228124 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.219632    2572 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 31 00:27:25.228124 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.219634    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 31 00:27:25.228124 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.219636    2572 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 31 00:27:25.228124 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.219639    2572 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 31 00:27:25.228124 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.219641    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 31 00:27:25.228676 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.219643    2572 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 31 00:27:25.228676 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.219647    2572 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 31 00:27:25.228676 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.219651    2572 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 31 00:27:25.228676 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.219654    2572 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 31 00:27:25.228676 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.219657    2572 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 31 00:27:25.228676 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.219660    2572 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 31 00:27:25.228676 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.219663    2572 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 31 00:27:25.228676 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.219666    2572 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 31 00:27:25.228676 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.219669    2572 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 31 00:27:25.228676 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.219672    2572 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 31 00:27:25.228676 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.219674    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 31 00:27:25.228676 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.219677    2572 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 31 00:27:25.228676 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.219680    2572 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 31 00:27:25.228676 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.219682    2572 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 31 00:27:25.228676 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.219685    2572 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 31 00:27:25.228676 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.219687    2572 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 31 00:27:25.228676 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.219689    2572 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 31 00:27:25.228676 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.219691    2572 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 31 00:27:25.228676 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.219694    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 31 00:27:25.229110 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.219696    2572 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 31 00:27:25.229110 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.219698    2572 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 31 00:27:25.229110 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.219702    2572 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 31 00:27:25.229110 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.219705    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 31 00:27:25.229110 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.219707    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 31 00:27:25.229110 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.219710    2572 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 31 00:27:25.229110 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.219712    2572 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 31 00:27:25.229110 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.219714    2572 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 31 00:27:25.229110 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.219717    2572 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 31 00:27:25.229110 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.219719    2572 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 31 00:27:25.229110 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.219722    2572 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 31 00:27:25.229110 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.219724    2572 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 31 00:27:25.229110 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.219726    2572 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 31 00:27:25.229110 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.219729    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 31 00:27:25.229110 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.219731    2572 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 31 00:27:25.229110 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.219734    2572 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 31 00:27:25.229110 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.219736    2572 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 31 00:27:25.229110 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.219738    2572 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 31 00:27:25.229110 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.219741    2572 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 31 00:27:25.229110 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.219743    2572 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 31 00:27:25.229591 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.219745    2572 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 31 00:27:25.229591 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.219750    2572 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 31 00:27:25.229591 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.219752    2572 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 31 00:27:25.229591 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.219756    2572 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 31 00:27:25.229591 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.219760    2572 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 31 00:27:25.229591 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.219762    2572 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 31 00:27:25.229591 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.219765    2572 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 31 00:27:25.229591 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.219768    2572 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 31 00:27:25.229591 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.219770    2572 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 31 00:27:25.229591 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.219772    2572 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 31 00:27:25.229591 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.219775    2572 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 31 00:27:25.229591 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.219777    2572 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 31 00:27:25.229591 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.219779    2572 feature_gate.go:386] unrecognized feature gate: Example2
Oct 31 00:27:25.229591 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.219782    2572 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 31 00:27:25.229591 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.219785    2572 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 31 00:27:25.229591 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.219788    2572 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 31 00:27:25.229591 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.219791    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 31 00:27:25.229591 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.219794    2572 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 31 00:27:25.229591 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.219796    2572 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 31 00:27:25.230061 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.219799    2572 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 31 00:27:25.230061 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.219801    2572 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 31 00:27:25.230061 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.219803    2572 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 31 00:27:25.230061 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.219806    2572 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 31 00:27:25.230061 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.219808    2572 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 31 00:27:25.230061 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.219810    2572 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 31 00:27:25.230061 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.219813    2572 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 31 00:27:25.230061 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.219815    2572 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 31 00:27:25.230061 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.219818    2572 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 31 00:27:25.230061 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.219820    2572 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 31 00:27:25.230061 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.219822    2572 feature_gate.go:386] unrecognized feature gate: Example
Oct 31 00:27:25.230061 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.219824    2572 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 31 00:27:25.230061 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.219827    2572 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 31 00:27:25.230061 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.219829    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 31 00:27:25.230061 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.219833    2572 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 31 00:27:25.230061 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.219835    2572 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 31 00:27:25.230061 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.219838    2572 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 31 00:27:25.230061 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.219840    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 31 00:27:25.230061 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.219843    2572 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 31 00:27:25.230550 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.219845    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 31 00:27:25.230550 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.219847    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 31 00:27:25.230550 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.219850    2572 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 31 00:27:25.230550 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.219852    2572 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 31 00:27:25.230550 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.219855    2572 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 31 00:27:25.230550 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.219857    2572 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 31 00:27:25.230550 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.219860    2572 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 31 00:27:25.230550 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.219862    2572 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 31 00:27:25.230550 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.219865    2572 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 31 00:27:25.230550 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.219868    2572 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 31 00:27:25.230550 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.219870    2572 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 31 00:27:25.230550 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.219873    2572 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 31 00:27:25.230550 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.219876    2572 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 31 00:27:25.230550 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.219878    2572 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 31 00:27:25.230550 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.219880    2572 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 31 00:27:25.230550 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.219883    2572 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 31 00:27:25.230929 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.219907    2572 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 31 00:27:25.230929 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.219942    2572 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 31 00:27:25.230929 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.219947    2572 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 31 00:27:25.230929 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.219949    2572 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 31 00:27:25.230929 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.219953    2572 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 31 00:27:25.230929 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.219955    2572 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 31 00:27:25.230929 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.219958    2572 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 31 00:27:25.230929 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.219961    2572 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 31 00:27:25.230929 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.219963    2572 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 31 00:27:25.230929 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.219965    2572 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 31 00:27:25.230929 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.219968    2572 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 31 00:27:25.230929 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.219970    2572 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 31 00:27:25.230929 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.219974    2572 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 31 00:27:25.230929 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.219976    2572 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 31 00:27:25.230929 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.219979    2572 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 31 00:27:25.230929 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.219981    2572 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 31 00:27:25.231327 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.219984    2572 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 31 00:27:25.231327 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.219986    2572 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 31 00:27:25.231327 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.219989    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 31 00:27:25.231327 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.219991    2572 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 31 00:27:25.231327 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.219994    2572 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 31 00:27:25.231327 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.219996    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 31 00:27:25.231327 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.219998    2572 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 31 00:27:25.231327 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.220002    2572 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 31 00:27:25.231327 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.220006    2572 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 31 00:27:25.231327 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.220010    2572 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 31 00:27:25.231327 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.220012    2572 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 31 00:27:25.231327 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.220015    2572 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 31 00:27:25.231327 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.220018    2572 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 31 00:27:25.231327 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.220020    2572 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 31 00:27:25.231327 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.220023    2572 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 31 00:27:25.231327 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.220025    2572 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 31 00:27:25.231327 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.220028    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 31 00:27:25.231327 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.220030    2572 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 31 00:27:25.231327 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.220032    2572 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 31 00:27:25.231773 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.220035    2572 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 31 00:27:25.231773 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.220037    2572 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 31 00:27:25.231773 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.220039    2572 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 31 00:27:25.231773 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.220042    2572 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 31 00:27:25.231773 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.220044    2572 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 31 00:27:25.231773 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.220046    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 31 00:27:25.231773 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.220049    2572 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 31 00:27:25.231773 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.220051    2572 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 31 00:27:25.231773 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.220053    2572 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 31 00:27:25.231773 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.220059    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 31 00:27:25.231773 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.220061    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 31 00:27:25.231773 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.220064    2572 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 31 00:27:25.231773 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.220066    2572 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 31 00:27:25.231773 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.220069    2572 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 31 00:27:25.231773 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.220071    2572 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 31 00:27:25.231773 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.220074    2572 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 31 00:27:25.231773 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.220076    2572 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 31 00:27:25.231773 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.220078    2572 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 31 00:27:25.231773 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.220081    2572 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 31 00:27:25.232205 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.220083    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 31 00:27:25.232205 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.220086    2572 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 31 00:27:25.232205 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.220089    2572 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 31 00:27:25.232205 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.220092    2572 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 31 00:27:25.232205 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.220094    2572 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 31 00:27:25.232205 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.220097    2572 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 31 00:27:25.232205 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.220099    2572 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 31 00:27:25.232205 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.220102    2572 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 31 00:27:25.232205 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.220104    2572 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 31 00:27:25.232205 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.220106    2572 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 31 00:27:25.232205 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.220111    2572 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 31 00:27:25.232205 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.220114    2572 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 31 00:27:25.232205 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.220117    2572 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 31 00:27:25.232205 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.220120    2572 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 31 00:27:25.232205 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.220122    2572 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 31 00:27:25.232205 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.220125    2572 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 31 00:27:25.232205 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.220127    2572 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 31 00:27:25.232205 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.220129    2572 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 31 00:27:25.232205 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.220131    2572 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 31 00:27:25.232205 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.220134    2572 feature_gate.go:386] unrecognized feature gate: Example2
Oct 31 00:27:25.232690 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.220136    2572 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 31 00:27:25.232690 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.220138    2572 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 31 00:27:25.232690 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.220142    2572 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 31 00:27:25.232690 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.220145    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 31 00:27:25.232690 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.220147    2572 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 31 00:27:25.232690 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.220150    2572 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 31 00:27:25.232690 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.220153    2572 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 31 00:27:25.232690 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.220155    2572 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 31 00:27:25.232690 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.220158    2572 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 31 00:27:25.232690 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.220160    2572 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 31 00:27:25.232690 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.220162    2572 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 31 00:27:25.232690 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.220165    2572 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 31 00:27:25.232690 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.220167    2572 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 31 00:27:25.232690 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.220169    2572 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 31 00:27:25.232690 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.220173    2572 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 31 00:27:25.232690 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.220176    2572 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 31 00:27:25.232690 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.220178    2572 feature_gate.go:386] unrecognized feature gate: Example
Oct 31 00:27:25.232690 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.220181    2572 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 31 00:27:25.232690 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.220183    2572 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 31 00:27:25.233137 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.220185    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 31 00:27:25.233137 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.220188    2572 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 31 00:27:25.233137 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.220191    2572 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 31 00:27:25.233137 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.220193    2572 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 31 00:27:25.233137 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.220195    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 31 00:27:25.233137 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.220198    2572 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 31 00:27:25.233137 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.220200    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 31 00:27:25.233137 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.220202    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 31 00:27:25.233137 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.220205    2572 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 31 00:27:25.233137 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.220219    2572 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 31 00:27:25.233137 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.220222    2572 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 31 00:27:25.233137 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.220225    2572 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 31 00:27:25.233137 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.220227    2572 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 31 00:27:25.233137 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.220231    2572 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 31 00:27:25.233137 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.229016    2572 server.go:525] "Kubelet version" kubeletVersion="v1.35.9"
Oct 31 00:27:25.233137 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.229033    2572 server.go:527] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Oct 31 00:27:25.233137 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.229711    2572 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Oct 31 00:27:25.233137 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.229717    2572 watchdog_linux.go:138] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Oct 31 00:27:25.233137 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.229774    2572 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 31 00:27:25.233137 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.229779    2572 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 31 00:27:25.233658 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.229782    2572 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 31 00:27:25.233658 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.229785    2572 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 31 00:27:25.233658 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.229787    2572 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 31 00:27:25.233658 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.229790    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 31 00:27:25.233658 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.229793    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 31 00:27:25.233658 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.229795    2572 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 31 00:27:25.233658 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.229798    2572 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 31 00:27:25.233658 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.229801    2572 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 31 00:27:25.233658 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.229803    2572 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 31 00:27:25.233658 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.229806    2572 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 31 00:27:25.233658 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.229810    2572 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 31 00:27:25.233658 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.229813    2572 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 31 00:27:25.233658 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.229816    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 31 00:27:25.233658 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.229820    2572 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 31 00:27:25.233658 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.229822    2572 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 31 00:27:25.233658 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.229825    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 31 00:27:25.233658 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.229827    2572 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 31 00:27:25.233658 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.229830    2572 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 31 00:27:25.233658 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.229832    2572 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 31 00:27:25.233658 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.229835    2572 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 31 00:27:25.234110 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.229838    2572 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 31 00:27:25.234110 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.229840    2572 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 31 00:27:25.234110 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.229843    2572 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 31 00:27:25.234110 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.229846    2572 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 31 00:27:25.234110 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.229848    2572 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 31 00:27:25.234110 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.229852    2572 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 31 00:27:25.234110 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.229855    2572 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 31 00:27:25.234110 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.229859    2572 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 31 00:27:25.234110 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.229862    2572 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 31 00:27:25.234110 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.229864    2572 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 31 00:27:25.234110 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.229867    2572 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 31 00:27:25.234110 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.229870    2572 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 31 00:27:25.234110 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.229872    2572 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 31 00:27:25.234110 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.229875    2572 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 31 00:27:25.234110 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.229877    2572 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 31 00:27:25.234110 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.229880    2572 feature_gate.go:386] unrecognized feature gate: Example
Oct 31 00:27:25.234110 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.229882    2572 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 31 00:27:25.234110 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.229885    2572 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 31 00:27:25.234110 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.229887    2572 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 31 00:27:25.234110 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.229890    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 31 00:27:25.234760 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.229892    2572 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 31 00:27:25.234760 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.229895    2572 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 31 00:27:25.234760 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.229897    2572 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 31 00:27:25.234760 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.229900    2572 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 31 00:27:25.234760 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.229902    2572 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 31 00:27:25.234760 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.229905    2572 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 31 00:27:25.234760 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.229907    2572 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 31 00:27:25.234760 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.229910    2572 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 31 00:27:25.234760 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.229912    2572 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 31 00:27:25.234760 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.229915    2572 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 31 00:27:25.234760 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.229917    2572 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 31 00:27:25.234760 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.229919    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 31 00:27:25.234760 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.229922    2572 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 31 00:27:25.234760 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.229924    2572 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 31 00:27:25.234760 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.229927    2572 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 31 00:27:25.234760 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.229930    2572 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 31 00:27:25.234760 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.229933    2572 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 31 00:27:25.234760 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.229935    2572 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 31 00:27:25.234760 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.229938    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 31 00:27:25.235196 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.229941    2572 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 31 00:27:25.235196 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.229944    2572 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 31 00:27:25.235196 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.229946    2572 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 31 00:27:25.235196 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.229949    2572 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 31 00:27:25.235196 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.229951    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 31 00:27:25.235196 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.229954    2572 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 31 00:27:25.235196 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.229956    2572 feature_gate.go:386] unrecognized feature gate: Example2
Oct 31 00:27:25.235196 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.229959    2572 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 31 00:27:25.235196 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.229961    2572 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 31 00:27:25.235196 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.229963    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 31 00:27:25.235196 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.229966    2572 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 31 00:27:25.235196 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.229968    2572 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 31 00:27:25.235196 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.229970    2572 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 31 00:27:25.235196 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.229973    2572 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 31 00:27:25.235196 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.229975    2572 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 31 00:27:25.235196 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.229978    2572 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 31 00:27:25.235196 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.229980    2572 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 31 00:27:25.235196 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.229982    2572 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 31 00:27:25.235196 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.229986    2572 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 31 00:27:25.235196 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.229988    2572 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 31 00:27:25.235713 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.229990    2572 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 31 00:27:25.235713 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.229993    2572 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 31 00:27:25.235713 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.229995    2572 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 31 00:27:25.235713 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.229997    2572 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 31 00:27:25.235713 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.230000    2572 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 31 00:27:25.235713 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.230002    2572 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 31 00:27:25.235713 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.230005    2572 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 31 00:27:25.235713 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.230007    2572 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 31 00:27:25.235713 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.230010    2572 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 31 00:27:25.235713 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.230012    2572 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 31 00:27:25.235713 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.230015    2572 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 31 00:27:25.235713 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.230017    2572 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 31 00:27:25.235713 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.230019    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 31 00:27:25.235713 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.230022    2572 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 31 00:27:25.235713 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.230025    2572 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 31 00:27:25.235713 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.230027    2572 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 31 00:27:25.235713 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.230031    2572 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 31 00:27:25.235713 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.230034    2572 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 31 00:27:25.235713 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.230037    2572 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 31 00:27:25.236138 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.230041    2572 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 31 00:27:25.236138 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.230043    2572 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 31 00:27:25.236138 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.230045    2572 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 31 00:27:25.236138 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.230048    2572 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 31 00:27:25.236138 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.230050    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 31 00:27:25.236138 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.230053    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 31 00:27:25.236138 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.230078    2572 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 31 00:27:25.236138 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.230229    2572 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 31 00:27:25.236138 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.230234    2572 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 31 00:27:25.236138 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.230237    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 31 00:27:25.236138 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.230239    2572 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 31 00:27:25.236138 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.230242    2572 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 31 00:27:25.236138 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.230247    2572 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 31 00:27:25.236138 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.230250    2572 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 31 00:27:25.236138 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.230253    2572 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 31 00:27:25.236138 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.230255    2572 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 31 00:27:25.236526 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.230258    2572 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 31 00:27:25.236526 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.230260    2572 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 31 00:27:25.236526 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.230263    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 31 00:27:25.236526 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.230265    2572 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 31 00:27:25.236526 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.230268    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 31 00:27:25.236526 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.230276    2572 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 31 00:27:25.236526 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.230279    2572 feature_gate.go:386] unrecognized feature gate: Example2
Oct 31 00:27:25.236526 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.230282    2572 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 31 00:27:25.236526 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.230285    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 31 00:27:25.236526 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.230287    2572 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 31 00:27:25.236526 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.230290    2572 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 31 00:27:25.236526 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.230293    2572 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 31 00:27:25.236526 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.230295    2572 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 31 00:27:25.236526 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.230298    2572 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 31 00:27:25.236526 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.230300    2572 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 31 00:27:25.236526 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.230303    2572 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 31 00:27:25.236526 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.230306    2572 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 31 00:27:25.236526 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.230328    2572 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 31 00:27:25.236526 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.230330    2572 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 31 00:27:25.236526 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.230333    2572 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 31 00:27:25.236973 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.230335    2572 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 31 00:27:25.236973 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.230338    2572 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 31 00:27:25.236973 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.230340    2572 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 31 00:27:25.236973 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.230342    2572 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 31 00:27:25.236973 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.230345    2572 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 31 00:27:25.236973 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.230347    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 31 00:27:25.236973 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.230350    2572 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 31 00:27:25.236973 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.230352    2572 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 31 00:27:25.236973 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.230355    2572 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 31 00:27:25.236973 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.230358    2572 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 31 00:27:25.236973 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.230361    2572 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 31 00:27:25.236973 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.230363    2572 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 31 00:27:25.236973 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.230366    2572 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 31 00:27:25.236973 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.230368    2572 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 31 00:27:25.236973 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.230371    2572 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 31 00:27:25.236973 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.230373    2572 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 31 00:27:25.236973 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.230376    2572 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 31 00:27:25.236973 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.230379    2572 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 31 00:27:25.236973 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.230381    2572 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 31 00:27:25.236973 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.230384    2572 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 31 00:27:25.237453 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.230386    2572 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 31 00:27:25.237453 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.230389    2572 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 31 00:27:25.237453 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.230391    2572 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 31 00:27:25.237453 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.230394    2572 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 31 00:27:25.237453 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.230396    2572 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 31 00:27:25.237453 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.230399    2572 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 31 00:27:25.237453 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.230401    2572 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 31 00:27:25.237453 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.230404    2572 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 31 00:27:25.237453 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.230406    2572 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 31 00:27:25.237453 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.230408    2572 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 31 00:27:25.237453 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.230411    2572 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 31 00:27:25.237453 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.230413    2572 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 31 00:27:25.237453 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.230416    2572 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 31 00:27:25.237453 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.230418    2572 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 31 00:27:25.237453 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.230422    2572 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 31 00:27:25.237453 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.230425    2572 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 31 00:27:25.237453 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.230428    2572 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 31 00:27:25.237453 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.230430    2572 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 31 00:27:25.237453 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.230433    2572 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 31 00:27:25.237453 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.230436    2572 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 31 00:27:25.237911 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.230438    2572 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 31 00:27:25.237911 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.230441    2572 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 31 00:27:25.237911 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.230443    2572 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 31 00:27:25.237911 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.230446    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 31 00:27:25.237911 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.230448    2572 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 31 00:27:25.237911 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.230451    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 31 00:27:25.237911 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.230453    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 31 00:27:25.237911 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.230456    2572 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 31 00:27:25.237911 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.230458    2572 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 31 00:27:25.237911 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.230461    2572 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 31 00:27:25.237911 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.230463    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 31 00:27:25.237911 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.230466    2572 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 31 00:27:25.237911 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.230468    2572 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 31 00:27:25.237911 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.230473    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 31 00:27:25.237911 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.230475    2572 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 31 00:27:25.237911 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.230478    2572 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 31 00:27:25.237911 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.230482    2572 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 31 00:27:25.237911 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.230486    2572 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 31 00:27:25.238408 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.230489    2572 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 31 00:27:25.238408 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.230492    2572 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 31 00:27:25.238408 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.230494    2572 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 31 00:27:25.238408 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.230497    2572 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 31 00:27:25.238408 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.230499    2572 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 31 00:27:25.238408 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.230501    2572 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 31 00:27:25.238408 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.230504    2572 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 31 00:27:25.238408 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.230506    2572 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 31 00:27:25.238408 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.230508    2572 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 31 00:27:25.238408 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.230511    2572 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 31 00:27:25.238408 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.230513    2572 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 31 00:27:25.238408 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.230516    2572 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 31 00:27:25.238408 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.230518    2572 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 31 00:27:25.238408 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.230520    2572 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 31 00:27:25.238408 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.230523    2572 feature_gate.go:386] unrecognized feature gate: Example
Oct 31 00:27:25.238408 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.230525    2572 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 31 00:27:25.238408 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.230528    2572 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 31 00:27:25.238408 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.230530    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 31 00:27:25.238408 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.230533    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 31 00:27:25.238839 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.230556    2572 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 31 00:27:25.238839 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.230588    2572 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 31 00:27:25.238839 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.230592    2572 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 31 00:27:25.238839 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.230595    2572 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 31 00:27:25.238839 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.230597    2572 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 31 00:27:25.238839 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.230601    2572 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 31 00:27:25.238839 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.230603    2572 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 31 00:27:25.238839 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.230606    2572 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 31 00:27:25.238839 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.230610    2572 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 31 00:27:25.238839 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.230613    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 31 00:27:25.238839 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.230615    2572 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 31 00:27:25.238839 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.230618    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 31 00:27:25.238839 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.230620    2572 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 31 00:27:25.238839 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.230623    2572 feature_gate.go:386] unrecognized feature gate: Example2
Oct 31 00:27:25.238839 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.230625    2572 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 31 00:27:25.238839 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.230628    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 31 00:27:25.239241 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.230630    2572 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 31 00:27:25.239241 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.230633    2572 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 31 00:27:25.239241 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.230635    2572 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 31 00:27:25.239241 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.230638    2572 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 31 00:27:25.239241 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.230640    2572 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 31 00:27:25.239241 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.230643    2572 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 31 00:27:25.239241 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.230645    2572 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 31 00:27:25.239241 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.230648    2572 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 31 00:27:25.239241 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.230650    2572 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 31 00:27:25.239241 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.230652    2572 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 31 00:27:25.239241 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.230655    2572 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 31 00:27:25.239241 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.230657    2572 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 31 00:27:25.239241 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.230660    2572 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 31 00:27:25.239241 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.230662    2572 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 31 00:27:25.239241 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.230664    2572 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 31 00:27:25.239241 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.230667    2572 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 31 00:27:25.239241 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.230670    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 31 00:27:25.239241 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.230672    2572 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 31 00:27:25.239241 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.230675    2572 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 31 00:27:25.239241 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.230677    2572 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 31 00:27:25.239709 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.230679    2572 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 31 00:27:25.239709 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.230682    2572 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 31 00:27:25.239709 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.230684    2572 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 31 00:27:25.239709 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.230687    2572 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 31 00:27:25.239709 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.230693    2572 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 31 00:27:25.239709 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.230696    2572 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 31 00:27:25.239709 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.230699    2572 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 31 00:27:25.239709 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.230701    2572 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 31 00:27:25.239709 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.230704    2572 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 31 00:27:25.239709 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.230707    2572 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 31 00:27:25.239709 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.230709    2572 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 31 00:27:25.239709 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.230712    2572 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 31 00:27:25.239709 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.230714    2572 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 31 00:27:25.239709 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.230716    2572 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 31 00:27:25.239709 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.230719    2572 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 31 00:27:25.239709 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.230721    2572 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 31 00:27:25.239709 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.230723    2572 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 31 00:27:25.239709 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.230726    2572 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 31 00:27:25.239709 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.230728    2572 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 31 00:27:25.239709 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.230731    2572 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 31 00:27:25.240159 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.230733    2572 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 31 00:27:25.240159 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.230735    2572 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 31 00:27:25.240159 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.230738    2572 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 31 00:27:25.240159 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.230740    2572 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 31 00:27:25.240159 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.230743    2572 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 31 00:27:25.240159 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.230746    2572 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 31 00:27:25.240159 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.230750    2572 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 31 00:27:25.240159 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.230753    2572 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 31 00:27:25.240159 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.230756    2572 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 31 00:27:25.240159 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.230759    2572 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 31 00:27:25.240159 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.230762    2572 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 31 00:27:25.240159 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.230764    2572 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 31 00:27:25.240159 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.230767    2572 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 31 00:27:25.240159 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.230769    2572 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 31 00:27:25.240159 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.230772    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 31 00:27:25.240159 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.230774    2572 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 31 00:27:25.240159 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.230777    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 31 00:27:25.240159 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.230780    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 31 00:27:25.240159 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.230782    2572 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 31 00:27:25.240159 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.230785    2572 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 31 00:27:25.240629 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.230787    2572 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 31 00:27:25.240629 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.230790    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 31 00:27:25.240629 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.230792    2572 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 31 00:27:25.240629 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.230794    2572 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 31 00:27:25.240629 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.230797    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 31 00:27:25.240629 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.230799    2572 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 31 00:27:25.240629 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.230802    2572 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 31 00:27:25.240629 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.230805    2572 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 31 00:27:25.240629 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.230809    2572 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 31 00:27:25.240629 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.230811    2572 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 31 00:27:25.240629 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.230814    2572 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 31 00:27:25.240629 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.230816    2572 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 31 00:27:25.240629 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.230819    2572 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 31 00:27:25.240629 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.230821    2572 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 31 00:27:25.240629 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.230824    2572 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 31 00:27:25.240629 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.230826    2572 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 31 00:27:25.240629 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.230829    2572 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 31 00:27:25.240629 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.230831    2572 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 31 00:27:25.240629 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.230834    2572 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 31 00:27:25.241050 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.230836    2572 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 31 00:27:25.241050 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.230838    2572 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 31 00:27:25.241050 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.230841    2572 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 31 00:27:25.241050 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.230844    2572 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 31 00:27:25.241050 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.230846    2572 feature_gate.go:386] unrecognized feature gate: Example
Oct 31 00:27:25.241050 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.230849    2572 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 31 00:27:25.241050 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.230852    2572 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 31 00:27:25.241050 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.230854    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 31 00:27:25.241050 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.230857    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 31 00:27:25.241050 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.230860    2572 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 31 00:27:25.241050 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.230863    2572 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 31 00:27:25.241050 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:25.230865    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 31 00:27:25.241050 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.230923    2572 server.go:957] "Client rotation is on, will bootstrap in background"
Oct 31 00:27:25.241050 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.232989    2572 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Oct 31 00:27:25.241050 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.234115    2572 server.go:1014] "Starting client certificate rotation"
Oct 31 00:27:25.241050 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.234238    2572 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Oct 31 00:27:25.241050 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.235082    2572 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Oct 31 00:27:25.241050 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.237288    2572 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Oct 31 00:27:25.241469 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.238343    2572 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Oct 31 00:27:25.249983 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.249969    2572 log.go:25] "Validated CRI v1 runtime API"
Oct 31 00:27:25.252691 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.252677    2572 log.go:25] "Validated CRI v1 image API"
Oct 31 00:27:25.253742 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.253700    2572 server.go:1424] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Oct 31 00:27:25.257952 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.257927    2572 fs.go:135] Filesystem UUIDs: map[08366a60-f410-4403-861b-20a69f500d7d:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 b31872fd-a5d9-412e-b841-083f2dc08025:/dev/nvme0n1p3]
Oct 31 00:27:25.258026 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.257951    2572 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:24 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:26 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:32 fsType:tmpfs blockSize:0} composefs_0-34:{mountpoint:/ major:0 minor:34 fsType:overlay blockSize:0}]
Oct 31 00:27:25.264834 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.264728    2572 manager.go:217] Machine: {Timestamp:2026-10-31 00:27:25.263399456 +0000 UTC m=+0.363942281 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3250991 MemoryCapacity:32812019712 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2bd4a8a49e2a5ad51eb8832cab50c2 SystemUUID:ec2bd4a8-a49e-2a5a-d51e-b8832cab50c2 BootID:251962ca-cb1d-4df9-9f05-f29568131ac4 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:24 Capacity:16406007808 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:26 Capacity:6562406400 Type:vfs Inodes:819200 HasInodes:true} {Device:composefs_0-34 DeviceMajor:0 DeviceMinor:34 Capacity:6422528 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:32 Capacity:16406011904 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:06:ff:cc:06:45:87 Speed:0 Mtu:9001} {Name:ens5 MacAddress:06:ff:cc:06:45:87 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:3a:b4:36:ce:b8:78 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812019712 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Oct 31 00:27:25.264834 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.264828    2572 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Oct 31 00:27:25.264981 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.264922    2572 manager.go:233] Version: {KernelVersion:5.14.0-687.60.1.el9_8.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.8.20261023-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Oct 31 00:27:25.266622 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.266599    2572 container_manager_linux.go:275] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Oct 31 00:27:25.266751 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.266624    2572 container_manager_linux.go:280] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-132-173.us-west-2.compute.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"/system.slice","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{},"system-reserved-compressible":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Oct 31 00:27:25.266791 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.266757    2572 topology_manager.go:143] "Creating topology manager with none policy"
Oct 31 00:27:25.266791 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.266764    2572 container_manager_linux.go:311] "Creating device plugin manager"
Oct 31 00:27:25.266791 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.266780    2572 manager.go:144] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Oct 31 00:27:25.266868 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.266793    2572 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Oct 31 00:27:25.267846 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.267836    2572 container_manager_linux.go:320] "Creating Dynamic Resource Allocation (DRA) manager"
Oct 31 00:27:25.271151 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.271139    2572 state_mem.go:41] "Initialized" logger="CPUManager state memory"
Oct 31 00:27:25.271283 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.271274    2572 server.go:1235] "Using root directory" path="/var/lib/kubelet"
Oct 31 00:27:25.272163 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.272153    2572 kubelet.go:493] "Attempting to sync node with API server"
Oct 31 00:27:25.272199 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.272170    2572 kubelet.go:394] "Adding static pod path" path="/etc/kubernetes/manifests"
Oct 31 00:27:25.272199 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.272187    2572 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Oct 31 00:27:25.272264 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.272200    2572 kubelet.go:405] "Adding apiserver pod source"
Oct 31 00:27:25.272264 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.272228    2572 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Oct 31 00:27:25.272581 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.272566    2572 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Oct 31 00:27:25.275969 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.275950    2572 kuberuntime_manager.go:304] "Container runtime initialized" containerRuntime="cri-o" version="1.35.10-3.rhaos4.22.git9383502.el9" apiVersion="v1"
Oct 31 00:27:25.281266 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.281254    2572 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Oct 31 00:27:25.282103 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.282092    2572 kubelet.go:963] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Oct 31 00:27:25.282140 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.282122    2572 kubelet.go:990] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Oct 31 00:27:25.282333 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.282301    2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Oct 31 00:27:25.282370 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.282346    2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Oct 31 00:27:25.282370 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.282361    2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Oct 31 00:27:25.282429 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.282374    2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Oct 31 00:27:25.282429 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.282387    2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Oct 31 00:27:25.282429 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.282400    2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Oct 31 00:27:25.283009 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.283000    2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Oct 31 00:27:25.283037 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.283011    2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Oct 31 00:27:25.283037 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.283019    2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Oct 31 00:27:25.283037 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.283024    2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Oct 31 00:27:25.283037 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.283032    2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Oct 31 00:27:25.283245 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.283043    2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Oct 31 00:27:25.284402 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.284391    2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Oct 31 00:27:25.284457 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.284404    2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Oct 31 00:27:25.288239 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.288225    2572 server.go:1263] "Started kubelet"
Oct 31 00:27:25.288307 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.288265    2572 server.go:182] "Starting to listen" address="0.0.0.0" port=10250
Oct 31 00:27:25.288355 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.288318    2572 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Oct 31 00:27:25.288414 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.288400    2572 server_v1.go:49] "podresources" method="list" useActivePods=true
Oct 31 00:27:25.288904 ip-10-0-132-173 systemd[1]: Started Kubernetes Kubelet.
Oct 31 00:27:25.289757 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.289743    2572 server.go:254] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Oct 31 00:27:25.292281 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.292267    2572 server.go:317] "Adding debug handlers to kubelet server"
Oct 31 00:27:25.294927 ip-10-0-132-173 kubenswrapper[2572]: E1031 00:27:25.294902    2572 kubelet.go:1679] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Oct 31 00:27:25.295747 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.295731    2572 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Oct 31 00:27:25.297591 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.297572    2572 fs_resource_analyzer.go:69] "Starting FS ResourceAnalyzer"
Oct 31 00:27:25.297749 ip-10-0-132-173 kubenswrapper[2572]: E1031 00:27:25.297727    2572 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-132-173.us-west-2.compute.internal\" not found"
Oct 31 00:27:25.298408 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.298386    2572 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Oct 31 00:27:25.298517 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.298500    2572 volume_manager.go:309] "The desired_state_of_world populator starts"
Oct 31 00:27:25.298517 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.298517    2572 volume_manager.go:311] "Starting Kubelet Volume Manager"
Oct 31 00:27:25.298660 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.298578    2572 factory.go:55] Registering systemd factory
Oct 31 00:27:25.298660 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.298595    2572 factory.go:223] Registration of the systemd container factory successfully
Oct 31 00:27:25.298908 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.298889    2572 factory.go:153] Registering CRI-O factory
Oct 31 00:27:25.298975 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.298910    2572 reconstruct.go:97] "Volume reconstruction finished"
Oct 31 00:27:25.298975 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.298919    2572 reconciler.go:29] "Reconciler: start to sync state"
Oct 31 00:27:25.298975 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.298913    2572 factory.go:223] Registration of the crio container factory successfully
Oct 31 00:27:25.298975 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.298973    2572 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Oct 31 00:27:25.299187 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.298994    2572 factory.go:103] Registering Raw factory
Oct 31 00:27:25.299187 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.299016    2572 manager.go:1199] Started watching for new ooms in manager
Oct 31 00:27:25.300110 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.300093    2572 manager.go:319] Starting recovery of all containers
Oct 31 00:27:25.309230 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.309198    2572 manager.go:324] Recovery completed
Oct 31 00:27:25.313601 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.313588    2572 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 31 00:27:25.315687 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.315664    2572 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-173.us-west-2.compute.internal" event="NodeHasSufficientMemory"
Oct 31 00:27:25.315735 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.315702    2572 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-173.us-west-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 31 00:27:25.315735 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.315712    2572 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-173.us-west-2.compute.internal" event="NodeHasSufficientPID"
Oct 31 00:27:25.316235 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.316223    2572 cpu_manager.go:226] "Starting" policy="none"
Oct 31 00:27:25.316276 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.316235    2572 cpu_manager.go:227] "Reconciling" reconcilePeriod="10s"
Oct 31 00:27:25.316276 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.316246    2572 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 31 00:27:25.316276 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.316254    2572 state_mem.go:41] "Initialized" logger="CPUManager state checkpoint.CPUManager state memory"
Oct 31 00:27:25.317664 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.317638    2572 policy_none.go:50] "Start"
Oct 31 00:27:25.317728 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.317667    2572 memory_manager.go:187] "Starting memorymanager" policy="None"
Oct 31 00:27:25.317728 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.317682    2572 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Oct 31 00:27:25.317728 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.317702    2572 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-173.us-west-2.compute.internal" event="NodeHasSufficientMemory"
Oct 31 00:27:25.317822 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.317729    2572 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-173.us-west-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 31 00:27:25.317822 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.317743    2572 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-173.us-west-2.compute.internal" event="NodeHasSufficientPID"
Oct 31 00:27:25.318909 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.318895    2572 policy_none.go:44] "Start"
Oct 31 00:27:25.367873 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.367837    2572 node_container_manager_linux.go:130] "Enforcing system reserved compressible on cgroup" cgroupName="/system.slice" limits={"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"}
Oct 31 00:27:25.371856 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.371842    2572 manager.go:341] "Starting Device Plugin manager"
Oct 31 00:27:25.372435 ip-10-0-132-173 kubenswrapper[2572]: E1031 00:27:25.372418    2572 manager.go:525] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Oct 31 00:27:25.372435 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.372434    2572 server.go:87] "Starting device plugin registration server"
Oct 31 00:27:25.372649 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.372639    2572 eviction_manager.go:194] "Eviction manager: starting control loop"
Oct 31 00:27:25.372702 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.372651    2572 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Oct 31 00:27:25.372783 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.372768    2572 plugin_watcher.go:53] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Oct 31 00:27:25.372884 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.372872    2572 plugin_manager.go:119] "The desired_state_of_world populator (plugin watcher) starts"
Oct 31 00:27:25.372943 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.372893    2572 plugin_manager.go:121] "Starting Kubelet Plugin Manager"
Oct 31 00:27:25.373394 ip-10-0-132-173 kubenswrapper[2572]: E1031 00:27:25.373376    2572 eviction_manager.go:272] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Oct 31 00:27:25.373470 ip-10-0-132-173 kubenswrapper[2572]: E1031 00:27:25.373422    2572 eviction_manager.go:297] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-132-173.us-west-2.compute.internal\" not found"
Oct 31 00:27:25.424900 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.423555    2572 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-132-173.us-west-2.compute.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Oct 31 00:27:25.424900 ip-10-0-132-173 kubenswrapper[2572]: E1031 00:27:25.423557    2572 controller.go:201] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-132-173.us-west-2.compute.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Oct 31 00:27:25.424900 ip-10-0-132-173 kubenswrapper[2572]: E1031 00:27:25.423561    2572 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-132-173.us-west-2.compute.internal.18e375a8fd4c3743  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-132-173.us-west-2.compute.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-132-173.us-west-2.compute.internal,},FirstTimestamp:2026-10-31 00:27:25.288175427 +0000 UTC m=+0.388718250,LastTimestamp:2026-10-31 00:27:25.288175427 +0000 UTC m=+0.388718250,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-132-173.us-west-2.compute.internal,}"
Oct 31 00:27:25.463327 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.463244    2572 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Oct 31 00:27:25.464533 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.464518    2572 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Oct 31 00:27:25.464589 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.464540    2572 status_manager.go:249] "Starting to sync pod status with apiserver"
Oct 31 00:27:25.464589 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.464559    2572 kubelet.go:2521] "Starting kubelet main sync loop"
Oct 31 00:27:25.464658 ip-10-0-132-173 kubenswrapper[2572]: E1031 00:27:25.464589    2572 kubelet.go:2545] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Oct 31 00:27:25.473057 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.473038    2572 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 31 00:27:25.473727 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.473708    2572 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-173.us-west-2.compute.internal" event="NodeHasSufficientMemory"
Oct 31 00:27:25.473820 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.473739    2572 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-173.us-west-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 31 00:27:25.473820 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.473748    2572 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-173.us-west-2.compute.internal" event="NodeHasSufficientPID"
Oct 31 00:27:25.473820 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.473777    2572 kubelet_node_status.go:77] "Attempting to register node" node="ip-10-0-132-173.us-west-2.compute.internal"
Oct 31 00:27:25.548671 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.548642    2572 reflector.go:446] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Oct 31 00:27:25.548812 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.548717    2572 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-5m7m7"
Oct 31 00:27:25.565615 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.565575    2572 kubelet.go:2608] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-173.us-west-2.compute.internal","kube-system/kube-apiserver-proxy-ip-10-0-132-173.us-west-2.compute.internal"]
Oct 31 00:27:25.565717 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.565647    2572 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 31 00:27:25.566924 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.566909    2572 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-173.us-west-2.compute.internal" event="NodeHasSufficientMemory"
Oct 31 00:27:25.567007 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.566940    2572 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-173.us-west-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 31 00:27:25.567007 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.566953    2572 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-173.us-west-2.compute.internal" event="NodeHasSufficientPID"
Oct 31 00:27:25.568530 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.568516    2572 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 31 00:27:25.568680 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.568666    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-173.us-west-2.compute.internal"
Oct 31 00:27:25.568719 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.568697    2572 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 31 00:27:25.571123 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.571100    2572 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-173.us-west-2.compute.internal" event="NodeHasSufficientMemory"
Oct 31 00:27:25.571205 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.571134    2572 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-173.us-west-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 31 00:27:25.571205 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.571104    2572 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-173.us-west-2.compute.internal" event="NodeHasSufficientMemory"
Oct 31 00:27:25.571205 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.571166    2572 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-173.us-west-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 31 00:27:25.571205 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.571181    2572 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-173.us-west-2.compute.internal" event="NodeHasSufficientPID"
Oct 31 00:27:25.571205 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.571145    2572 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-173.us-west-2.compute.internal" event="NodeHasSufficientPID"
Oct 31 00:27:25.572652 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.572635    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-173.us-west-2.compute.internal"
Oct 31 00:27:25.572743 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.572660    2572 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 31 00:27:25.573698 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.573682    2572 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-173.us-west-2.compute.internal" event="NodeHasSufficientMemory"
Oct 31 00:27:25.573777 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.573713    2572 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-173.us-west-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 31 00:27:25.573777 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.573724    2572 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-173.us-west-2.compute.internal" event="NodeHasSufficientPID"
Oct 31 00:27:25.585546 ip-10-0-132-173 kubenswrapper[2572]: E1031 00:27:25.585456    2572 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-132-173.us-west-2.compute.internal.18e375a8fef003d3  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-132-173.us-west-2.compute.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-132-173.us-west-2.compute.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-132-173.us-west-2.compute.internal,},FirstTimestamp:2026-10-31 00:27:25.315687379 +0000 UTC m=+0.416230202,LastTimestamp:2026-10-31 00:27:25.315687379 +0000 UTC m=+0.416230202,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-132-173.us-west-2.compute.internal,}"
Oct 31 00:27:25.585665 ip-10-0-132-173 kubenswrapper[2572]: E1031 00:27:25.585608    2572 reflector.go:204] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.Service"
Oct 31 00:27:25.585712 ip-10-0-132-173 kubenswrapper[2572]: E1031 00:27:25.585679    2572 reflector.go:204] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.CSIDriver"
Oct 31 00:27:25.585812 ip-10-0-132-173 kubenswrapper[2572]: E1031 00:27:25.585789    2572 reflector.go:204] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-132-173.us-west-2.compute.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.Node"
Oct 31 00:27:25.599768 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.599750    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6b005c94812deb7bd218c8f0cf290c09-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-173.us-west-2.compute.internal\" (UID: \"6b005c94812deb7bd218c8f0cf290c09\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-173.us-west-2.compute.internal"
Oct 31 00:27:25.599849 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.599816    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/6c9ceace9ec30530fd2581e47cd10afe-config\") pod \"kube-apiserver-proxy-ip-10-0-132-173.us-west-2.compute.internal\" (UID: \"6c9ceace9ec30530fd2581e47cd10afe\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-173.us-west-2.compute.internal"
Oct 31 00:27:25.599887 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.599858    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/6b005c94812deb7bd218c8f0cf290c09-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-173.us-west-2.compute.internal\" (UID: \"6b005c94812deb7bd218c8f0cf290c09\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-173.us-west-2.compute.internal"
Oct 31 00:27:25.601200 ip-10-0-132-173 kubenswrapper[2572]: E1031 00:27:25.601186    2572 kubelet.go:3364] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-132-173.us-west-2.compute.internal\" not found" node="ip-10-0-132-173.us-west-2.compute.internal"
Oct 31 00:27:25.604758 ip-10-0-132-173 kubenswrapper[2572]: E1031 00:27:25.604740    2572 kubelet.go:3364] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-132-173.us-west-2.compute.internal\" not found" node="ip-10-0-132-173.us-west-2.compute.internal"
Oct 31 00:27:25.639061 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.639038    2572 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 31 00:27:25.639598 ip-10-0-132-173 kubenswrapper[2572]: E1031 00:27:25.639582    2572 kubelet_node_status.go:115] "Unable to register node with API server, error getting existing node" err="nodes \"ip-10-0-132-173.us-west-2.compute.internal\" not found" node="ip-10-0-132-173.us-west-2.compute.internal"
Oct 31 00:27:25.700448 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.700419    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/6c9ceace9ec30530fd2581e47cd10afe-config\") pod \"kube-apiserver-proxy-ip-10-0-132-173.us-west-2.compute.internal\" (UID: \"6c9ceace9ec30530fd2581e47cd10afe\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-173.us-west-2.compute.internal"
Oct 31 00:27:25.700448 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.700451    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/6b005c94812deb7bd218c8f0cf290c09-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-173.us-west-2.compute.internal\" (UID: \"6b005c94812deb7bd218c8f0cf290c09\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-173.us-west-2.compute.internal"
Oct 31 00:27:25.700615 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.700468    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6b005c94812deb7bd218c8f0cf290c09-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-173.us-west-2.compute.internal\" (UID: \"6b005c94812deb7bd218c8f0cf290c09\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-173.us-west-2.compute.internal"
Oct 31 00:27:25.700615 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.700517    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6b005c94812deb7bd218c8f0cf290c09-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-173.us-west-2.compute.internal\" (UID: \"6b005c94812deb7bd218c8f0cf290c09\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-173.us-west-2.compute.internal"
Oct 31 00:27:25.700615 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.700532    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/6b005c94812deb7bd218c8f0cf290c09-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-173.us-west-2.compute.internal\" (UID: \"6b005c94812deb7bd218c8f0cf290c09\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-173.us-west-2.compute.internal"
Oct 31 00:27:25.700615 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.700532    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/6c9ceace9ec30530fd2581e47cd10afe-config\") pod \"kube-apiserver-proxy-ip-10-0-132-173.us-west-2.compute.internal\" (UID: \"6c9ceace9ec30530fd2581e47cd10afe\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-173.us-west-2.compute.internal"
Oct 31 00:27:25.732026 ip-10-0-132-173 kubenswrapper[2572]: E1031 00:27:25.731957    2572 nodelease.go:50] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-132-173.us-west-2.compute.internal\" not found" node="ip-10-0-132-173.us-west-2.compute.internal"
Oct 31 00:27:25.840175 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.840149    2572 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 31 00:27:25.841229 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.841197    2572 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-173.us-west-2.compute.internal" event="NodeHasSufficientMemory"
Oct 31 00:27:25.841323 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.841249    2572 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-173.us-west-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 31 00:27:25.841323 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.841259    2572 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-173.us-west-2.compute.internal" event="NodeHasSufficientPID"
Oct 31 00:27:25.841323 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.841286    2572 kubelet_node_status.go:77] "Attempting to register node" node="ip-10-0-132-173.us-west-2.compute.internal"
Oct 31 00:27:25.900134 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.900109    2572 kubelet_node_status.go:80] "Successfully registered node" node="ip-10-0-132-173.us-west-2.compute.internal"
Oct 31 00:27:25.900134 ip-10-0-132-173 kubenswrapper[2572]: E1031 00:27:25.900135    2572 kubelet_node_status.go:524] "Error updating node status, will retry" err="error getting node \"ip-10-0-132-173.us-west-2.compute.internal\": node \"ip-10-0-132-173.us-west-2.compute.internal\" not found"
Oct 31 00:27:25.905368 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.905348    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-173.us-west-2.compute.internal"
Oct 31 00:27:25.908262 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:25.908245    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-173.us-west-2.compute.internal"
Oct 31 00:27:26.017835 ip-10-0-132-173 kubenswrapper[2572]: E1031 00:27:26.017757    2572 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-132-173.us-west-2.compute.internal\" not found"
Oct 31 00:27:26.118805 ip-10-0-132-173 kubenswrapper[2572]: E1031 00:27:26.118762    2572 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-132-173.us-west-2.compute.internal\" not found"
Oct 31 00:27:26.219584 ip-10-0-132-173 kubenswrapper[2572]: E1031 00:27:26.219540    2572 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-132-173.us-west-2.compute.internal\" not found"
Oct 31 00:27:26.234033 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:26.234011    2572 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Oct 31 00:27:26.234234 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:26.234196    2572 reflector.go:578] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:161: Unexpected watch close - watch lasted less than a second and no items received"
Oct 31 00:27:26.234337 ip-10-0-132-173 kubenswrapper[2572]: E1031 00:27:26.234180    2572 event.go:368] "Unable to write event (may retry after sleeping)" err="Patch \"https://a5bf11d8d49064afc9ef2bd5f2126127-faee8f219fa21b80.elb.us-east-2.amazonaws.com:6443/api/v1/namespaces/default/events/ip-10-0-132-173.us-west-2.compute.internal.18e375a8fef003d3\": read tcp 10.0.132.173:48388->13.58.200.127:6443: use of closed network connection" event="&Event{ObjectMeta:{ip-10-0-132-173.us-west-2.compute.internal.18e375a8fef003d3  default   5216 0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-132-173.us-west-2.compute.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-132-173.us-west-2.compute.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-132-173.us-west-2.compute.internal,},FirstTimestamp:2026-10-31 00:27:25 +0000 UTC,LastTimestamp:2026-10-31 00:27:25.566923912 +0000 UTC m=+0.667466736,Count:4,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-132-173.us-west-2.compute.internal,}"
Oct 31 00:27:26.319948 ip-10-0-132-173 kubenswrapper[2572]: E1031 00:27:26.319887    2572 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-132-173.us-west-2.compute.internal\" not found"
Oct 31 00:27:26.420565 ip-10-0-132-173 kubenswrapper[2572]: E1031 00:27:26.420534    2572 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-132-173.us-west-2.compute.internal\" not found"
Oct 31 00:27:26.517100 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:26.517075    2572 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 31 00:27:26.518207 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:26.518184    2572 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-173.us-west-2.compute.internal" event="NodeHasSufficientMemory"
Oct 31 00:27:26.518352 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:26.518231    2572 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-173.us-west-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 31 00:27:26.518352 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:26.518247    2572 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-173.us-west-2.compute.internal" event="NodeHasSufficientPID"
Oct 31 00:27:26.521067 ip-10-0-132-173 kubenswrapper[2572]: E1031 00:27:26.521051    2572 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-132-173.us-west-2.compute.internal\" not found"
Oct 31 00:27:26.553671 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:26.553629    2572 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-10-30 00:22:26 +0000 UTC" deadline="2028-08-02 01:32:07.023456836 +0000 UTC"
Oct 31 00:27:26.553671 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:26.553664    2572 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15385h4m40.469797642s"
Oct 31 00:27:26.621444 ip-10-0-132-173 kubenswrapper[2572]: E1031 00:27:26.621378    2572 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-132-173.us-west-2.compute.internal\" not found"
Oct 31 00:27:26.674225 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:26.674180    2572 reflector.go:446] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 31 00:27:26.721729 ip-10-0-132-173 kubenswrapper[2572]: E1031 00:27:26.721706    2572 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-132-173.us-west-2.compute.internal\" not found"
Oct 31 00:27:26.822248 ip-10-0-132-173 kubenswrapper[2572]: E1031 00:27:26.822203    2572 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-132-173.us-west-2.compute.internal\" not found"
Oct 31 00:27:26.922761 ip-10-0-132-173 kubenswrapper[2572]: E1031 00:27:26.922733    2572 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-132-173.us-west-2.compute.internal\" not found"
Oct 31 00:27:27.023890 ip-10-0-132-173 kubenswrapper[2572]: E1031 00:27:27.023643    2572 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-132-173.us-west-2.compute.internal\" not found"
Oct 31 00:27:27.046044 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:27.046012    2572 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6b005c94812deb7bd218c8f0cf290c09.slice/crio-320bed655f31eb0479e8e1af7d7a9978e83b9bb1f2756fa39f548f8954a6c756 WatchSource:0}: Error finding container 320bed655f31eb0479e8e1af7d7a9978e83b9bb1f2756fa39f548f8954a6c756: Status 404 returned error can't find the container with id 320bed655f31eb0479e8e1af7d7a9978e83b9bb1f2756fa39f548f8954a6c756
Oct 31 00:27:27.051128 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:27.051098    2572 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6c9ceace9ec30530fd2581e47cd10afe.slice/crio-959d55d527ec7a6c7178ba7b1926ee7fb817f7975b3031a70bd9a576e2e3b1ce WatchSource:0}: Error finding container 959d55d527ec7a6c7178ba7b1926ee7fb817f7975b3031a70bd9a576e2e3b1ce: Status 404 returned error can't find the container with id 959d55d527ec7a6c7178ba7b1926ee7fb817f7975b3031a70bd9a576e2e3b1ce
Oct 31 00:27:27.053047 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.053019    2572 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 31 00:27:27.124376 ip-10-0-132-173 kubenswrapper[2572]: E1031 00:27:27.124343    2572 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-132-173.us-west-2.compute.internal\" not found"
Oct 31 00:27:27.176011 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.175936    2572 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 31 00:27:27.186885 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.186862    2572 reflector.go:446] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 31 00:27:27.225660 ip-10-0-132-173 kubenswrapper[2572]: E1031 00:27:27.225629    2572 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-132-173.us-west-2.compute.internal\" not found"
Oct 31 00:27:27.249895 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.249876    2572 reflector.go:446] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 31 00:27:27.273772 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.273726    2572 apiserver.go:52] "Watching apiserver"
Oct 31 00:27:27.297316 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.297286    2572 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Oct 31 00:27:27.298471 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.298457    2572 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-173.us-west-2.compute.internal"
Oct 31 00:27:27.380049 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.380023    2572 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 31 00:27:27.380212 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.380095    2572 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-173.us-west-2.compute.internal"
Oct 31 00:27:27.380429 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.380413    2572 reflector.go:446] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Oct 31 00:27:27.380758 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.380738    2572 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-target-jktkx","openshift-ovn-kubernetes/ovnkube-node-rrk77","kube-system/global-pull-secret-syncer-24sxq","openshift-dns/node-resolver-zjb6f","openshift-multus/multus-additional-cni-plugins-2t5fx","openshift-multus/multus-d7msb","openshift-network-operator/iptables-alerter-p4txz","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k6fgp","openshift-cluster-node-tuning-operator/tuned-hrsmn","openshift-image-registry/node-ca-qchks","kube-system/konnectivity-agent-k2sk2","openshift-multus/network-metrics-daemon-plnkm"]
Oct 31 00:27:27.384248 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.383808    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jktkx"
Oct 31 00:27:27.384248 ip-10-0-132-173 kubenswrapper[2572]: E1031 00:27:27.383928    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jktkx" podUID="c4ce4bad-af31-4f2e-a684-b2123e9da265"
Oct 31 00:27:27.386152 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.386131    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-rrk77"
Oct 31 00:27:27.387854 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.387835    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-24sxq"
Oct 31 00:27:27.387952 ip-10-0-132-173 kubenswrapper[2572]: E1031 00:27:27.387895    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-24sxq" podUID="97410648-b7ae-4cf4-accf-3164c398b525"
Oct 31 00:27:27.389278 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.389263    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-zjb6f"
Oct 31 00:27:27.390683 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.390668    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-2t5fx"
Oct 31 00:27:27.392133 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.392113    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-d7msb"
Oct 31 00:27:27.393610 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.393590    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-p4txz"
Oct 31 00:27:27.395271 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.395252    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k6fgp"
Oct 31 00:27:27.396766 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.396748    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-hrsmn"
Oct 31 00:27:27.398324 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.398301    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-qchks"
Oct 31 00:27:27.399740 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.399722    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-k2sk2"
Oct 31 00:27:27.401416 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.401397    2572 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-173.us-west-2.compute.internal"]
Oct 31 00:27:27.401485 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.401470    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-plnkm"
Oct 31 00:27:27.401541 ip-10-0-132-173 kubenswrapper[2572]: E1031 00:27:27.401525    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-plnkm" podUID="85e4b969-79f8-4954-8539-0905bd5db99d"
Oct 31 00:27:27.411118 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.411103    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6926e9b8-7c44-413e-bc7e-63cfff593066-host-run-netns\") pod \"multus-d7msb\" (UID: \"6926e9b8-7c44-413e-bc7e-63cfff593066\") " pod="openshift-multus/multus-d7msb"
Oct 31 00:27:27.411188 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.411128    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/9daae1e9-e437-437c-ba8b-3d9c206b64a3-iptables-alerter-script\") pod \"iptables-alerter-p4txz\" (UID: \"9daae1e9-e437-437c-ba8b-3d9c206b64a3\") " pod="openshift-network-operator/iptables-alerter-p4txz"
Oct 31 00:27:27.411188 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.411146    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/21df46e4-2158-4385-ab9a-4b1790b64b55-etc-tuned\") pod \"tuned-hrsmn\" (UID: \"21df46e4-2158-4385-ab9a-4b1790b64b55\") " pod="openshift-cluster-node-tuning-operator/tuned-hrsmn"
Oct 31 00:27:27.411188 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.411161    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/f93a119c-fb17-44bd-973c-699a3b87523b-konnectivity-ca\") pod \"konnectivity-agent-k2sk2\" (UID: \"f93a119c-fb17-44bd-973c-699a3b87523b\") " pod="kube-system/konnectivity-agent-k2sk2"
Oct 31 00:27:27.411188 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.411176    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/63bedd38-6ab7-493d-b94c-b3c288af3f48-log-socket\") pod \"ovnkube-node-rrk77\" (UID: \"63bedd38-6ab7-493d-b94c-b3c288af3f48\") " pod="openshift-ovn-kubernetes/ovnkube-node-rrk77"
Oct 31 00:27:27.411388 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.411266    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/1a72f876-1157-4447-845c-923cdfd5aa5e-kubelet-dir\") pod \"aws-ebs-csi-driver-node-k6fgp\" (UID: \"1a72f876-1157-4447-845c-923cdfd5aa5e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k6fgp"
Oct 31 00:27:27.411388 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.411326    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/63bedd38-6ab7-493d-b94c-b3c288af3f48-host-kubelet\") pod \"ovnkube-node-rrk77\" (UID: \"63bedd38-6ab7-493d-b94c-b3c288af3f48\") " pod="openshift-ovn-kubernetes/ovnkube-node-rrk77"
Oct 31 00:27:27.411388 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.411371    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/21df46e4-2158-4385-ab9a-4b1790b64b55-etc-sysctl-conf\") pod \"tuned-hrsmn\" (UID: \"21df46e4-2158-4385-ab9a-4b1790b64b55\") " pod="openshift-cluster-node-tuning-operator/tuned-hrsmn"
Oct 31 00:27:27.411492 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.411423    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/21df46e4-2158-4385-ab9a-4b1790b64b55-host\") pod \"tuned-hrsmn\" (UID: \"21df46e4-2158-4385-ab9a-4b1790b64b55\") " pod="openshift-cluster-node-tuning-operator/tuned-hrsmn"
Oct 31 00:27:27.411492 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.411442    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/63bedd38-6ab7-493d-b94c-b3c288af3f48-ovnkube-script-lib\") pod \"ovnkube-node-rrk77\" (UID: \"63bedd38-6ab7-493d-b94c-b3c288af3f48\") " pod="openshift-ovn-kubernetes/ovnkube-node-rrk77"
Oct 31 00:27:27.411492 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.411457    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m2phw\" (UniqueName: \"kubernetes.io/projected/6926e9b8-7c44-413e-bc7e-63cfff593066-kube-api-access-m2phw\") pod \"multus-d7msb\" (UID: \"6926e9b8-7c44-413e-bc7e-63cfff593066\") " pod="openshift-multus/multus-d7msb"
Oct 31 00:27:27.411492 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.411485    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/f93a119c-fb17-44bd-973c-699a3b87523b-agent-certs\") pod \"konnectivity-agent-k2sk2\" (UID: \"f93a119c-fb17-44bd-973c-699a3b87523b\") " pod="kube-system/konnectivity-agent-k2sk2"
Oct 31 00:27:27.411638 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.411532    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/63bedd38-6ab7-493d-b94c-b3c288af3f48-etc-openvswitch\") pod \"ovnkube-node-rrk77\" (UID: \"63bedd38-6ab7-493d-b94c-b3c288af3f48\") " pod="openshift-ovn-kubernetes/ovnkube-node-rrk77"
Oct 31 00:27:27.411638 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.411551    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6926e9b8-7c44-413e-bc7e-63cfff593066-host-var-lib-cni-bin\") pod \"multus-d7msb\" (UID: \"6926e9b8-7c44-413e-bc7e-63cfff593066\") " pod="openshift-multus/multus-d7msb"
Oct 31 00:27:27.411638 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.411586    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sslf9\" (UniqueName: \"kubernetes.io/projected/61c60450-eb50-4181-8ab7-e69e864ffb51-kube-api-access-sslf9\") pod \"node-ca-qchks\" (UID: \"61c60450-eb50-4181-8ab7-e69e864ffb51\") " pod="openshift-image-registry/node-ca-qchks"
Oct 31 00:27:27.411638 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.411615    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/63bedd38-6ab7-493d-b94c-b3c288af3f48-systemd-units\") pod \"ovnkube-node-rrk77\" (UID: \"63bedd38-6ab7-493d-b94c-b3c288af3f48\") " pod="openshift-ovn-kubernetes/ovnkube-node-rrk77"
Oct 31 00:27:27.411638 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.411633    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/97410648-b7ae-4cf4-accf-3164c398b525-original-pull-secret\") pod \"global-pull-secret-syncer-24sxq\" (UID: \"97410648-b7ae-4cf4-accf-3164c398b525\") " pod="kube-system/global-pull-secret-syncer-24sxq"
Oct 31 00:27:27.411807 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.411659    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/6926e9b8-7c44-413e-bc7e-63cfff593066-multus-socket-dir-parent\") pod \"multus-d7msb\" (UID: \"6926e9b8-7c44-413e-bc7e-63cfff593066\") " pod="openshift-multus/multus-d7msb"
Oct 31 00:27:27.411807 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.411705    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6926e9b8-7c44-413e-bc7e-63cfff593066-multus-conf-dir\") pod \"multus-d7msb\" (UID: \"6926e9b8-7c44-413e-bc7e-63cfff593066\") " pod="openshift-multus/multus-d7msb"
Oct 31 00:27:27.411807 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.411740    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/97410648-b7ae-4cf4-accf-3164c398b525-dbus\") pod \"global-pull-secret-syncer-24sxq\" (UID: \"97410648-b7ae-4cf4-accf-3164c398b525\") " pod="kube-system/global-pull-secret-syncer-24sxq"
Oct 31 00:27:27.411807 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.411774    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2c9341ef-0860-4fb6-8a86-272ac063756a-tmp-dir\") pod \"node-resolver-zjb6f\" (UID: \"2c9341ef-0860-4fb6-8a86-272ac063756a\") " pod="openshift-dns/node-resolver-zjb6f"
Oct 31 00:27:27.411807 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.411801    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/1a72f876-1157-4447-845c-923cdfd5aa5e-sys-fs\") pod \"aws-ebs-csi-driver-node-k6fgp\" (UID: \"1a72f876-1157-4447-845c-923cdfd5aa5e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k6fgp"
Oct 31 00:27:27.411984 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.411828    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/21df46e4-2158-4385-ab9a-4b1790b64b55-tmp\") pod \"tuned-hrsmn\" (UID: \"21df46e4-2158-4385-ab9a-4b1790b64b55\") " pod="openshift-cluster-node-tuning-operator/tuned-hrsmn"
Oct 31 00:27:27.411984 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.411854    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/63bedd38-6ab7-493d-b94c-b3c288af3f48-host-slash\") pod \"ovnkube-node-rrk77\" (UID: \"63bedd38-6ab7-493d-b94c-b3c288af3f48\") " pod="openshift-ovn-kubernetes/ovnkube-node-rrk77"
Oct 31 00:27:27.411984 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.411869    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/41c4cc4b-b387-4c70-8a27-58255ba632b9-cnibin\") pod \"multus-additional-cni-plugins-2t5fx\" (UID: \"41c4cc4b-b387-4c70-8a27-58255ba632b9\") " pod="openshift-multus/multus-additional-cni-plugins-2t5fx"
Oct 31 00:27:27.411984 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.411892    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/41c4cc4b-b387-4c70-8a27-58255ba632b9-cni-binary-copy\") pod \"multus-additional-cni-plugins-2t5fx\" (UID: \"41c4cc4b-b387-4c70-8a27-58255ba632b9\") " pod="openshift-multus/multus-additional-cni-plugins-2t5fx"
Oct 31 00:27:27.411984 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.411937    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vrldg\" (UniqueName: \"kubernetes.io/projected/41c4cc4b-b387-4c70-8a27-58255ba632b9-kube-api-access-vrldg\") pod \"multus-additional-cni-plugins-2t5fx\" (UID: \"41c4cc4b-b387-4c70-8a27-58255ba632b9\") " pod="openshift-multus/multus-additional-cni-plugins-2t5fx"
Oct 31 00:27:27.412153 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.411985    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/61c60450-eb50-4181-8ab7-e69e864ffb51-serviceca\") pod \"node-ca-qchks\" (UID: \"61c60450-eb50-4181-8ab7-e69e864ffb51\") " pod="openshift-image-registry/node-ca-qchks"
Oct 31 00:27:27.412153 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.412019    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/63bedd38-6ab7-493d-b94c-b3c288af3f48-var-lib-openvswitch\") pod \"ovnkube-node-rrk77\" (UID: \"63bedd38-6ab7-493d-b94c-b3c288af3f48\") " pod="openshift-ovn-kubernetes/ovnkube-node-rrk77"
Oct 31 00:27:27.412153 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.412064    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/63bedd38-6ab7-493d-b94c-b3c288af3f48-run-openvswitch\") pod \"ovnkube-node-rrk77\" (UID: \"63bedd38-6ab7-493d-b94c-b3c288af3f48\") " pod="openshift-ovn-kubernetes/ovnkube-node-rrk77"
Oct 31 00:27:27.412153 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.412098    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/63bedd38-6ab7-493d-b94c-b3c288af3f48-host-cni-netd\") pod \"ovnkube-node-rrk77\" (UID: \"63bedd38-6ab7-493d-b94c-b3c288af3f48\") " pod="openshift-ovn-kubernetes/ovnkube-node-rrk77"
Oct 31 00:27:27.412153 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.412125    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6926e9b8-7c44-413e-bc7e-63cfff593066-system-cni-dir\") pod \"multus-d7msb\" (UID: \"6926e9b8-7c44-413e-bc7e-63cfff593066\") " pod="openshift-multus/multus-d7msb"
Oct 31 00:27:27.412153 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.412148    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/6926e9b8-7c44-413e-bc7e-63cfff593066-host-run-k8s-cni-cncf-io\") pod \"multus-d7msb\" (UID: \"6926e9b8-7c44-413e-bc7e-63cfff593066\") " pod="openshift-multus/multus-d7msb"
Oct 31 00:27:27.412382 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.412173    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/6926e9b8-7c44-413e-bc7e-63cfff593066-host-var-lib-cni-multus\") pod \"multus-d7msb\" (UID: \"6926e9b8-7c44-413e-bc7e-63cfff593066\") " pod="openshift-multus/multus-d7msb"
Oct 31 00:27:27.412382 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.412195    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/9daae1e9-e437-437c-ba8b-3d9c206b64a3-host-slash\") pod \"iptables-alerter-p4txz\" (UID: \"9daae1e9-e437-437c-ba8b-3d9c206b64a3\") " pod="openshift-network-operator/iptables-alerter-p4txz"
Oct 31 00:27:27.412382 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.412249    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/63bedd38-6ab7-493d-b94c-b3c288af3f48-env-overrides\") pod \"ovnkube-node-rrk77\" (UID: \"63bedd38-6ab7-493d-b94c-b3c288af3f48\") " pod="openshift-ovn-kubernetes/ovnkube-node-rrk77"
Oct 31 00:27:27.412382 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.412266    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/21df46e4-2158-4385-ab9a-4b1790b64b55-etc-modprobe-d\") pod \"tuned-hrsmn\" (UID: \"21df46e4-2158-4385-ab9a-4b1790b64b55\") " pod="openshift-cluster-node-tuning-operator/tuned-hrsmn"
Oct 31 00:27:27.412382 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.412282    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/21df46e4-2158-4385-ab9a-4b1790b64b55-etc-sysctl-d\") pod \"tuned-hrsmn\" (UID: \"21df46e4-2158-4385-ab9a-4b1790b64b55\") " pod="openshift-cluster-node-tuning-operator/tuned-hrsmn"
Oct 31 00:27:27.412382 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.412296    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/21df46e4-2158-4385-ab9a-4b1790b64b55-lib-modules\") pod \"tuned-hrsmn\" (UID: \"21df46e4-2158-4385-ab9a-4b1790b64b55\") " pod="openshift-cluster-node-tuning-operator/tuned-hrsmn"
Oct 31 00:27:27.412382 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.412310    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/63bedd38-6ab7-493d-b94c-b3c288af3f48-host-run-ovn-kubernetes\") pod \"ovnkube-node-rrk77\" (UID: \"63bedd38-6ab7-493d-b94c-b3c288af3f48\") " pod="openshift-ovn-kubernetes/ovnkube-node-rrk77"
Oct 31 00:27:27.412382 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.412328    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/63bedd38-6ab7-493d-b94c-b3c288af3f48-ovnkube-config\") pod \"ovnkube-node-rrk77\" (UID: \"63bedd38-6ab7-493d-b94c-b3c288af3f48\") " pod="openshift-ovn-kubernetes/ovnkube-node-rrk77"
Oct 31 00:27:27.412382 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.412363    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/2c9341ef-0860-4fb6-8a86-272ac063756a-hosts-file\") pod \"node-resolver-zjb6f\" (UID: \"2c9341ef-0860-4fb6-8a86-272ac063756a\") " pod="openshift-dns/node-resolver-zjb6f"
Oct 31 00:27:27.412691 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.412389    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/41c4cc4b-b387-4c70-8a27-58255ba632b9-system-cni-dir\") pod \"multus-additional-cni-plugins-2t5fx\" (UID: \"41c4cc4b-b387-4c70-8a27-58255ba632b9\") " pod="openshift-multus/multus-additional-cni-plugins-2t5fx"
Oct 31 00:27:27.412691 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.412408    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/41c4cc4b-b387-4c70-8a27-58255ba632b9-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-2t5fx\" (UID: \"41c4cc4b-b387-4c70-8a27-58255ba632b9\") " pod="openshift-multus/multus-additional-cni-plugins-2t5fx"
Oct 31 00:27:27.412691 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.412441    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/6926e9b8-7c44-413e-bc7e-63cfff593066-multus-daemon-config\") pod \"multus-d7msb\" (UID: \"6926e9b8-7c44-413e-bc7e-63cfff593066\") " pod="openshift-multus/multus-d7msb"
Oct 31 00:27:27.412691 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.412457    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/21df46e4-2158-4385-ab9a-4b1790b64b55-sys\") pod \"tuned-hrsmn\" (UID: \"21df46e4-2158-4385-ab9a-4b1790b64b55\") " pod="openshift-cluster-node-tuning-operator/tuned-hrsmn"
Oct 31 00:27:27.412691 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.412495    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6926e9b8-7c44-413e-bc7e-63cfff593066-host-var-lib-kubelet\") pod \"multus-d7msb\" (UID: \"6926e9b8-7c44-413e-bc7e-63cfff593066\") " pod="openshift-multus/multus-d7msb"
Oct 31 00:27:27.412691 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.412529    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/6926e9b8-7c44-413e-bc7e-63cfff593066-host-run-multus-certs\") pod \"multus-d7msb\" (UID: \"6926e9b8-7c44-413e-bc7e-63cfff593066\") " pod="openshift-multus/multus-d7msb"
Oct 31 00:27:27.412691 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.412548    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6926e9b8-7c44-413e-bc7e-63cfff593066-etc-kubernetes\") pod \"multus-d7msb\" (UID: \"6926e9b8-7c44-413e-bc7e-63cfff593066\") " pod="openshift-multus/multus-d7msb"
Oct 31 00:27:27.412691 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.412571    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/21df46e4-2158-4385-ab9a-4b1790b64b55-etc-sysconfig\") pod \"tuned-hrsmn\" (UID: \"21df46e4-2158-4385-ab9a-4b1790b64b55\") " pod="openshift-cluster-node-tuning-operator/tuned-hrsmn"
Oct 31 00:27:27.412691 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.412594    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/21df46e4-2158-4385-ab9a-4b1790b64b55-etc-kubernetes\") pod \"tuned-hrsmn\" (UID: \"21df46e4-2158-4385-ab9a-4b1790b64b55\") " pod="openshift-cluster-node-tuning-operator/tuned-hrsmn"
Oct 31 00:27:27.412691 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.412622    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/1a72f876-1157-4447-845c-923cdfd5aa5e-device-dir\") pod \"aws-ebs-csi-driver-node-k6fgp\" (UID: \"1a72f876-1157-4447-845c-923cdfd5aa5e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k6fgp"
Oct 31 00:27:27.412691 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.412661    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/61c60450-eb50-4181-8ab7-e69e864ffb51-host\") pod \"node-ca-qchks\" (UID: \"61c60450-eb50-4181-8ab7-e69e864ffb51\") " pod="openshift-image-registry/node-ca-qchks"
Oct 31 00:27:27.413035 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.412719    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/63bedd38-6ab7-493d-b94c-b3c288af3f48-host-cni-bin\") pod \"ovnkube-node-rrk77\" (UID: \"63bedd38-6ab7-493d-b94c-b3c288af3f48\") " pod="openshift-ovn-kubernetes/ovnkube-node-rrk77"
Oct 31 00:27:27.413035 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.412760    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/63bedd38-6ab7-493d-b94c-b3c288af3f48-ovn-node-metrics-cert\") pod \"ovnkube-node-rrk77\" (UID: \"63bedd38-6ab7-493d-b94c-b3c288af3f48\") " pod="openshift-ovn-kubernetes/ovnkube-node-rrk77"
Oct 31 00:27:27.413035 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.412802    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x6bgk\" (UniqueName: \"kubernetes.io/projected/2c9341ef-0860-4fb6-8a86-272ac063756a-kube-api-access-x6bgk\") pod \"node-resolver-zjb6f\" (UID: \"2c9341ef-0860-4fb6-8a86-272ac063756a\") " pod="openshift-dns/node-resolver-zjb6f"
Oct 31 00:27:27.413035 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.412841    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/1a72f876-1157-4447-845c-923cdfd5aa5e-socket-dir\") pod \"aws-ebs-csi-driver-node-k6fgp\" (UID: \"1a72f876-1157-4447-845c-923cdfd5aa5e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k6fgp"
Oct 31 00:27:27.413035 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.412873    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xhp7k\" (UniqueName: \"kubernetes.io/projected/1a72f876-1157-4447-845c-923cdfd5aa5e-kube-api-access-xhp7k\") pod \"aws-ebs-csi-driver-node-k6fgp\" (UID: \"1a72f876-1157-4447-845c-923cdfd5aa5e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k6fgp"
Oct 31 00:27:27.413035 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.412892    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gm472\" (UniqueName: \"kubernetes.io/projected/21df46e4-2158-4385-ab9a-4b1790b64b55-kube-api-access-gm472\") pod \"tuned-hrsmn\" (UID: \"21df46e4-2158-4385-ab9a-4b1790b64b55\") " pod="openshift-cluster-node-tuning-operator/tuned-hrsmn"
Oct 31 00:27:27.413035 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.412915    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hxhdf\" (UniqueName: \"kubernetes.io/projected/c4ce4bad-af31-4f2e-a684-b2123e9da265-kube-api-access-hxhdf\") pod \"network-check-target-jktkx\" (UID: \"c4ce4bad-af31-4f2e-a684-b2123e9da265\") " pod="openshift-network-diagnostics/network-check-target-jktkx"
Oct 31 00:27:27.413035 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.412930    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/97410648-b7ae-4cf4-accf-3164c398b525-kubelet-config\") pod \"global-pull-secret-syncer-24sxq\" (UID: \"97410648-b7ae-4cf4-accf-3164c398b525\") " pod="kube-system/global-pull-secret-syncer-24sxq"
Oct 31 00:27:27.413035 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.412972    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/6926e9b8-7c44-413e-bc7e-63cfff593066-hostroot\") pod \"multus-d7msb\" (UID: \"6926e9b8-7c44-413e-bc7e-63cfff593066\") " pod="openshift-multus/multus-d7msb"
Oct 31 00:27:27.413035 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.412997    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vp9kn\" (UniqueName: \"kubernetes.io/projected/63bedd38-6ab7-493d-b94c-b3c288af3f48-kube-api-access-vp9kn\") pod \"ovnkube-node-rrk77\" (UID: \"63bedd38-6ab7-493d-b94c-b3c288af3f48\") " pod="openshift-ovn-kubernetes/ovnkube-node-rrk77"
Oct 31 00:27:27.413035 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.413023    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/1a72f876-1157-4447-845c-923cdfd5aa5e-etc-selinux\") pod \"aws-ebs-csi-driver-node-k6fgp\" (UID: \"1a72f876-1157-4447-845c-923cdfd5aa5e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k6fgp"
Oct 31 00:27:27.413455 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.413044    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/21df46e4-2158-4385-ab9a-4b1790b64b55-etc-systemd\") pod \"tuned-hrsmn\" (UID: \"21df46e4-2158-4385-ab9a-4b1790b64b55\") " pod="openshift-cluster-node-tuning-operator/tuned-hrsmn"
Oct 31 00:27:27.413455 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.413059    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/63bedd38-6ab7-493d-b94c-b3c288af3f48-host-run-netns\") pod \"ovnkube-node-rrk77\" (UID: \"63bedd38-6ab7-493d-b94c-b3c288af3f48\") " pod="openshift-ovn-kubernetes/ovnkube-node-rrk77"
Oct 31 00:27:27.413455 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.413092    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/63bedd38-6ab7-493d-b94c-b3c288af3f48-run-systemd\") pod \"ovnkube-node-rrk77\" (UID: \"63bedd38-6ab7-493d-b94c-b3c288af3f48\") " pod="openshift-ovn-kubernetes/ovnkube-node-rrk77"
Oct 31 00:27:27.413455 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.413140    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/63bedd38-6ab7-493d-b94c-b3c288af3f48-node-log\") pod \"ovnkube-node-rrk77\" (UID: \"63bedd38-6ab7-493d-b94c-b3c288af3f48\") " pod="openshift-ovn-kubernetes/ovnkube-node-rrk77"
Oct 31 00:27:27.413455 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.413174    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4xxhd\" (UniqueName: \"kubernetes.io/projected/9daae1e9-e437-437c-ba8b-3d9c206b64a3-kube-api-access-4xxhd\") pod \"iptables-alerter-p4txz\" (UID: \"9daae1e9-e437-437c-ba8b-3d9c206b64a3\") " pod="openshift-network-operator/iptables-alerter-p4txz"
Oct 31 00:27:27.413455 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.413216    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/21df46e4-2158-4385-ab9a-4b1790b64b55-run\") pod \"tuned-hrsmn\" (UID: \"21df46e4-2158-4385-ab9a-4b1790b64b55\") " pod="openshift-cluster-node-tuning-operator/tuned-hrsmn"
Oct 31 00:27:27.413455 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.413270    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/63bedd38-6ab7-493d-b94c-b3c288af3f48-run-ovn\") pod \"ovnkube-node-rrk77\" (UID: \"63bedd38-6ab7-493d-b94c-b3c288af3f48\") " pod="openshift-ovn-kubernetes/ovnkube-node-rrk77"
Oct 31 00:27:27.413455 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.413305    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/63bedd38-6ab7-493d-b94c-b3c288af3f48-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-rrk77\" (UID: \"63bedd38-6ab7-493d-b94c-b3c288af3f48\") " pod="openshift-ovn-kubernetes/ovnkube-node-rrk77"
Oct 31 00:27:27.413455 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.413333    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/41c4cc4b-b387-4c70-8a27-58255ba632b9-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-2t5fx\" (UID: \"41c4cc4b-b387-4c70-8a27-58255ba632b9\") " pod="openshift-multus/multus-additional-cni-plugins-2t5fx"
Oct 31 00:27:27.413455 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.413404    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6926e9b8-7c44-413e-bc7e-63cfff593066-cni-binary-copy\") pod \"multus-d7msb\" (UID: \"6926e9b8-7c44-413e-bc7e-63cfff593066\") " pod="openshift-multus/multus-d7msb"
Oct 31 00:27:27.413455 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.413431    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/1a72f876-1157-4447-845c-923cdfd5aa5e-registration-dir\") pod \"aws-ebs-csi-driver-node-k6fgp\" (UID: \"1a72f876-1157-4447-845c-923cdfd5aa5e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k6fgp"
Oct 31 00:27:27.413455 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.413456    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/21df46e4-2158-4385-ab9a-4b1790b64b55-var-lib-kubelet\") pod \"tuned-hrsmn\" (UID: \"21df46e4-2158-4385-ab9a-4b1790b64b55\") " pod="openshift-cluster-node-tuning-operator/tuned-hrsmn"
Oct 31 00:27:27.413815 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.413499    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/41c4cc4b-b387-4c70-8a27-58255ba632b9-tuning-conf-dir\") pod \"multus-additional-cni-plugins-2t5fx\" (UID: \"41c4cc4b-b387-4c70-8a27-58255ba632b9\") " pod="openshift-multus/multus-additional-cni-plugins-2t5fx"
Oct 31 00:27:27.413815 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.413525    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6926e9b8-7c44-413e-bc7e-63cfff593066-multus-cni-dir\") pod \"multus-d7msb\" (UID: \"6926e9b8-7c44-413e-bc7e-63cfff593066\") " pod="openshift-multus/multus-d7msb"
Oct 31 00:27:27.413815 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.413548    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6926e9b8-7c44-413e-bc7e-63cfff593066-cnibin\") pod \"multus-d7msb\" (UID: \"6926e9b8-7c44-413e-bc7e-63cfff593066\") " pod="openshift-multus/multus-d7msb"
Oct 31 00:27:27.437989 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.437966    2572 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 31 00:27:27.438366 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.438346    2572 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kube-system/kube-apiserver-proxy-ip-10-0-132-173.us-west-2.compute.internal"]
Oct 31 00:27:27.440229 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.440204    2572 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Oct 31 00:27:27.440229 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.440209    2572 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Oct 31 00:27:27.440801 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.440788    2572 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Oct 31 00:27:27.440872 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.440828    2572 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-v6gdh\""
Oct 31 00:27:27.440872 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.440838    2572 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Oct 31 00:27:27.440872 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.440866    2572 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Oct 31 00:27:27.440977 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.440887    2572 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Oct 31 00:27:27.442661 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.442638    2572 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Oct 31 00:27:27.442924 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.442911    2572 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-rw8bx\""
Oct 31 00:27:27.471237 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.471130    2572 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-173.us-west-2.compute.internal" event={"ID":"6b005c94812deb7bd218c8f0cf290c09","Type":"ContainerStarted","Data":"320bed655f31eb0479e8e1af7d7a9978e83b9bb1f2756fa39f548f8954a6c756"}
Oct 31 00:27:27.472599 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.472574    2572 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-173.us-west-2.compute.internal" event={"ID":"6c9ceace9ec30530fd2581e47cd10afe","Type":"ContainerStarted","Data":"959d55d527ec7a6c7178ba7b1926ee7fb817f7975b3031a70bd9a576e2e3b1ce"}
Oct 31 00:27:27.499227 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.499182    2572 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Oct 31 00:27:27.514045 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.514019    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/21df46e4-2158-4385-ab9a-4b1790b64b55-etc-modprobe-d\") pod \"tuned-hrsmn\" (UID: \"21df46e4-2158-4385-ab9a-4b1790b64b55\") " pod="openshift-cluster-node-tuning-operator/tuned-hrsmn"
Oct 31 00:27:27.514133 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.514051    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/21df46e4-2158-4385-ab9a-4b1790b64b55-etc-sysctl-d\") pod \"tuned-hrsmn\" (UID: \"21df46e4-2158-4385-ab9a-4b1790b64b55\") " pod="openshift-cluster-node-tuning-operator/tuned-hrsmn"
Oct 31 00:27:27.514133 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.514068    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/21df46e4-2158-4385-ab9a-4b1790b64b55-lib-modules\") pod \"tuned-hrsmn\" (UID: \"21df46e4-2158-4385-ab9a-4b1790b64b55\") " pod="openshift-cluster-node-tuning-operator/tuned-hrsmn"
Oct 31 00:27:27.514133 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.514084    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/63bedd38-6ab7-493d-b94c-b3c288af3f48-host-run-ovn-kubernetes\") pod \"ovnkube-node-rrk77\" (UID: \"63bedd38-6ab7-493d-b94c-b3c288af3f48\") " pod="openshift-ovn-kubernetes/ovnkube-node-rrk77"
Oct 31 00:27:27.514133 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.514106    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/63bedd38-6ab7-493d-b94c-b3c288af3f48-ovnkube-config\") pod \"ovnkube-node-rrk77\" (UID: \"63bedd38-6ab7-493d-b94c-b3c288af3f48\") " pod="openshift-ovn-kubernetes/ovnkube-node-rrk77"
Oct 31 00:27:27.514133 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.514129    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/2c9341ef-0860-4fb6-8a86-272ac063756a-hosts-file\") pod \"node-resolver-zjb6f\" (UID: \"2c9341ef-0860-4fb6-8a86-272ac063756a\") " pod="openshift-dns/node-resolver-zjb6f"
Oct 31 00:27:27.514394 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.514164    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/21df46e4-2158-4385-ab9a-4b1790b64b55-etc-modprobe-d\") pod \"tuned-hrsmn\" (UID: \"21df46e4-2158-4385-ab9a-4b1790b64b55\") " pod="openshift-cluster-node-tuning-operator/tuned-hrsmn"
Oct 31 00:27:27.514394 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.514179    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/63bedd38-6ab7-493d-b94c-b3c288af3f48-host-run-ovn-kubernetes\") pod \"ovnkube-node-rrk77\" (UID: \"63bedd38-6ab7-493d-b94c-b3c288af3f48\") " pod="openshift-ovn-kubernetes/ovnkube-node-rrk77"
Oct 31 00:27:27.514394 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.514199    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/21df46e4-2158-4385-ab9a-4b1790b64b55-lib-modules\") pod \"tuned-hrsmn\" (UID: \"21df46e4-2158-4385-ab9a-4b1790b64b55\") " pod="openshift-cluster-node-tuning-operator/tuned-hrsmn"
Oct 31 00:27:27.514394 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.514208    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/41c4cc4b-b387-4c70-8a27-58255ba632b9-system-cni-dir\") pod \"multus-additional-cni-plugins-2t5fx\" (UID: \"41c4cc4b-b387-4c70-8a27-58255ba632b9\") " pod="openshift-multus/multus-additional-cni-plugins-2t5fx"
Oct 31 00:27:27.514394 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.514209    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/21df46e4-2158-4385-ab9a-4b1790b64b55-etc-sysctl-d\") pod \"tuned-hrsmn\" (UID: \"21df46e4-2158-4385-ab9a-4b1790b64b55\") " pod="openshift-cluster-node-tuning-operator/tuned-hrsmn"
Oct 31 00:27:27.514394 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.514189    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/2c9341ef-0860-4fb6-8a86-272ac063756a-hosts-file\") pod \"node-resolver-zjb6f\" (UID: \"2c9341ef-0860-4fb6-8a86-272ac063756a\") " pod="openshift-dns/node-resolver-zjb6f"
Oct 31 00:27:27.514394 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.514252    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/41c4cc4b-b387-4c70-8a27-58255ba632b9-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-2t5fx\" (UID: \"41c4cc4b-b387-4c70-8a27-58255ba632b9\") " pod="openshift-multus/multus-additional-cni-plugins-2t5fx"
Oct 31 00:27:27.514394 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.514308    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/41c4cc4b-b387-4c70-8a27-58255ba632b9-system-cni-dir\") pod \"multus-additional-cni-plugins-2t5fx\" (UID: \"41c4cc4b-b387-4c70-8a27-58255ba632b9\") " pod="openshift-multus/multus-additional-cni-plugins-2t5fx"
Oct 31 00:27:27.514394 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.514310    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/6926e9b8-7c44-413e-bc7e-63cfff593066-multus-daemon-config\") pod \"multus-d7msb\" (UID: \"6926e9b8-7c44-413e-bc7e-63cfff593066\") " pod="openshift-multus/multus-d7msb"
Oct 31 00:27:27.514394 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.514377    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/21df46e4-2158-4385-ab9a-4b1790b64b55-sys\") pod \"tuned-hrsmn\" (UID: \"21df46e4-2158-4385-ab9a-4b1790b64b55\") " pod="openshift-cluster-node-tuning-operator/tuned-hrsmn"
Oct 31 00:27:27.514772 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.514402    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6926e9b8-7c44-413e-bc7e-63cfff593066-host-var-lib-kubelet\") pod \"multus-d7msb\" (UID: \"6926e9b8-7c44-413e-bc7e-63cfff593066\") " pod="openshift-multus/multus-d7msb"
Oct 31 00:27:27.514772 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.514419    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/6926e9b8-7c44-413e-bc7e-63cfff593066-host-run-multus-certs\") pod \"multus-d7msb\" (UID: \"6926e9b8-7c44-413e-bc7e-63cfff593066\") " pod="openshift-multus/multus-d7msb"
Oct 31 00:27:27.514772 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.514417    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/21df46e4-2158-4385-ab9a-4b1790b64b55-sys\") pod \"tuned-hrsmn\" (UID: \"21df46e4-2158-4385-ab9a-4b1790b64b55\") " pod="openshift-cluster-node-tuning-operator/tuned-hrsmn"
Oct 31 00:27:27.514772 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.514434    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6926e9b8-7c44-413e-bc7e-63cfff593066-etc-kubernetes\") pod \"multus-d7msb\" (UID: \"6926e9b8-7c44-413e-bc7e-63cfff593066\") " pod="openshift-multus/multus-d7msb"
Oct 31 00:27:27.514772 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.514448    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/21df46e4-2158-4385-ab9a-4b1790b64b55-etc-sysconfig\") pod \"tuned-hrsmn\" (UID: \"21df46e4-2158-4385-ab9a-4b1790b64b55\") " pod="openshift-cluster-node-tuning-operator/tuned-hrsmn"
Oct 31 00:27:27.514772 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.514461    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6926e9b8-7c44-413e-bc7e-63cfff593066-host-var-lib-kubelet\") pod \"multus-d7msb\" (UID: \"6926e9b8-7c44-413e-bc7e-63cfff593066\") " pod="openshift-multus/multus-d7msb"
Oct 31 00:27:27.514772 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.514469    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/21df46e4-2158-4385-ab9a-4b1790b64b55-etc-kubernetes\") pod \"tuned-hrsmn\" (UID: \"21df46e4-2158-4385-ab9a-4b1790b64b55\") " pod="openshift-cluster-node-tuning-operator/tuned-hrsmn"
Oct 31 00:27:27.514772 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.514480    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6926e9b8-7c44-413e-bc7e-63cfff593066-etc-kubernetes\") pod \"multus-d7msb\" (UID: \"6926e9b8-7c44-413e-bc7e-63cfff593066\") " pod="openshift-multus/multus-d7msb"
Oct 31 00:27:27.514772 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.514492    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/1a72f876-1157-4447-845c-923cdfd5aa5e-device-dir\") pod \"aws-ebs-csi-driver-node-k6fgp\" (UID: \"1a72f876-1157-4447-845c-923cdfd5aa5e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k6fgp"
Oct 31 00:27:27.514772 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.514492    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/6926e9b8-7c44-413e-bc7e-63cfff593066-host-run-multus-certs\") pod \"multus-d7msb\" (UID: \"6926e9b8-7c44-413e-bc7e-63cfff593066\") " pod="openshift-multus/multus-d7msb"
Oct 31 00:27:27.514772 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.514507    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/61c60450-eb50-4181-8ab7-e69e864ffb51-host\") pod \"node-ca-qchks\" (UID: \"61c60450-eb50-4181-8ab7-e69e864ffb51\") " pod="openshift-image-registry/node-ca-qchks"
Oct 31 00:27:27.514772 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.514511    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/21df46e4-2158-4385-ab9a-4b1790b64b55-etc-sysconfig\") pod \"tuned-hrsmn\" (UID: \"21df46e4-2158-4385-ab9a-4b1790b64b55\") " pod="openshift-cluster-node-tuning-operator/tuned-hrsmn"
Oct 31 00:27:27.514772 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.514523    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/63bedd38-6ab7-493d-b94c-b3c288af3f48-host-cni-bin\") pod \"ovnkube-node-rrk77\" (UID: \"63bedd38-6ab7-493d-b94c-b3c288af3f48\") " pod="openshift-ovn-kubernetes/ovnkube-node-rrk77"
Oct 31 00:27:27.514772 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.514540    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/63bedd38-6ab7-493d-b94c-b3c288af3f48-ovn-node-metrics-cert\") pod \"ovnkube-node-rrk77\" (UID: \"63bedd38-6ab7-493d-b94c-b3c288af3f48\") " pod="openshift-ovn-kubernetes/ovnkube-node-rrk77"
Oct 31 00:27:27.514772 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.514551    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/61c60450-eb50-4181-8ab7-e69e864ffb51-host\") pod \"node-ca-qchks\" (UID: \"61c60450-eb50-4181-8ab7-e69e864ffb51\") " pod="openshift-image-registry/node-ca-qchks"
Oct 31 00:27:27.514772 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.514555    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/21df46e4-2158-4385-ab9a-4b1790b64b55-etc-kubernetes\") pod \"tuned-hrsmn\" (UID: \"21df46e4-2158-4385-ab9a-4b1790b64b55\") " pod="openshift-cluster-node-tuning-operator/tuned-hrsmn"
Oct 31 00:27:27.514772 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.514561    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/1a72f876-1157-4447-845c-923cdfd5aa5e-device-dir\") pod \"aws-ebs-csi-driver-node-k6fgp\" (UID: \"1a72f876-1157-4447-845c-923cdfd5aa5e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k6fgp"
Oct 31 00:27:27.514772 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.514562    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x6bgk\" (UniqueName: \"kubernetes.io/projected/2c9341ef-0860-4fb6-8a86-272ac063756a-kube-api-access-x6bgk\") pod \"node-resolver-zjb6f\" (UID: \"2c9341ef-0860-4fb6-8a86-272ac063756a\") " pod="openshift-dns/node-resolver-zjb6f"
Oct 31 00:27:27.515698 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.514594    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/63bedd38-6ab7-493d-b94c-b3c288af3f48-host-cni-bin\") pod \"ovnkube-node-rrk77\" (UID: \"63bedd38-6ab7-493d-b94c-b3c288af3f48\") " pod="openshift-ovn-kubernetes/ovnkube-node-rrk77"
Oct 31 00:27:27.515698 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.514672    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/1a72f876-1157-4447-845c-923cdfd5aa5e-socket-dir\") pod \"aws-ebs-csi-driver-node-k6fgp\" (UID: \"1a72f876-1157-4447-845c-923cdfd5aa5e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k6fgp"
Oct 31 00:27:27.515698 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.514697    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xhp7k\" (UniqueName: \"kubernetes.io/projected/1a72f876-1157-4447-845c-923cdfd5aa5e-kube-api-access-xhp7k\") pod \"aws-ebs-csi-driver-node-k6fgp\" (UID: \"1a72f876-1157-4447-845c-923cdfd5aa5e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k6fgp"
Oct 31 00:27:27.515698 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.514717    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/63bedd38-6ab7-493d-b94c-b3c288af3f48-ovnkube-config\") pod \"ovnkube-node-rrk77\" (UID: \"63bedd38-6ab7-493d-b94c-b3c288af3f48\") " pod="openshift-ovn-kubernetes/ovnkube-node-rrk77"
Oct 31 00:27:27.515698 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.514724    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gm472\" (UniqueName: \"kubernetes.io/projected/21df46e4-2158-4385-ab9a-4b1790b64b55-kube-api-access-gm472\") pod \"tuned-hrsmn\" (UID: \"21df46e4-2158-4385-ab9a-4b1790b64b55\") " pod="openshift-cluster-node-tuning-operator/tuned-hrsmn"
Oct 31 00:27:27.515698 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.514748    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hxhdf\" (UniqueName: \"kubernetes.io/projected/c4ce4bad-af31-4f2e-a684-b2123e9da265-kube-api-access-hxhdf\") pod \"network-check-target-jktkx\" (UID: \"c4ce4bad-af31-4f2e-a684-b2123e9da265\") " pod="openshift-network-diagnostics/network-check-target-jktkx"
Oct 31 00:27:27.515698 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.514771    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/97410648-b7ae-4cf4-accf-3164c398b525-kubelet-config\") pod \"global-pull-secret-syncer-24sxq\" (UID: \"97410648-b7ae-4cf4-accf-3164c398b525\") " pod="kube-system/global-pull-secret-syncer-24sxq"
Oct 31 00:27:27.515698 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.514807    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/6926e9b8-7c44-413e-bc7e-63cfff593066-hostroot\") pod \"multus-d7msb\" (UID: \"6926e9b8-7c44-413e-bc7e-63cfff593066\") " pod="openshift-multus/multus-d7msb"
Oct 31 00:27:27.515698 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.514834    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kplpm\" (UniqueName: \"kubernetes.io/projected/85e4b969-79f8-4954-8539-0905bd5db99d-kube-api-access-kplpm\") pod \"network-metrics-daemon-plnkm\" (UID: \"85e4b969-79f8-4954-8539-0905bd5db99d\") " pod="openshift-multus/network-metrics-daemon-plnkm"
Oct 31 00:27:27.515698 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.514861    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vp9kn\" (UniqueName: \"kubernetes.io/projected/63bedd38-6ab7-493d-b94c-b3c288af3f48-kube-api-access-vp9kn\") pod \"ovnkube-node-rrk77\" (UID: \"63bedd38-6ab7-493d-b94c-b3c288af3f48\") " pod="openshift-ovn-kubernetes/ovnkube-node-rrk77"
Oct 31 00:27:27.515698 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.514884    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/1a72f876-1157-4447-845c-923cdfd5aa5e-etc-selinux\") pod \"aws-ebs-csi-driver-node-k6fgp\" (UID: \"1a72f876-1157-4447-845c-923cdfd5aa5e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k6fgp"
Oct 31 00:27:27.515698 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.514892    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/97410648-b7ae-4cf4-accf-3164c398b525-kubelet-config\") pod \"global-pull-secret-syncer-24sxq\" (UID: \"97410648-b7ae-4cf4-accf-3164c398b525\") " pod="kube-system/global-pull-secret-syncer-24sxq"
Oct 31 00:27:27.515698 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.514934    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/21df46e4-2158-4385-ab9a-4b1790b64b55-etc-systemd\") pod \"tuned-hrsmn\" (UID: \"21df46e4-2158-4385-ab9a-4b1790b64b55\") " pod="openshift-cluster-node-tuning-operator/tuned-hrsmn"
Oct 31 00:27:27.515698 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.514952    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/6926e9b8-7c44-413e-bc7e-63cfff593066-hostroot\") pod \"multus-d7msb\" (UID: \"6926e9b8-7c44-413e-bc7e-63cfff593066\") " pod="openshift-multus/multus-d7msb"
Oct 31 00:27:27.515698 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.514966    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/63bedd38-6ab7-493d-b94c-b3c288af3f48-host-run-netns\") pod \"ovnkube-node-rrk77\" (UID: \"63bedd38-6ab7-493d-b94c-b3c288af3f48\") " pod="openshift-ovn-kubernetes/ovnkube-node-rrk77"
Oct 31 00:27:27.515698 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.515056    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/63bedd38-6ab7-493d-b94c-b3c288af3f48-run-systemd\") pod \"ovnkube-node-rrk77\" (UID: \"63bedd38-6ab7-493d-b94c-b3c288af3f48\") " pod="openshift-ovn-kubernetes/ovnkube-node-rrk77"
Oct 31 00:27:27.515698 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.515078    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/63bedd38-6ab7-493d-b94c-b3c288af3f48-node-log\") pod \"ovnkube-node-rrk77\" (UID: \"63bedd38-6ab7-493d-b94c-b3c288af3f48\") " pod="openshift-ovn-kubernetes/ovnkube-node-rrk77"
Oct 31 00:27:27.516503 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.515085    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/63bedd38-6ab7-493d-b94c-b3c288af3f48-run-systemd\") pod \"ovnkube-node-rrk77\" (UID: \"63bedd38-6ab7-493d-b94c-b3c288af3f48\") " pod="openshift-ovn-kubernetes/ovnkube-node-rrk77"
Oct 31 00:27:27.516503 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.515103    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4xxhd\" (UniqueName: \"kubernetes.io/projected/9daae1e9-e437-437c-ba8b-3d9c206b64a3-kube-api-access-4xxhd\") pod \"iptables-alerter-p4txz\" (UID: \"9daae1e9-e437-437c-ba8b-3d9c206b64a3\") " pod="openshift-network-operator/iptables-alerter-p4txz"
Oct 31 00:27:27.516503 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.515111    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/63bedd38-6ab7-493d-b94c-b3c288af3f48-host-run-netns\") pod \"ovnkube-node-rrk77\" (UID: \"63bedd38-6ab7-493d-b94c-b3c288af3f48\") " pod="openshift-ovn-kubernetes/ovnkube-node-rrk77"
Oct 31 00:27:27.516503 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.515128    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/21df46e4-2158-4385-ab9a-4b1790b64b55-run\") pod \"tuned-hrsmn\" (UID: \"21df46e4-2158-4385-ab9a-4b1790b64b55\") " pod="openshift-cluster-node-tuning-operator/tuned-hrsmn"
Oct 31 00:27:27.516503 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.515131    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/63bedd38-6ab7-493d-b94c-b3c288af3f48-node-log\") pod \"ovnkube-node-rrk77\" (UID: \"63bedd38-6ab7-493d-b94c-b3c288af3f48\") " pod="openshift-ovn-kubernetes/ovnkube-node-rrk77"
Oct 31 00:27:27.516503 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.515130    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/1a72f876-1157-4447-845c-923cdfd5aa5e-etc-selinux\") pod \"aws-ebs-csi-driver-node-k6fgp\" (UID: \"1a72f876-1157-4447-845c-923cdfd5aa5e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k6fgp"
Oct 31 00:27:27.516503 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.515136    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/21df46e4-2158-4385-ab9a-4b1790b64b55-etc-systemd\") pod \"tuned-hrsmn\" (UID: \"21df46e4-2158-4385-ab9a-4b1790b64b55\") " pod="openshift-cluster-node-tuning-operator/tuned-hrsmn"
Oct 31 00:27:27.516503 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.515151    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/63bedd38-6ab7-493d-b94c-b3c288af3f48-run-ovn\") pod \"ovnkube-node-rrk77\" (UID: \"63bedd38-6ab7-493d-b94c-b3c288af3f48\") " pod="openshift-ovn-kubernetes/ovnkube-node-rrk77"
Oct 31 00:27:27.516503 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.515193    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/63bedd38-6ab7-493d-b94c-b3c288af3f48-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-rrk77\" (UID: \"63bedd38-6ab7-493d-b94c-b3c288af3f48\") " pod="openshift-ovn-kubernetes/ovnkube-node-rrk77"
Oct 31 00:27:27.516503 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.515217    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/41c4cc4b-b387-4c70-8a27-58255ba632b9-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-2t5fx\" (UID: \"41c4cc4b-b387-4c70-8a27-58255ba632b9\") " pod="openshift-multus/multus-additional-cni-plugins-2t5fx"
Oct 31 00:27:27.516503 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.515250    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6926e9b8-7c44-413e-bc7e-63cfff593066-cni-binary-copy\") pod \"multus-d7msb\" (UID: \"6926e9b8-7c44-413e-bc7e-63cfff593066\") " pod="openshift-multus/multus-d7msb"
Oct 31 00:27:27.516503 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.515256    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/21df46e4-2158-4385-ab9a-4b1790b64b55-run\") pod \"tuned-hrsmn\" (UID: \"21df46e4-2158-4385-ab9a-4b1790b64b55\") " pod="openshift-cluster-node-tuning-operator/tuned-hrsmn"
Oct 31 00:27:27.516503 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.515250    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/63bedd38-6ab7-493d-b94c-b3c288af3f48-run-ovn\") pod \"ovnkube-node-rrk77\" (UID: \"63bedd38-6ab7-493d-b94c-b3c288af3f48\") " pod="openshift-ovn-kubernetes/ovnkube-node-rrk77"
Oct 31 00:27:27.516503 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.515274    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/63bedd38-6ab7-493d-b94c-b3c288af3f48-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-rrk77\" (UID: \"63bedd38-6ab7-493d-b94c-b3c288af3f48\") " pod="openshift-ovn-kubernetes/ovnkube-node-rrk77"
Oct 31 00:27:27.516503 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.515316    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/85e4b969-79f8-4954-8539-0905bd5db99d-metrics-certs\") pod \"network-metrics-daemon-plnkm\" (UID: \"85e4b969-79f8-4954-8539-0905bd5db99d\") " pod="openshift-multus/network-metrics-daemon-plnkm"
Oct 31 00:27:27.516503 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.515343    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/1a72f876-1157-4447-845c-923cdfd5aa5e-registration-dir\") pod \"aws-ebs-csi-driver-node-k6fgp\" (UID: \"1a72f876-1157-4447-845c-923cdfd5aa5e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k6fgp"
Oct 31 00:27:27.516503 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.515360    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/21df46e4-2158-4385-ab9a-4b1790b64b55-var-lib-kubelet\") pod \"tuned-hrsmn\" (UID: \"21df46e4-2158-4385-ab9a-4b1790b64b55\") " pod="openshift-cluster-node-tuning-operator/tuned-hrsmn"
Oct 31 00:27:27.517363 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.515378    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/41c4cc4b-b387-4c70-8a27-58255ba632b9-tuning-conf-dir\") pod \"multus-additional-cni-plugins-2t5fx\" (UID: \"41c4cc4b-b387-4c70-8a27-58255ba632b9\") " pod="openshift-multus/multus-additional-cni-plugins-2t5fx"
Oct 31 00:27:27.517363 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.515403    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6926e9b8-7c44-413e-bc7e-63cfff593066-multus-cni-dir\") pod \"multus-d7msb\" (UID: \"6926e9b8-7c44-413e-bc7e-63cfff593066\") " pod="openshift-multus/multus-d7msb"
Oct 31 00:27:27.517363 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.515430    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6926e9b8-7c44-413e-bc7e-63cfff593066-cnibin\") pod \"multus-d7msb\" (UID: \"6926e9b8-7c44-413e-bc7e-63cfff593066\") " pod="openshift-multus/multus-d7msb"
Oct 31 00:27:27.517363 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.515434    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/1a72f876-1157-4447-845c-923cdfd5aa5e-socket-dir\") pod \"aws-ebs-csi-driver-node-k6fgp\" (UID: \"1a72f876-1157-4447-845c-923cdfd5aa5e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k6fgp"
Oct 31 00:27:27.517363 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.515483    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6926e9b8-7c44-413e-bc7e-63cfff593066-cnibin\") pod \"multus-d7msb\" (UID: \"6926e9b8-7c44-413e-bc7e-63cfff593066\") " pod="openshift-multus/multus-d7msb"
Oct 31 00:27:27.517363 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.515504    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/1a72f876-1157-4447-845c-923cdfd5aa5e-registration-dir\") pod \"aws-ebs-csi-driver-node-k6fgp\" (UID: \"1a72f876-1157-4447-845c-923cdfd5aa5e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k6fgp"
Oct 31 00:27:27.517363 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.515513    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6926e9b8-7c44-413e-bc7e-63cfff593066-host-run-netns\") pod \"multus-d7msb\" (UID: \"6926e9b8-7c44-413e-bc7e-63cfff593066\") " pod="openshift-multus/multus-d7msb"
Oct 31 00:27:27.517363 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.515532    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/21df46e4-2158-4385-ab9a-4b1790b64b55-var-lib-kubelet\") pod \"tuned-hrsmn\" (UID: \"21df46e4-2158-4385-ab9a-4b1790b64b55\") " pod="openshift-cluster-node-tuning-operator/tuned-hrsmn"
Oct 31 00:27:27.517363 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.515546    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6926e9b8-7c44-413e-bc7e-63cfff593066-multus-cni-dir\") pod \"multus-d7msb\" (UID: \"6926e9b8-7c44-413e-bc7e-63cfff593066\") " pod="openshift-multus/multus-d7msb"
Oct 31 00:27:27.517363 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.515582    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/9daae1e9-e437-437c-ba8b-3d9c206b64a3-iptables-alerter-script\") pod \"iptables-alerter-p4txz\" (UID: \"9daae1e9-e437-437c-ba8b-3d9c206b64a3\") " pod="openshift-network-operator/iptables-alerter-p4txz"
Oct 31 00:27:27.517363 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.515579    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6926e9b8-7c44-413e-bc7e-63cfff593066-host-run-netns\") pod \"multus-d7msb\" (UID: \"6926e9b8-7c44-413e-bc7e-63cfff593066\") " pod="openshift-multus/multus-d7msb"
Oct 31 00:27:27.517363 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.515607    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/41c4cc4b-b387-4c70-8a27-58255ba632b9-tuning-conf-dir\") pod \"multus-additional-cni-plugins-2t5fx\" (UID: \"41c4cc4b-b387-4c70-8a27-58255ba632b9\") " pod="openshift-multus/multus-additional-cni-plugins-2t5fx"
Oct 31 00:27:27.517363 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.515632    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/21df46e4-2158-4385-ab9a-4b1790b64b55-etc-tuned\") pod \"tuned-hrsmn\" (UID: \"21df46e4-2158-4385-ab9a-4b1790b64b55\") " pod="openshift-cluster-node-tuning-operator/tuned-hrsmn"
Oct 31 00:27:27.517363 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.515668    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/f93a119c-fb17-44bd-973c-699a3b87523b-konnectivity-ca\") pod \"konnectivity-agent-k2sk2\" (UID: \"f93a119c-fb17-44bd-973c-699a3b87523b\") " pod="kube-system/konnectivity-agent-k2sk2"
Oct 31 00:27:27.517363 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.515692    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/63bedd38-6ab7-493d-b94c-b3c288af3f48-log-socket\") pod \"ovnkube-node-rrk77\" (UID: \"63bedd38-6ab7-493d-b94c-b3c288af3f48\") " pod="openshift-ovn-kubernetes/ovnkube-node-rrk77"
Oct 31 00:27:27.517363 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.515738    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/63bedd38-6ab7-493d-b94c-b3c288af3f48-log-socket\") pod \"ovnkube-node-rrk77\" (UID: \"63bedd38-6ab7-493d-b94c-b3c288af3f48\") " pod="openshift-ovn-kubernetes/ovnkube-node-rrk77"
Oct 31 00:27:27.517363 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.515781    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/1a72f876-1157-4447-845c-923cdfd5aa5e-kubelet-dir\") pod \"aws-ebs-csi-driver-node-k6fgp\" (UID: \"1a72f876-1157-4447-845c-923cdfd5aa5e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k6fgp"
Oct 31 00:27:27.517363 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.515786    2572 swap_util.go:78] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Oct 31 00:27:27.517874 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.515814    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/63bedd38-6ab7-493d-b94c-b3c288af3f48-host-kubelet\") pod \"ovnkube-node-rrk77\" (UID: \"63bedd38-6ab7-493d-b94c-b3c288af3f48\") " pod="openshift-ovn-kubernetes/ovnkube-node-rrk77"
Oct 31 00:27:27.517874 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.515828    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/1a72f876-1157-4447-845c-923cdfd5aa5e-kubelet-dir\") pod \"aws-ebs-csi-driver-node-k6fgp\" (UID: \"1a72f876-1157-4447-845c-923cdfd5aa5e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k6fgp"
Oct 31 00:27:27.517874 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.515831    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/21df46e4-2158-4385-ab9a-4b1790b64b55-etc-sysctl-conf\") pod \"tuned-hrsmn\" (UID: \"21df46e4-2158-4385-ab9a-4b1790b64b55\") " pod="openshift-cluster-node-tuning-operator/tuned-hrsmn"
Oct 31 00:27:27.517874 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.515864    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/21df46e4-2158-4385-ab9a-4b1790b64b55-host\") pod \"tuned-hrsmn\" (UID: \"21df46e4-2158-4385-ab9a-4b1790b64b55\") " pod="openshift-cluster-node-tuning-operator/tuned-hrsmn"
Oct 31 00:27:27.517874 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.515887    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/63bedd38-6ab7-493d-b94c-b3c288af3f48-ovnkube-script-lib\") pod \"ovnkube-node-rrk77\" (UID: \"63bedd38-6ab7-493d-b94c-b3c288af3f48\") " pod="openshift-ovn-kubernetes/ovnkube-node-rrk77"
Oct 31 00:27:27.517874 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.515910    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m2phw\" (UniqueName: \"kubernetes.io/projected/6926e9b8-7c44-413e-bc7e-63cfff593066-kube-api-access-m2phw\") pod \"multus-d7msb\" (UID: \"6926e9b8-7c44-413e-bc7e-63cfff593066\") " pod="openshift-multus/multus-d7msb"
Oct 31 00:27:27.517874 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.515918    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/21df46e4-2158-4385-ab9a-4b1790b64b55-etc-sysctl-conf\") pod \"tuned-hrsmn\" (UID: \"21df46e4-2158-4385-ab9a-4b1790b64b55\") " pod="openshift-cluster-node-tuning-operator/tuned-hrsmn"
Oct 31 00:27:27.517874 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.515921    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/63bedd38-6ab7-493d-b94c-b3c288af3f48-host-kubelet\") pod \"ovnkube-node-rrk77\" (UID: \"63bedd38-6ab7-493d-b94c-b3c288af3f48\") " pod="openshift-ovn-kubernetes/ovnkube-node-rrk77"
Oct 31 00:27:27.517874 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.515931    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/f93a119c-fb17-44bd-973c-699a3b87523b-agent-certs\") pod \"konnectivity-agent-k2sk2\" (UID: \"f93a119c-fb17-44bd-973c-699a3b87523b\") " pod="kube-system/konnectivity-agent-k2sk2"
Oct 31 00:27:27.517874 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.515943    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/21df46e4-2158-4385-ab9a-4b1790b64b55-host\") pod \"tuned-hrsmn\" (UID: \"21df46e4-2158-4385-ab9a-4b1790b64b55\") " pod="openshift-cluster-node-tuning-operator/tuned-hrsmn"
Oct 31 00:27:27.517874 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.515953    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/63bedd38-6ab7-493d-b94c-b3c288af3f48-etc-openvswitch\") pod \"ovnkube-node-rrk77\" (UID: \"63bedd38-6ab7-493d-b94c-b3c288af3f48\") " pod="openshift-ovn-kubernetes/ovnkube-node-rrk77"
Oct 31 00:27:27.517874 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.515978    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6926e9b8-7c44-413e-bc7e-63cfff593066-host-var-lib-cni-bin\") pod \"multus-d7msb\" (UID: \"6926e9b8-7c44-413e-bc7e-63cfff593066\") " pod="openshift-multus/multus-d7msb"
Oct 31 00:27:27.517874 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.516002    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sslf9\" (UniqueName: \"kubernetes.io/projected/61c60450-eb50-4181-8ab7-e69e864ffb51-kube-api-access-sslf9\") pod \"node-ca-qchks\" (UID: \"61c60450-eb50-4181-8ab7-e69e864ffb51\") " pod="openshift-image-registry/node-ca-qchks"
Oct 31 00:27:27.517874 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.516027    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/63bedd38-6ab7-493d-b94c-b3c288af3f48-systemd-units\") pod \"ovnkube-node-rrk77\" (UID: \"63bedd38-6ab7-493d-b94c-b3c288af3f48\") " pod="openshift-ovn-kubernetes/ovnkube-node-rrk77"
Oct 31 00:27:27.517874 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.516030    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/63bedd38-6ab7-493d-b94c-b3c288af3f48-etc-openvswitch\") pod \"ovnkube-node-rrk77\" (UID: \"63bedd38-6ab7-493d-b94c-b3c288af3f48\") " pod="openshift-ovn-kubernetes/ovnkube-node-rrk77"
Oct 31 00:27:27.517874 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.516068    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/97410648-b7ae-4cf4-accf-3164c398b525-original-pull-secret\") pod \"global-pull-secret-syncer-24sxq\" (UID: \"97410648-b7ae-4cf4-accf-3164c398b525\") " pod="kube-system/global-pull-secret-syncer-24sxq"
Oct 31 00:27:27.517874 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.516076    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/63bedd38-6ab7-493d-b94c-b3c288af3f48-systemd-units\") pod \"ovnkube-node-rrk77\" (UID: \"63bedd38-6ab7-493d-b94c-b3c288af3f48\") " pod="openshift-ovn-kubernetes/ovnkube-node-rrk77"
Oct 31 00:27:27.517874 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.516040    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6926e9b8-7c44-413e-bc7e-63cfff593066-host-var-lib-cni-bin\") pod \"multus-d7msb\" (UID: \"6926e9b8-7c44-413e-bc7e-63cfff593066\") " pod="openshift-multus/multus-d7msb"
Oct 31 00:27:27.518386 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.516099    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/6926e9b8-7c44-413e-bc7e-63cfff593066-multus-socket-dir-parent\") pod \"multus-d7msb\" (UID: \"6926e9b8-7c44-413e-bc7e-63cfff593066\") " pod="openshift-multus/multus-d7msb"
Oct 31 00:27:27.518386 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.516129    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6926e9b8-7c44-413e-bc7e-63cfff593066-multus-conf-dir\") pod \"multus-d7msb\" (UID: \"6926e9b8-7c44-413e-bc7e-63cfff593066\") " pod="openshift-multus/multus-d7msb"
Oct 31 00:27:27.518386 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.516147    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/6926e9b8-7c44-413e-bc7e-63cfff593066-multus-socket-dir-parent\") pod \"multus-d7msb\" (UID: \"6926e9b8-7c44-413e-bc7e-63cfff593066\") " pod="openshift-multus/multus-d7msb"
Oct 31 00:27:27.518386 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.516155    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/97410648-b7ae-4cf4-accf-3164c398b525-dbus\") pod \"global-pull-secret-syncer-24sxq\" (UID: \"97410648-b7ae-4cf4-accf-3164c398b525\") " pod="kube-system/global-pull-secret-syncer-24sxq"
Oct 31 00:27:27.518386 ip-10-0-132-173 kubenswrapper[2572]: E1031 00:27:27.516192    2572 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 31 00:27:27.518386 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.516214    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2c9341ef-0860-4fb6-8a86-272ac063756a-tmp-dir\") pod \"node-resolver-zjb6f\" (UID: \"2c9341ef-0860-4fb6-8a86-272ac063756a\") " pod="openshift-dns/node-resolver-zjb6f"
Oct 31 00:27:27.518386 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.516259    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/1a72f876-1157-4447-845c-923cdfd5aa5e-sys-fs\") pod \"aws-ebs-csi-driver-node-k6fgp\" (UID: \"1a72f876-1157-4447-845c-923cdfd5aa5e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k6fgp"
Oct 31 00:27:27.518386 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.516195    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6926e9b8-7c44-413e-bc7e-63cfff593066-multus-conf-dir\") pod \"multus-d7msb\" (UID: \"6926e9b8-7c44-413e-bc7e-63cfff593066\") " pod="openshift-multus/multus-d7msb"
Oct 31 00:27:27.518386 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.516306    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/1a72f876-1157-4447-845c-923cdfd5aa5e-sys-fs\") pod \"aws-ebs-csi-driver-node-k6fgp\" (UID: \"1a72f876-1157-4447-845c-923cdfd5aa5e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k6fgp"
Oct 31 00:27:27.518386 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.516215    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/97410648-b7ae-4cf4-accf-3164c398b525-dbus\") pod \"global-pull-secret-syncer-24sxq\" (UID: \"97410648-b7ae-4cf4-accf-3164c398b525\") " pod="kube-system/global-pull-secret-syncer-24sxq"
Oct 31 00:27:27.518386 ip-10-0-132-173 kubenswrapper[2572]: E1031 00:27:27.516333    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/97410648-b7ae-4cf4-accf-3164c398b525-original-pull-secret podName:97410648-b7ae-4cf4-accf-3164c398b525 nodeName:}" failed. No retries permitted until 2026-10-31 00:27:28.016273078 +0000 UTC m=+3.116815913 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/97410648-b7ae-4cf4-accf-3164c398b525-original-pull-secret") pod "global-pull-secret-syncer-24sxq" (UID: "97410648-b7ae-4cf4-accf-3164c398b525") : object "kube-system"/"original-pull-secret" not registered
Oct 31 00:27:27.518386 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.516356    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/21df46e4-2158-4385-ab9a-4b1790b64b55-tmp\") pod \"tuned-hrsmn\" (UID: \"21df46e4-2158-4385-ab9a-4b1790b64b55\") " pod="openshift-cluster-node-tuning-operator/tuned-hrsmn"
Oct 31 00:27:27.518386 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.516387    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/63bedd38-6ab7-493d-b94c-b3c288af3f48-host-slash\") pod \"ovnkube-node-rrk77\" (UID: \"63bedd38-6ab7-493d-b94c-b3c288af3f48\") " pod="openshift-ovn-kubernetes/ovnkube-node-rrk77"
Oct 31 00:27:27.518386 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.516416    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/63bedd38-6ab7-493d-b94c-b3c288af3f48-ovnkube-script-lib\") pod \"ovnkube-node-rrk77\" (UID: \"63bedd38-6ab7-493d-b94c-b3c288af3f48\") " pod="openshift-ovn-kubernetes/ovnkube-node-rrk77"
Oct 31 00:27:27.518386 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.516437    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/41c4cc4b-b387-4c70-8a27-58255ba632b9-cnibin\") pod \"multus-additional-cni-plugins-2t5fx\" (UID: \"41c4cc4b-b387-4c70-8a27-58255ba632b9\") " pod="openshift-multus/multus-additional-cni-plugins-2t5fx"
Oct 31 00:27:27.518386 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.516472    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/63bedd38-6ab7-493d-b94c-b3c288af3f48-host-slash\") pod \"ovnkube-node-rrk77\" (UID: \"63bedd38-6ab7-493d-b94c-b3c288af3f48\") " pod="openshift-ovn-kubernetes/ovnkube-node-rrk77"
Oct 31 00:27:27.518386 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.516466    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/41c4cc4b-b387-4c70-8a27-58255ba632b9-cni-binary-copy\") pod \"multus-additional-cni-plugins-2t5fx\" (UID: \"41c4cc4b-b387-4c70-8a27-58255ba632b9\") " pod="openshift-multus/multus-additional-cni-plugins-2t5fx"
Oct 31 00:27:27.518850 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.516503    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2c9341ef-0860-4fb6-8a86-272ac063756a-tmp-dir\") pod \"node-resolver-zjb6f\" (UID: \"2c9341ef-0860-4fb6-8a86-272ac063756a\") " pod="openshift-dns/node-resolver-zjb6f"
Oct 31 00:27:27.518850 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.516519    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vrldg\" (UniqueName: \"kubernetes.io/projected/41c4cc4b-b387-4c70-8a27-58255ba632b9-kube-api-access-vrldg\") pod \"multus-additional-cni-plugins-2t5fx\" (UID: \"41c4cc4b-b387-4c70-8a27-58255ba632b9\") " pod="openshift-multus/multus-additional-cni-plugins-2t5fx"
Oct 31 00:27:27.518850 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.516532    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/41c4cc4b-b387-4c70-8a27-58255ba632b9-cnibin\") pod \"multus-additional-cni-plugins-2t5fx\" (UID: \"41c4cc4b-b387-4c70-8a27-58255ba632b9\") " pod="openshift-multus/multus-additional-cni-plugins-2t5fx"
Oct 31 00:27:27.518850 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.516545    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/61c60450-eb50-4181-8ab7-e69e864ffb51-serviceca\") pod \"node-ca-qchks\" (UID: \"61c60450-eb50-4181-8ab7-e69e864ffb51\") " pod="openshift-image-registry/node-ca-qchks"
Oct 31 00:27:27.518850 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.516601    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/63bedd38-6ab7-493d-b94c-b3c288af3f48-var-lib-openvswitch\") pod \"ovnkube-node-rrk77\" (UID: \"63bedd38-6ab7-493d-b94c-b3c288af3f48\") " pod="openshift-ovn-kubernetes/ovnkube-node-rrk77"
Oct 31 00:27:27.518850 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.516633    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/63bedd38-6ab7-493d-b94c-b3c288af3f48-run-openvswitch\") pod \"ovnkube-node-rrk77\" (UID: \"63bedd38-6ab7-493d-b94c-b3c288af3f48\") " pod="openshift-ovn-kubernetes/ovnkube-node-rrk77"
Oct 31 00:27:27.518850 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.516635    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/63bedd38-6ab7-493d-b94c-b3c288af3f48-var-lib-openvswitch\") pod \"ovnkube-node-rrk77\" (UID: \"63bedd38-6ab7-493d-b94c-b3c288af3f48\") " pod="openshift-ovn-kubernetes/ovnkube-node-rrk77"
Oct 31 00:27:27.518850 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.516661    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/63bedd38-6ab7-493d-b94c-b3c288af3f48-host-cni-netd\") pod \"ovnkube-node-rrk77\" (UID: \"63bedd38-6ab7-493d-b94c-b3c288af3f48\") " pod="openshift-ovn-kubernetes/ovnkube-node-rrk77"
Oct 31 00:27:27.518850 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.516678    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/63bedd38-6ab7-493d-b94c-b3c288af3f48-run-openvswitch\") pod \"ovnkube-node-rrk77\" (UID: \"63bedd38-6ab7-493d-b94c-b3c288af3f48\") " pod="openshift-ovn-kubernetes/ovnkube-node-rrk77"
Oct 31 00:27:27.518850 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.516686    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6926e9b8-7c44-413e-bc7e-63cfff593066-system-cni-dir\") pod \"multus-d7msb\" (UID: \"6926e9b8-7c44-413e-bc7e-63cfff593066\") " pod="openshift-multus/multus-d7msb"
Oct 31 00:27:27.518850 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.516717    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/63bedd38-6ab7-493d-b94c-b3c288af3f48-host-cni-netd\") pod \"ovnkube-node-rrk77\" (UID: \"63bedd38-6ab7-493d-b94c-b3c288af3f48\") " pod="openshift-ovn-kubernetes/ovnkube-node-rrk77"
Oct 31 00:27:27.518850 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.516721    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/6926e9b8-7c44-413e-bc7e-63cfff593066-host-run-k8s-cni-cncf-io\") pod \"multus-d7msb\" (UID: \"6926e9b8-7c44-413e-bc7e-63cfff593066\") " pod="openshift-multus/multus-d7msb"
Oct 31 00:27:27.518850 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.516747    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/6926e9b8-7c44-413e-bc7e-63cfff593066-host-var-lib-cni-multus\") pod \"multus-d7msb\" (UID: \"6926e9b8-7c44-413e-bc7e-63cfff593066\") " pod="openshift-multus/multus-d7msb"
Oct 31 00:27:27.518850 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.516764    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6926e9b8-7c44-413e-bc7e-63cfff593066-system-cni-dir\") pod \"multus-d7msb\" (UID: \"6926e9b8-7c44-413e-bc7e-63cfff593066\") " pod="openshift-multus/multus-d7msb"
Oct 31 00:27:27.518850 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.516773    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/9daae1e9-e437-437c-ba8b-3d9c206b64a3-host-slash\") pod \"iptables-alerter-p4txz\" (UID: \"9daae1e9-e437-437c-ba8b-3d9c206b64a3\") " pod="openshift-network-operator/iptables-alerter-p4txz"
Oct 31 00:27:27.518850 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.516778    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/6926e9b8-7c44-413e-bc7e-63cfff593066-host-run-k8s-cni-cncf-io\") pod \"multus-d7msb\" (UID: \"6926e9b8-7c44-413e-bc7e-63cfff593066\") " pod="openshift-multus/multus-d7msb"
Oct 31 00:27:27.518850 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.516801    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/63bedd38-6ab7-493d-b94c-b3c288af3f48-env-overrides\") pod \"ovnkube-node-rrk77\" (UID: \"63bedd38-6ab7-493d-b94c-b3c288af3f48\") " pod="openshift-ovn-kubernetes/ovnkube-node-rrk77"
Oct 31 00:27:27.518850 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.516807    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/6926e9b8-7c44-413e-bc7e-63cfff593066-host-var-lib-cni-multus\") pod \"multus-d7msb\" (UID: \"6926e9b8-7c44-413e-bc7e-63cfff593066\") " pod="openshift-multus/multus-d7msb"
Oct 31 00:27:27.519366 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.516853    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/9daae1e9-e437-437c-ba8b-3d9c206b64a3-host-slash\") pod \"iptables-alerter-p4txz\" (UID: \"9daae1e9-e437-437c-ba8b-3d9c206b64a3\") " pod="openshift-network-operator/iptables-alerter-p4txz"
Oct 31 00:27:27.519366 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.517213    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/63bedd38-6ab7-493d-b94c-b3c288af3f48-env-overrides\") pod \"ovnkube-node-rrk77\" (UID: \"63bedd38-6ab7-493d-b94c-b3c288af3f48\") " pod="openshift-ovn-kubernetes/ovnkube-node-rrk77"
Oct 31 00:27:27.519678 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.519658    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/21df46e4-2158-4385-ab9a-4b1790b64b55-etc-tuned\") pod \"tuned-hrsmn\" (UID: \"21df46e4-2158-4385-ab9a-4b1790b64b55\") " pod="openshift-cluster-node-tuning-operator/tuned-hrsmn"
Oct 31 00:27:27.519753 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.519685    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/63bedd38-6ab7-493d-b94c-b3c288af3f48-ovn-node-metrics-cert\") pod \"ovnkube-node-rrk77\" (UID: \"63bedd38-6ab7-493d-b94c-b3c288af3f48\") " pod="openshift-ovn-kubernetes/ovnkube-node-rrk77"
Oct 31 00:27:27.519753 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.519693    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/21df46e4-2158-4385-ab9a-4b1790b64b55-tmp\") pod \"tuned-hrsmn\" (UID: \"21df46e4-2158-4385-ab9a-4b1790b64b55\") " pod="openshift-cluster-node-tuning-operator/tuned-hrsmn"
Oct 31 00:27:27.543882 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.543860    2572 reflector.go:446] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Oct 31 00:27:27.543962 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.543911    2572 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-4k4l2"
Oct 31 00:27:27.549773 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.549753    2572 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Oct 31 00:27:27.550018 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.549972    2572 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Oct 31 00:27:27.550018 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.549977    2572 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent-dockercfg-78fsr\""
Oct 31 00:27:27.550018 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.549988    2572 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-46b97\""
Oct 31 00:27:27.550018 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.549997    2572 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Oct 31 00:27:27.550240 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.550179    2572 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Oct 31 00:27:27.550240 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.550193    2572 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-node-identity-dockercfg-lwt4s\""
Oct 31 00:27:27.550240 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.550197    2572 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Oct 31 00:27:27.550240 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.550208    2572 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-rlgrr\""
Oct 31 00:27:27.550240 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.550232    2572 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Oct 31 00:27:27.550240 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.550238    2572 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-jsqnl\""
Oct 31 00:27:27.550553 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.550273    2572 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Oct 31 00:27:27.550553 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.550276    2572 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Oct 31 00:27:27.550553 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.550285    2572 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Oct 31 00:27:27.550553 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.550362    2572 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Oct 31 00:27:27.550553 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.550370    2572 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-cdw8b\""
Oct 31 00:27:27.550553 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.550416    2572 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Oct 31 00:27:27.550553 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.550430    2572 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-p2znq\""
Oct 31 00:27:27.550553 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.550432    2572 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Oct 31 00:27:27.550553 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.550459    2572 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Oct 31 00:27:27.550553 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.550470    2572 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Oct 31 00:27:27.550553 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.550475    2572 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Oct 31 00:27:27.550553 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.550498    2572 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Oct 31 00:27:27.550553 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.550503    2572 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Oct 31 00:27:27.550553 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.550514    2572 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Oct 31 00:27:27.550553 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.550521    2572 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Oct 31 00:27:27.550553 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.550541    2572 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Oct 31 00:27:27.555347 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.555328    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/6926e9b8-7c44-413e-bc7e-63cfff593066-multus-daemon-config\") pod \"multus-d7msb\" (UID: \"6926e9b8-7c44-413e-bc7e-63cfff593066\") " pod="openshift-multus/multus-d7msb"
Oct 31 00:27:27.555421 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.555404    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/41c4cc4b-b387-4c70-8a27-58255ba632b9-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-2t5fx\" (UID: \"41c4cc4b-b387-4c70-8a27-58255ba632b9\") " pod="openshift-multus/multus-additional-cni-plugins-2t5fx"
Oct 31 00:27:27.555788 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.555769    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6926e9b8-7c44-413e-bc7e-63cfff593066-cni-binary-copy\") pod \"multus-d7msb\" (UID: \"6926e9b8-7c44-413e-bc7e-63cfff593066\") " pod="openshift-multus/multus-d7msb"
Oct 31 00:27:27.555843 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.555825    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/41c4cc4b-b387-4c70-8a27-58255ba632b9-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-2t5fx\" (UID: \"41c4cc4b-b387-4c70-8a27-58255ba632b9\") " pod="openshift-multus/multus-additional-cni-plugins-2t5fx"
Oct 31 00:27:27.556159 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.556140    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/9daae1e9-e437-437c-ba8b-3d9c206b64a3-iptables-alerter-script\") pod \"iptables-alerter-p4txz\" (UID: \"9daae1e9-e437-437c-ba8b-3d9c206b64a3\") " pod="openshift-network-operator/iptables-alerter-p4txz"
Oct 31 00:27:27.556214 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.556185    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/f93a119c-fb17-44bd-973c-699a3b87523b-konnectivity-ca\") pod \"konnectivity-agent-k2sk2\" (UID: \"f93a119c-fb17-44bd-973c-699a3b87523b\") " pod="kube-system/konnectivity-agent-k2sk2"
Oct 31 00:27:27.557508 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.557485    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/41c4cc4b-b387-4c70-8a27-58255ba632b9-cni-binary-copy\") pod \"multus-additional-cni-plugins-2t5fx\" (UID: \"41c4cc4b-b387-4c70-8a27-58255ba632b9\") " pod="openshift-multus/multus-additional-cni-plugins-2t5fx"
Oct 31 00:27:27.557598 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.557486    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/61c60450-eb50-4181-8ab7-e69e864ffb51-serviceca\") pod \"node-ca-qchks\" (UID: \"61c60450-eb50-4181-8ab7-e69e864ffb51\") " pod="openshift-image-registry/node-ca-qchks"
Oct 31 00:27:27.558306 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.558273    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/f93a119c-fb17-44bd-973c-699a3b87523b-agent-certs\") pod \"konnectivity-agent-k2sk2\" (UID: \"f93a119c-fb17-44bd-973c-699a3b87523b\") " pod="kube-system/konnectivity-agent-k2sk2"
Oct 31 00:27:27.575648 ip-10-0-132-173 kubenswrapper[2572]: E1031 00:27:27.575623    2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 31 00:27:27.575648 ip-10-0-132-173 kubenswrapper[2572]: E1031 00:27:27.575650    2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 31 00:27:27.575787 ip-10-0-132-173 kubenswrapper[2572]: E1031 00:27:27.575665    2572 projected.go:196] Error preparing data for projected volume kube-api-access-hxhdf for pod openshift-network-diagnostics/network-check-target-jktkx: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 31 00:27:27.575787 ip-10-0-132-173 kubenswrapper[2572]: E1031 00:27:27.575745    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c4ce4bad-af31-4f2e-a684-b2123e9da265-kube-api-access-hxhdf podName:c4ce4bad-af31-4f2e-a684-b2123e9da265 nodeName:}" failed. No retries permitted until 2026-10-31 00:27:28.075725599 +0000 UTC m=+3.176268431 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-hxhdf" (UniqueName: "kubernetes.io/projected/c4ce4bad-af31-4f2e-a684-b2123e9da265-kube-api-access-hxhdf") pod "network-check-target-jktkx" (UID: "c4ce4bad-af31-4f2e-a684-b2123e9da265") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 31 00:27:27.577512 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.577491    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x6bgk\" (UniqueName: \"kubernetes.io/projected/2c9341ef-0860-4fb6-8a86-272ac063756a-kube-api-access-x6bgk\") pod \"node-resolver-zjb6f\" (UID: \"2c9341ef-0860-4fb6-8a86-272ac063756a\") " pod="openshift-dns/node-resolver-zjb6f"
Oct 31 00:27:27.578628 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.578598    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gm472\" (UniqueName: \"kubernetes.io/projected/21df46e4-2158-4385-ab9a-4b1790b64b55-kube-api-access-gm472\") pod \"tuned-hrsmn\" (UID: \"21df46e4-2158-4385-ab9a-4b1790b64b55\") " pod="openshift-cluster-node-tuning-operator/tuned-hrsmn"
Oct 31 00:27:27.578792 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.578773    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xhp7k\" (UniqueName: \"kubernetes.io/projected/1a72f876-1157-4447-845c-923cdfd5aa5e-kube-api-access-xhp7k\") pod \"aws-ebs-csi-driver-node-k6fgp\" (UID: \"1a72f876-1157-4447-845c-923cdfd5aa5e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k6fgp"
Oct 31 00:27:27.578854 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.578789    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4xxhd\" (UniqueName: \"kubernetes.io/projected/9daae1e9-e437-437c-ba8b-3d9c206b64a3-kube-api-access-4xxhd\") pod \"iptables-alerter-p4txz\" (UID: \"9daae1e9-e437-437c-ba8b-3d9c206b64a3\") " pod="openshift-network-operator/iptables-alerter-p4txz"
Oct 31 00:27:27.578854 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.578782    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vp9kn\" (UniqueName: \"kubernetes.io/projected/63bedd38-6ab7-493d-b94c-b3c288af3f48-kube-api-access-vp9kn\") pod \"ovnkube-node-rrk77\" (UID: \"63bedd38-6ab7-493d-b94c-b3c288af3f48\") " pod="openshift-ovn-kubernetes/ovnkube-node-rrk77"
Oct 31 00:27:27.579140 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.579125    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vrldg\" (UniqueName: \"kubernetes.io/projected/41c4cc4b-b387-4c70-8a27-58255ba632b9-kube-api-access-vrldg\") pod \"multus-additional-cni-plugins-2t5fx\" (UID: \"41c4cc4b-b387-4c70-8a27-58255ba632b9\") " pod="openshift-multus/multus-additional-cni-plugins-2t5fx"
Oct 31 00:27:27.579211 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.579195    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sslf9\" (UniqueName: \"kubernetes.io/projected/61c60450-eb50-4181-8ab7-e69e864ffb51-kube-api-access-sslf9\") pod \"node-ca-qchks\" (UID: \"61c60450-eb50-4181-8ab7-e69e864ffb51\") " pod="openshift-image-registry/node-ca-qchks"
Oct 31 00:27:27.583128 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.583109    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m2phw\" (UniqueName: \"kubernetes.io/projected/6926e9b8-7c44-413e-bc7e-63cfff593066-kube-api-access-m2phw\") pod \"multus-d7msb\" (UID: \"6926e9b8-7c44-413e-bc7e-63cfff593066\") " pod="openshift-multus/multus-d7msb"
Oct 31 00:27:27.617806 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.617775    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kplpm\" (UniqueName: \"kubernetes.io/projected/85e4b969-79f8-4954-8539-0905bd5db99d-kube-api-access-kplpm\") pod \"network-metrics-daemon-plnkm\" (UID: \"85e4b969-79f8-4954-8539-0905bd5db99d\") " pod="openshift-multus/network-metrics-daemon-plnkm"
Oct 31 00:27:27.617921 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.617816    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/85e4b969-79f8-4954-8539-0905bd5db99d-metrics-certs\") pod \"network-metrics-daemon-plnkm\" (UID: \"85e4b969-79f8-4954-8539-0905bd5db99d\") " pod="openshift-multus/network-metrics-daemon-plnkm"
Oct 31 00:27:27.617976 ip-10-0-132-173 kubenswrapper[2572]: E1031 00:27:27.617932    2572 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 31 00:27:27.618047 ip-10-0-132-173 kubenswrapper[2572]: E1031 00:27:27.618014    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/85e4b969-79f8-4954-8539-0905bd5db99d-metrics-certs podName:85e4b969-79f8-4954-8539-0905bd5db99d nodeName:}" failed. No retries permitted until 2026-10-31 00:27:28.117994684 +0000 UTC m=+3.218537496 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/85e4b969-79f8-4954-8539-0905bd5db99d-metrics-certs") pod "network-metrics-daemon-plnkm" (UID: "85e4b969-79f8-4954-8539-0905bd5db99d") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 31 00:27:27.684823 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.684766    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kplpm\" (UniqueName: \"kubernetes.io/projected/85e4b969-79f8-4954-8539-0905bd5db99d-kube-api-access-kplpm\") pod \"network-metrics-daemon-plnkm\" (UID: \"85e4b969-79f8-4954-8539-0905bd5db99d\") " pod="openshift-multus/network-metrics-daemon-plnkm"
Oct 31 00:27:27.706519 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.706496    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-rrk77"
Oct 31 00:27:27.707622 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.707606    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-zjb6f"
Oct 31 00:27:27.716019 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:27.715994    2572 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod63bedd38_6ab7_493d_b94c_b3c288af3f48.slice/crio-a17b6e11450f47a75f14e2684c8652ce4cc819ae80934ec6121b0c93e6af2407 WatchSource:0}: Error finding container a17b6e11450f47a75f14e2684c8652ce4cc819ae80934ec6121b0c93e6af2407: Status 404 returned error can't find the container with id a17b6e11450f47a75f14e2684c8652ce4cc819ae80934ec6121b0c93e6af2407
Oct 31 00:27:27.718448 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.718432    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-2t5fx"
Oct 31 00:27:27.718505 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.718457    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-d7msb"
Oct 31 00:27:27.718714 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:27.718695    2572 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2c9341ef_0860_4fb6_8a86_272ac063756a.slice/crio-9c4c014bdeff186fadbbba83b7498c289b19af588e40e2bf964ae400fb92f5eb WatchSource:0}: Error finding container 9c4c014bdeff186fadbbba83b7498c289b19af588e40e2bf964ae400fb92f5eb: Status 404 returned error can't find the container with id 9c4c014bdeff186fadbbba83b7498c289b19af588e40e2bf964ae400fb92f5eb
Oct 31 00:27:27.723578 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.723564    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-p4txz"
Oct 31 00:27:27.730900 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:27.729699    2572 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod41c4cc4b_b387_4c70_8a27_58255ba632b9.slice/crio-1188e094eabb027d1abb7b9851edccf1d8c7f701d4ac9866b902992f255fdb47 WatchSource:0}: Error finding container 1188e094eabb027d1abb7b9851edccf1d8c7f701d4ac9866b902992f255fdb47: Status 404 returned error can't find the container with id 1188e094eabb027d1abb7b9851edccf1d8c7f701d4ac9866b902992f255fdb47
Oct 31 00:27:27.731591 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.731560    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k6fgp"
Oct 31 00:27:27.732925 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:27.732902    2572 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6926e9b8_7c44_413e_bc7e_63cfff593066.slice/crio-6044eeab0fcad7ea4b7fc0c1977044cd6744c491b566b1cd496034659c3f2490 WatchSource:0}: Error finding container 6044eeab0fcad7ea4b7fc0c1977044cd6744c491b566b1cd496034659c3f2490: Status 404 returned error can't find the container with id 6044eeab0fcad7ea4b7fc0c1977044cd6744c491b566b1cd496034659c3f2490
Oct 31 00:27:27.735180 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.735163    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-hrsmn"
Oct 31 00:27:27.735323 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:27.735293    2572 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9daae1e9_e437_437c_ba8b_3d9c206b64a3.slice/crio-e0c303a73e7bc56cbce47921d95d76eeabf37b5c2954b84d3cf1cb7f2d06787d WatchSource:0}: Error finding container e0c303a73e7bc56cbce47921d95d76eeabf37b5c2954b84d3cf1cb7f2d06787d: Status 404 returned error can't find the container with id e0c303a73e7bc56cbce47921d95d76eeabf37b5c2954b84d3cf1cb7f2d06787d
Oct 31 00:27:27.741333 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.741305    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-qchks"
Oct 31 00:27:27.742070 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:27.741967    2572 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1a72f876_1157_4447_845c_923cdfd5aa5e.slice/crio-eb65fa9a666c91c9ac2ffe7057788f9538e79d38857116a648ae44c11f219605 WatchSource:0}: Error finding container eb65fa9a666c91c9ac2ffe7057788f9538e79d38857116a648ae44c11f219605: Status 404 returned error can't find the container with id eb65fa9a666c91c9ac2ffe7057788f9538e79d38857116a648ae44c11f219605
Oct 31 00:27:27.744388 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:27.744257    2572 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod21df46e4_2158_4385_ab9a_4b1790b64b55.slice/crio-6dadfdf70d702465b2d703308d22e1030445b10a6bdff2e6ed1f87d6eb4d529c WatchSource:0}: Error finding container 6dadfdf70d702465b2d703308d22e1030445b10a6bdff2e6ed1f87d6eb4d529c: Status 404 returned error can't find the container with id 6dadfdf70d702465b2d703308d22e1030445b10a6bdff2e6ed1f87d6eb4d529c
Oct 31 00:27:27.746795 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:27.746775    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-k2sk2"
Oct 31 00:27:27.749452 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:27.749401    2572 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod61c60450_eb50_4181_8ab7_e69e864ffb51.slice/crio-8f0c1b598b3f2bc9b83fd8a0d75ad2e544d28447ea8afcd04398bee142bd99db WatchSource:0}: Error finding container 8f0c1b598b3f2bc9b83fd8a0d75ad2e544d28447ea8afcd04398bee142bd99db: Status 404 returned error can't find the container with id 8f0c1b598b3f2bc9b83fd8a0d75ad2e544d28447ea8afcd04398bee142bd99db
Oct 31 00:27:27.754910 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:27.754892    2572 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf93a119c_fb17_44bd_973c_699a3b87523b.slice/crio-cf5491ca921b7277d410f4b081d5adf249e326fbdb3c5f39211ac00a72bfd7fb WatchSource:0}: Error finding container cf5491ca921b7277d410f4b081d5adf249e326fbdb3c5f39211ac00a72bfd7fb: Status 404 returned error can't find the container with id cf5491ca921b7277d410f4b081d5adf249e326fbdb3c5f39211ac00a72bfd7fb
Oct 31 00:27:28.021056 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:28.020977    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/97410648-b7ae-4cf4-accf-3164c398b525-original-pull-secret\") pod \"global-pull-secret-syncer-24sxq\" (UID: \"97410648-b7ae-4cf4-accf-3164c398b525\") " pod="kube-system/global-pull-secret-syncer-24sxq"
Oct 31 00:27:28.021190 ip-10-0-132-173 kubenswrapper[2572]: E1031 00:27:28.021131    2572 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 31 00:27:28.021264 ip-10-0-132-173 kubenswrapper[2572]: E1031 00:27:28.021191    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/97410648-b7ae-4cf4-accf-3164c398b525-original-pull-secret podName:97410648-b7ae-4cf4-accf-3164c398b525 nodeName:}" failed. No retries permitted until 2026-10-31 00:27:29.021176298 +0000 UTC m=+4.121719110 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/97410648-b7ae-4cf4-accf-3164c398b525-original-pull-secret") pod "global-pull-secret-syncer-24sxq" (UID: "97410648-b7ae-4cf4-accf-3164c398b525") : object "kube-system"/"original-pull-secret" not registered
Oct 31 00:27:28.122232 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:28.122180    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hxhdf\" (UniqueName: \"kubernetes.io/projected/c4ce4bad-af31-4f2e-a684-b2123e9da265-kube-api-access-hxhdf\") pod \"network-check-target-jktkx\" (UID: \"c4ce4bad-af31-4f2e-a684-b2123e9da265\") " pod="openshift-network-diagnostics/network-check-target-jktkx"
Oct 31 00:27:28.122232 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:28.122222    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/85e4b969-79f8-4954-8539-0905bd5db99d-metrics-certs\") pod \"network-metrics-daemon-plnkm\" (UID: \"85e4b969-79f8-4954-8539-0905bd5db99d\") " pod="openshift-multus/network-metrics-daemon-plnkm"
Oct 31 00:27:28.122443 ip-10-0-132-173 kubenswrapper[2572]: E1031 00:27:28.122341    2572 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 31 00:27:28.122443 ip-10-0-132-173 kubenswrapper[2572]: E1031 00:27:28.122365    2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 31 00:27:28.122443 ip-10-0-132-173 kubenswrapper[2572]: E1031 00:27:28.122392    2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 31 00:27:28.122443 ip-10-0-132-173 kubenswrapper[2572]: E1031 00:27:28.122394    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/85e4b969-79f8-4954-8539-0905bd5db99d-metrics-certs podName:85e4b969-79f8-4954-8539-0905bd5db99d nodeName:}" failed. No retries permitted until 2026-10-31 00:27:29.122380439 +0000 UTC m=+4.222923251 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/85e4b969-79f8-4954-8539-0905bd5db99d-metrics-certs") pod "network-metrics-daemon-plnkm" (UID: "85e4b969-79f8-4954-8539-0905bd5db99d") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 31 00:27:28.122443 ip-10-0-132-173 kubenswrapper[2572]: E1031 00:27:28.122404    2572 projected.go:196] Error preparing data for projected volume kube-api-access-hxhdf for pod openshift-network-diagnostics/network-check-target-jktkx: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 31 00:27:28.122626 ip-10-0-132-173 kubenswrapper[2572]: E1031 00:27:28.122456    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c4ce4bad-af31-4f2e-a684-b2123e9da265-kube-api-access-hxhdf podName:c4ce4bad-af31-4f2e-a684-b2123e9da265 nodeName:}" failed. No retries permitted until 2026-10-31 00:27:29.122439976 +0000 UTC m=+4.222982808 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-hxhdf" (UniqueName: "kubernetes.io/projected/c4ce4bad-af31-4f2e-a684-b2123e9da265-kube-api-access-hxhdf") pod "network-check-target-jktkx" (UID: "c4ce4bad-af31-4f2e-a684-b2123e9da265") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 31 00:27:28.476600 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:28.476549    2572 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k6fgp" event={"ID":"1a72f876-1157-4447-845c-923cdfd5aa5e","Type":"ContainerStarted","Data":"eb65fa9a666c91c9ac2ffe7057788f9538e79d38857116a648ae44c11f219605"}
Oct 31 00:27:28.477980 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:28.477950    2572 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-d7msb" event={"ID":"6926e9b8-7c44-413e-bc7e-63cfff593066","Type":"ContainerStarted","Data":"6044eeab0fcad7ea4b7fc0c1977044cd6744c491b566b1cd496034659c3f2490"}
Oct 31 00:27:28.479306 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:28.479279    2572 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2t5fx" event={"ID":"41c4cc4b-b387-4c70-8a27-58255ba632b9","Type":"ContainerStarted","Data":"1188e094eabb027d1abb7b9851edccf1d8c7f701d4ac9866b902992f255fdb47"}
Oct 31 00:27:28.480945 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:28.480920    2572 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-zjb6f" event={"ID":"2c9341ef-0860-4fb6-8a86-272ac063756a","Type":"ContainerStarted","Data":"9c4c014bdeff186fadbbba83b7498c289b19af588e40e2bf964ae400fb92f5eb"}
Oct 31 00:27:28.482625 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:28.482603    2572 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-hrsmn" event={"ID":"21df46e4-2158-4385-ab9a-4b1790b64b55","Type":"ContainerStarted","Data":"6dadfdf70d702465b2d703308d22e1030445b10a6bdff2e6ed1f87d6eb4d529c"}
Oct 31 00:27:28.483971 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:28.483941    2572 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-p4txz" event={"ID":"9daae1e9-e437-437c-ba8b-3d9c206b64a3","Type":"ContainerStarted","Data":"e0c303a73e7bc56cbce47921d95d76eeabf37b5c2954b84d3cf1cb7f2d06787d"}
Oct 31 00:27:28.485385 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:28.485365    2572 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rrk77" event={"ID":"63bedd38-6ab7-493d-b94c-b3c288af3f48","Type":"ContainerStarted","Data":"a17b6e11450f47a75f14e2684c8652ce4cc819ae80934ec6121b0c93e6af2407"}
Oct 31 00:27:28.486943 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:28.486915    2572 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-k2sk2" event={"ID":"f93a119c-fb17-44bd-973c-699a3b87523b","Type":"ContainerStarted","Data":"cf5491ca921b7277d410f4b081d5adf249e326fbdb3c5f39211ac00a72bfd7fb"}
Oct 31 00:27:28.488707 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:28.488682    2572 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-qchks" event={"ID":"61c60450-eb50-4181-8ab7-e69e864ffb51","Type":"ContainerStarted","Data":"8f0c1b598b3f2bc9b83fd8a0d75ad2e544d28447ea8afcd04398bee142bd99db"}
Oct 31 00:27:28.547453 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:28.547389    2572 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-10-30 00:22:28 +0000 UTC" deadline="2028-07-26 22:51:00.362914928 +0000 UTC"
Oct 31 00:27:28.547453 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:28.547449    2572 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15238h23m31.815470029s"
Oct 31 00:27:29.031100 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:29.031062    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/97410648-b7ae-4cf4-accf-3164c398b525-original-pull-secret\") pod \"global-pull-secret-syncer-24sxq\" (UID: \"97410648-b7ae-4cf4-accf-3164c398b525\") " pod="kube-system/global-pull-secret-syncer-24sxq"
Oct 31 00:27:29.031281 ip-10-0-132-173 kubenswrapper[2572]: E1031 00:27:29.031263    2572 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 31 00:27:29.031394 ip-10-0-132-173 kubenswrapper[2572]: E1031 00:27:29.031334    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/97410648-b7ae-4cf4-accf-3164c398b525-original-pull-secret podName:97410648-b7ae-4cf4-accf-3164c398b525 nodeName:}" failed. No retries permitted until 2026-10-31 00:27:31.031314915 +0000 UTC m=+6.131857731 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/97410648-b7ae-4cf4-accf-3164c398b525-original-pull-secret") pod "global-pull-secret-syncer-24sxq" (UID: "97410648-b7ae-4cf4-accf-3164c398b525") : object "kube-system"/"original-pull-secret" not registered
Oct 31 00:27:29.131712 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:29.131669    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hxhdf\" (UniqueName: \"kubernetes.io/projected/c4ce4bad-af31-4f2e-a684-b2123e9da265-kube-api-access-hxhdf\") pod \"network-check-target-jktkx\" (UID: \"c4ce4bad-af31-4f2e-a684-b2123e9da265\") " pod="openshift-network-diagnostics/network-check-target-jktkx"
Oct 31 00:27:29.131905 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:29.131743    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/85e4b969-79f8-4954-8539-0905bd5db99d-metrics-certs\") pod \"network-metrics-daemon-plnkm\" (UID: \"85e4b969-79f8-4954-8539-0905bd5db99d\") " pod="openshift-multus/network-metrics-daemon-plnkm"
Oct 31 00:27:29.131975 ip-10-0-132-173 kubenswrapper[2572]: E1031 00:27:29.131932    2572 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 31 00:27:29.132030 ip-10-0-132-173 kubenswrapper[2572]: E1031 00:27:29.132002    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/85e4b969-79f8-4954-8539-0905bd5db99d-metrics-certs podName:85e4b969-79f8-4954-8539-0905bd5db99d nodeName:}" failed. No retries permitted until 2026-10-31 00:27:31.131980772 +0000 UTC m=+6.232523590 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/85e4b969-79f8-4954-8539-0905bd5db99d-metrics-certs") pod "network-metrics-daemon-plnkm" (UID: "85e4b969-79f8-4954-8539-0905bd5db99d") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 31 00:27:29.132130 ip-10-0-132-173 kubenswrapper[2572]: E1031 00:27:29.132113    2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 31 00:27:29.132193 ip-10-0-132-173 kubenswrapper[2572]: E1031 00:27:29.132137    2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 31 00:27:29.132193 ip-10-0-132-173 kubenswrapper[2572]: E1031 00:27:29.132152    2572 projected.go:196] Error preparing data for projected volume kube-api-access-hxhdf for pod openshift-network-diagnostics/network-check-target-jktkx: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 31 00:27:29.132309 ip-10-0-132-173 kubenswrapper[2572]: E1031 00:27:29.132196    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c4ce4bad-af31-4f2e-a684-b2123e9da265-kube-api-access-hxhdf podName:c4ce4bad-af31-4f2e-a684-b2123e9da265 nodeName:}" failed. No retries permitted until 2026-10-31 00:27:31.132182516 +0000 UTC m=+6.232725332 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-hxhdf" (UniqueName: "kubernetes.io/projected/c4ce4bad-af31-4f2e-a684-b2123e9da265-kube-api-access-hxhdf") pod "network-check-target-jktkx" (UID: "c4ce4bad-af31-4f2e-a684-b2123e9da265") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 31 00:27:29.467380 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:29.467346    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-plnkm"
Oct 31 00:27:29.467380 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:29.467365    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jktkx"
Oct 31 00:27:29.467624 ip-10-0-132-173 kubenswrapper[2572]: E1031 00:27:29.467472    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-plnkm" podUID="85e4b969-79f8-4954-8539-0905bd5db99d"
Oct 31 00:27:29.467624 ip-10-0-132-173 kubenswrapper[2572]: E1031 00:27:29.467564    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jktkx" podUID="c4ce4bad-af31-4f2e-a684-b2123e9da265"
Oct 31 00:27:29.467734 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:29.467643    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-24sxq"
Oct 31 00:27:29.467793 ip-10-0-132-173 kubenswrapper[2572]: E1031 00:27:29.467739    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-24sxq" podUID="97410648-b7ae-4cf4-accf-3164c398b525"
Oct 31 00:27:29.548018 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:29.547974    2572 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-10-30 00:22:28 +0000 UTC" deadline="2028-04-18 22:08:34.782726796 +0000 UTC"
Oct 31 00:27:29.548018 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:29.548018    2572 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12861h41m5.23471599s"
Oct 31 00:27:31.054224 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:31.053767    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/97410648-b7ae-4cf4-accf-3164c398b525-original-pull-secret\") pod \"global-pull-secret-syncer-24sxq\" (UID: \"97410648-b7ae-4cf4-accf-3164c398b525\") " pod="kube-system/global-pull-secret-syncer-24sxq"
Oct 31 00:27:31.054224 ip-10-0-132-173 kubenswrapper[2572]: E1031 00:27:31.053890    2572 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 31 00:27:31.054224 ip-10-0-132-173 kubenswrapper[2572]: E1031 00:27:31.053942    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/97410648-b7ae-4cf4-accf-3164c398b525-original-pull-secret podName:97410648-b7ae-4cf4-accf-3164c398b525 nodeName:}" failed. No retries permitted until 2026-10-31 00:27:35.05392811 +0000 UTC m=+10.154470925 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/97410648-b7ae-4cf4-accf-3164c398b525-original-pull-secret") pod "global-pull-secret-syncer-24sxq" (UID: "97410648-b7ae-4cf4-accf-3164c398b525") : object "kube-system"/"original-pull-secret" not registered
Oct 31 00:27:31.157160 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:31.154513    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hxhdf\" (UniqueName: \"kubernetes.io/projected/c4ce4bad-af31-4f2e-a684-b2123e9da265-kube-api-access-hxhdf\") pod \"network-check-target-jktkx\" (UID: \"c4ce4bad-af31-4f2e-a684-b2123e9da265\") " pod="openshift-network-diagnostics/network-check-target-jktkx"
Oct 31 00:27:31.157160 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:31.154564    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/85e4b969-79f8-4954-8539-0905bd5db99d-metrics-certs\") pod \"network-metrics-daemon-plnkm\" (UID: \"85e4b969-79f8-4954-8539-0905bd5db99d\") " pod="openshift-multus/network-metrics-daemon-plnkm"
Oct 31 00:27:31.157160 ip-10-0-132-173 kubenswrapper[2572]: E1031 00:27:31.154696    2572 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 31 00:27:31.157160 ip-10-0-132-173 kubenswrapper[2572]: E1031 00:27:31.154754    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/85e4b969-79f8-4954-8539-0905bd5db99d-metrics-certs podName:85e4b969-79f8-4954-8539-0905bd5db99d nodeName:}" failed. No retries permitted until 2026-10-31 00:27:35.154739137 +0000 UTC m=+10.255281948 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/85e4b969-79f8-4954-8539-0905bd5db99d-metrics-certs") pod "network-metrics-daemon-plnkm" (UID: "85e4b969-79f8-4954-8539-0905bd5db99d") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 31 00:27:31.157160 ip-10-0-132-173 kubenswrapper[2572]: E1031 00:27:31.155149    2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 31 00:27:31.157160 ip-10-0-132-173 kubenswrapper[2572]: E1031 00:27:31.155165    2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 31 00:27:31.157160 ip-10-0-132-173 kubenswrapper[2572]: E1031 00:27:31.155175    2572 projected.go:196] Error preparing data for projected volume kube-api-access-hxhdf for pod openshift-network-diagnostics/network-check-target-jktkx: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 31 00:27:31.157160 ip-10-0-132-173 kubenswrapper[2572]: E1031 00:27:31.155225    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c4ce4bad-af31-4f2e-a684-b2123e9da265-kube-api-access-hxhdf podName:c4ce4bad-af31-4f2e-a684-b2123e9da265 nodeName:}" failed. No retries permitted until 2026-10-31 00:27:35.155209245 +0000 UTC m=+10.255752084 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-hxhdf" (UniqueName: "kubernetes.io/projected/c4ce4bad-af31-4f2e-a684-b2123e9da265-kube-api-access-hxhdf") pod "network-check-target-jktkx" (UID: "c4ce4bad-af31-4f2e-a684-b2123e9da265") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 31 00:27:31.471848 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:31.470960    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-plnkm"
Oct 31 00:27:31.471848 ip-10-0-132-173 kubenswrapper[2572]: E1031 00:27:31.471096    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-plnkm" podUID="85e4b969-79f8-4954-8539-0905bd5db99d"
Oct 31 00:27:31.471848 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:31.471537    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jktkx"
Oct 31 00:27:31.471848 ip-10-0-132-173 kubenswrapper[2572]: E1031 00:27:31.471647    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jktkx" podUID="c4ce4bad-af31-4f2e-a684-b2123e9da265"
Oct 31 00:27:31.471848 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:31.471706    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-24sxq"
Oct 31 00:27:31.471848 ip-10-0-132-173 kubenswrapper[2572]: E1031 00:27:31.471771    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-24sxq" podUID="97410648-b7ae-4cf4-accf-3164c398b525"
Oct 31 00:27:32.545643 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:32.545416    2572 generic.go:356] "Generic (PLEG): container finished" podID="6b005c94812deb7bd218c8f0cf290c09" containerID="5b2371580e711da02ae94dfbe659fb42391bd9cb84cc78e18838a4bc8c55ae17" exitCode=0
Oct 31 00:27:32.546178 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:32.545694    2572 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-173.us-west-2.compute.internal" event={"ID":"6b005c94812deb7bd218c8f0cf290c09","Type":"ContainerDied","Data":"5b2371580e711da02ae94dfbe659fb42391bd9cb84cc78e18838a4bc8c55ae17"}
Oct 31 00:27:32.551168 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:32.551137    2572 generic.go:356] "Generic (PLEG): container finished" podID="41c4cc4b-b387-4c70-8a27-58255ba632b9" containerID="508a002da3205d4f14eae542dc1b75634c31b15fce4118dd41d9dc7c8d8d2e06" exitCode=0
Oct 31 00:27:32.551329 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:32.551227    2572 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2t5fx" event={"ID":"41c4cc4b-b387-4c70-8a27-58255ba632b9","Type":"ContainerDied","Data":"508a002da3205d4f14eae542dc1b75634c31b15fce4118dd41d9dc7c8d8d2e06"}
Oct 31 00:27:32.562992 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:32.562960    2572 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-173.us-west-2.compute.internal" event={"ID":"6c9ceace9ec30530fd2581e47cd10afe","Type":"ContainerStarted","Data":"3d715c11efa2b4718cdc3a5a05a77be833001e9aeba541a56f95fecb8aada745"}
Oct 31 00:27:32.789163 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:32.788886    2572 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-173.us-west-2.compute.internal" podStartSLOduration=4.788870606 podStartE2EDuration="4.788870606s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-31 00:27:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:27:32.787931374 +0000 UTC m=+7.888474210" watchObservedRunningTime="2026-10-31 00:27:32.788870606 +0000 UTC m=+7.889413430"
Oct 31 00:27:33.469051 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:33.468068    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-plnkm"
Oct 31 00:27:33.469051 ip-10-0-132-173 kubenswrapper[2572]: E1031 00:27:33.468262    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-plnkm" podUID="85e4b969-79f8-4954-8539-0905bd5db99d"
Oct 31 00:27:33.469051 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:33.468687    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jktkx"
Oct 31 00:27:33.469051 ip-10-0-132-173 kubenswrapper[2572]: E1031 00:27:33.468777    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jktkx" podUID="c4ce4bad-af31-4f2e-a684-b2123e9da265"
Oct 31 00:27:33.469051 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:33.468829    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-24sxq"
Oct 31 00:27:33.469051 ip-10-0-132-173 kubenswrapper[2572]: E1031 00:27:33.468900    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-24sxq" podUID="97410648-b7ae-4cf4-accf-3164c398b525"
Oct 31 00:27:33.577634 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:33.576728    2572 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-173.us-west-2.compute.internal" event={"ID":"6b005c94812deb7bd218c8f0cf290c09","Type":"ContainerStarted","Data":"43fa13eef47facac1d98a81cb1264617f9a2a72a6daaaec3cd2f9ea064206fc7"}
Oct 31 00:27:33.580494 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:33.580465    2572 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-k2sk2" event={"ID":"f93a119c-fb17-44bd-973c-699a3b87523b","Type":"ContainerStarted","Data":"1a78272cad9b0f187a88d1e18f1e9987074596e1374f79259c5c52159b81a1e6"}
Oct 31 00:27:33.817189 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:33.817068    2572 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-173.us-west-2.compute.internal" podStartSLOduration=5.817046079 podStartE2EDuration="5.817046079s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-31 00:27:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:27:33.697408538 +0000 UTC m=+8.797951398" watchObservedRunningTime="2026-10-31 00:27:33.817046079 +0000 UTC m=+8.917588919"
Oct 31 00:27:34.587411 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:34.587376    2572 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-qchks" event={"ID":"61c60450-eb50-4181-8ab7-e69e864ffb51","Type":"ContainerStarted","Data":"78d51712bcb03272c14d58b2369a2b521ece0187b6c2e14b623b3e25bcadc0be"}
Oct 31 00:27:34.595108 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:34.594506    2572 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k6fgp" event={"ID":"1a72f876-1157-4447-845c-923cdfd5aa5e","Type":"ContainerStarted","Data":"5b2e7c2d0c28ac167cbc043c750e0fd0a61e339845df1be0d96346c75d96deb4"}
Oct 31 00:27:34.706818 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:34.706758    2572 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/konnectivity-agent-k2sk2" podStartSLOduration=3.8377704059999997 podStartE2EDuration="8.706740878s" totalImagesPullingTime="4.868970472s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 00:27:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:27:33.817843273 +0000 UTC m=+8.918386109" watchObservedRunningTime="2026-10-31 00:27:34.706740878 +0000 UTC m=+9.807283714"
Oct 31 00:27:35.088367 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:35.088330    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/97410648-b7ae-4cf4-accf-3164c398b525-original-pull-secret\") pod \"global-pull-secret-syncer-24sxq\" (UID: \"97410648-b7ae-4cf4-accf-3164c398b525\") " pod="kube-system/global-pull-secret-syncer-24sxq"
Oct 31 00:27:35.088566 ip-10-0-132-173 kubenswrapper[2572]: E1031 00:27:35.088546    2572 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 31 00:27:35.088622 ip-10-0-132-173 kubenswrapper[2572]: E1031 00:27:35.088616    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/97410648-b7ae-4cf4-accf-3164c398b525-original-pull-secret podName:97410648-b7ae-4cf4-accf-3164c398b525 nodeName:}" failed. No retries permitted until 2026-10-31 00:27:43.088597325 +0000 UTC m=+18.189140153 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/97410648-b7ae-4cf4-accf-3164c398b525-original-pull-secret") pod "global-pull-secret-syncer-24sxq" (UID: "97410648-b7ae-4cf4-accf-3164c398b525") : object "kube-system"/"original-pull-secret" not registered
Oct 31 00:27:35.189139 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:35.189100    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hxhdf\" (UniqueName: \"kubernetes.io/projected/c4ce4bad-af31-4f2e-a684-b2123e9da265-kube-api-access-hxhdf\") pod \"network-check-target-jktkx\" (UID: \"c4ce4bad-af31-4f2e-a684-b2123e9da265\") " pod="openshift-network-diagnostics/network-check-target-jktkx"
Oct 31 00:27:35.189349 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:35.189155    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/85e4b969-79f8-4954-8539-0905bd5db99d-metrics-certs\") pod \"network-metrics-daemon-plnkm\" (UID: \"85e4b969-79f8-4954-8539-0905bd5db99d\") " pod="openshift-multus/network-metrics-daemon-plnkm"
Oct 31 00:27:35.189410 ip-10-0-132-173 kubenswrapper[2572]: E1031 00:27:35.189369    2572 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 31 00:27:35.189470 ip-10-0-132-173 kubenswrapper[2572]: E1031 00:27:35.189430    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/85e4b969-79f8-4954-8539-0905bd5db99d-metrics-certs podName:85e4b969-79f8-4954-8539-0905bd5db99d nodeName:}" failed. No retries permitted until 2026-10-31 00:27:43.189412347 +0000 UTC m=+18.289955185 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/85e4b969-79f8-4954-8539-0905bd5db99d-metrics-certs") pod "network-metrics-daemon-plnkm" (UID: "85e4b969-79f8-4954-8539-0905bd5db99d") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 31 00:27:35.189940 ip-10-0-132-173 kubenswrapper[2572]: E1031 00:27:35.189917    2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 31 00:27:35.190041 ip-10-0-132-173 kubenswrapper[2572]: E1031 00:27:35.189943    2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 31 00:27:35.190041 ip-10-0-132-173 kubenswrapper[2572]: E1031 00:27:35.189957    2572 projected.go:196] Error preparing data for projected volume kube-api-access-hxhdf for pod openshift-network-diagnostics/network-check-target-jktkx: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 31 00:27:35.190041 ip-10-0-132-173 kubenswrapper[2572]: E1031 00:27:35.190010    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c4ce4bad-af31-4f2e-a684-b2123e9da265-kube-api-access-hxhdf podName:c4ce4bad-af31-4f2e-a684-b2123e9da265 nodeName:}" failed. No retries permitted until 2026-10-31 00:27:43.189994248 +0000 UTC m=+18.290537087 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-hxhdf" (UniqueName: "kubernetes.io/projected/c4ce4bad-af31-4f2e-a684-b2123e9da265-kube-api-access-hxhdf") pod "network-check-target-jktkx" (UID: "c4ce4bad-af31-4f2e-a684-b2123e9da265") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 31 00:27:35.468471 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:35.467440    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-plnkm"
Oct 31 00:27:35.468471 ip-10-0-132-173 kubenswrapper[2572]: E1031 00:27:35.467592    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-plnkm" podUID="85e4b969-79f8-4954-8539-0905bd5db99d"
Oct 31 00:27:35.468471 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:35.468009    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jktkx"
Oct 31 00:27:35.468471 ip-10-0-132-173 kubenswrapper[2572]: E1031 00:27:35.468104    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jktkx" podUID="c4ce4bad-af31-4f2e-a684-b2123e9da265"
Oct 31 00:27:35.468471 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:35.468181    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-24sxq"
Oct 31 00:27:35.468471 ip-10-0-132-173 kubenswrapper[2572]: E1031 00:27:35.468271    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-24sxq" podUID="97410648-b7ae-4cf4-accf-3164c398b525"
Oct 31 00:27:36.002969 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:36.002417    2572 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-k2sk2"
Oct 31 00:27:36.003886 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:36.003694    2572 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-k2sk2"
Oct 31 00:27:36.122779 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:36.122729    2572 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-image-registry/node-ca-qchks" podStartSLOduration=3.873308787 podStartE2EDuration="10.122708785s" totalImagesPullingTime="6.249399998s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-31 00:27:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:27:34.70754778 +0000 UTC m=+9.808090616" watchObservedRunningTime="2026-10-31 00:27:36.122708785 +0000 UTC m=+11.223251611"
Oct 31 00:27:36.977354 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:36.977330    2572 plugin_watcher.go:201] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Oct 31 00:27:37.380321 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:37.380179    2572 reconciler.go:167] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-10-31T00:27:36.977351001Z","UUID":"95e755ac-acea-40a1-adb0-604981ff2ca1","Handler":null,"Name":"","Endpoint":""}
Oct 31 00:27:37.382122 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:37.381997    2572 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Oct 31 00:27:37.382122 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:37.382026    2572 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Oct 31 00:27:37.466423 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:37.466391    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-24sxq"
Oct 31 00:27:37.466523 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:37.466421    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-plnkm"
Oct 31 00:27:37.466523 ip-10-0-132-173 kubenswrapper[2572]: E1031 00:27:37.466511    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-24sxq" podUID="97410648-b7ae-4cf4-accf-3164c398b525"
Oct 31 00:27:37.466645 ip-10-0-132-173 kubenswrapper[2572]: E1031 00:27:37.466593    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-plnkm" podUID="85e4b969-79f8-4954-8539-0905bd5db99d"
Oct 31 00:27:37.466645 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:37.466627    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jktkx"
Oct 31 00:27:37.466750 ip-10-0-132-173 kubenswrapper[2572]: E1031 00:27:37.466709    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jktkx" podUID="c4ce4bad-af31-4f2e-a684-b2123e9da265"
Oct 31 00:27:37.603841 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:37.603803    2572 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k6fgp" event={"ID":"1a72f876-1157-4447-845c-923cdfd5aa5e","Type":"ContainerStarted","Data":"9f9102a52505e96784cd302b21806e456f25c128d322a248033a1ce83ae72c6b"}
Oct 31 00:27:37.605913 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:37.605877    2572 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-d7msb" event={"ID":"6926e9b8-7c44-413e-bc7e-63cfff593066","Type":"ContainerStarted","Data":"17ae858da7ba71387dad63e68214a4581914cb19810cbfc6dd33e3fdfccdee1a"}
Oct 31 00:27:37.608851 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:37.608783    2572 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2t5fx" event={"ID":"41c4cc4b-b387-4c70-8a27-58255ba632b9","Type":"ContainerStarted","Data":"3e3f91b116ad245d40596238a421239406c46c83c681aee7049eafd3ed15156c"}
Oct 31 00:27:37.610573 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:37.610546    2572 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-zjb6f" event={"ID":"2c9341ef-0860-4fb6-8a86-272ac063756a","Type":"ContainerStarted","Data":"4fa1163fa5fbdfade36d21cb24367b5006dae4ce826adde24416ab22b08bbacc"}
Oct 31 00:27:37.732752 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:37.732650    2572 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-multus/multus-d7msb" podStartSLOduration=2.643376393 podStartE2EDuration="11.732632978s" totalImagesPullingTime="9.089256585s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-31 00:27:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:27:37.732089769 +0000 UTC m=+12.832632606" watchObservedRunningTime="2026-10-31 00:27:37.732632978 +0000 UTC m=+12.833175814"
Oct 31 00:27:37.854207 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:37.854137    2572 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-dns/node-resolver-zjb6f" podStartSLOduration=3.019881366 podStartE2EDuration="11.854119255s" totalImagesPullingTime="8.834237889s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-31 00:27:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:27:37.853514373 +0000 UTC m=+12.954057208" watchObservedRunningTime="2026-10-31 00:27:37.854119255 +0000 UTC m=+12.954662093"
Oct 31 00:27:38.615205 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:38.614990    2572 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-p4txz" event={"ID":"9daae1e9-e437-437c-ba8b-3d9c206b64a3","Type":"ContainerStarted","Data":"9309ed537c8875c9eab7219ca513ceec362c7f406bb5e027041c1bc67ad6aec4"}
Oct 31 00:27:38.617584 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:38.617552    2572 generic.go:356] "Generic (PLEG): container finished" podID="41c4cc4b-b387-4c70-8a27-58255ba632b9" containerID="3e3f91b116ad245d40596238a421239406c46c83c681aee7049eafd3ed15156c" exitCode=0
Oct 31 00:27:38.617855 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:38.617827    2572 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2t5fx" event={"ID":"41c4cc4b-b387-4c70-8a27-58255ba632b9","Type":"ContainerDied","Data":"3e3f91b116ad245d40596238a421239406c46c83c681aee7049eafd3ed15156c"}
Oct 31 00:27:38.728792 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:38.728747    2572 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-p4txz" podStartSLOduration=3.456237761 podStartE2EDuration="12.728728357s" totalImagesPullingTime="9.272490596s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-31 00:27:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:27:38.728288967 +0000 UTC m=+13.828831801" watchObservedRunningTime="2026-10-31 00:27:38.728728357 +0000 UTC m=+13.829271227"
Oct 31 00:27:39.465970 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:39.465748    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jktkx"
Oct 31 00:27:39.466137 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:39.465783    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-24sxq"
Oct 31 00:27:39.466137 ip-10-0-132-173 kubenswrapper[2572]: E1031 00:27:39.466057    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jktkx" podUID="c4ce4bad-af31-4f2e-a684-b2123e9da265"
Oct 31 00:27:39.466288 ip-10-0-132-173 kubenswrapper[2572]: E1031 00:27:39.466156    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-24sxq" podUID="97410648-b7ae-4cf4-accf-3164c398b525"
Oct 31 00:27:39.466288 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:39.465838    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-plnkm"
Oct 31 00:27:39.466288 ip-10-0-132-173 kubenswrapper[2572]: E1031 00:27:39.466274    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-plnkm" podUID="85e4b969-79f8-4954-8539-0905bd5db99d"
Oct 31 00:27:39.621837 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:39.621801    2572 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k6fgp" event={"ID":"1a72f876-1157-4447-845c-923cdfd5aa5e","Type":"ContainerStarted","Data":"baeb8874ccd1a730d94575fd8b7319fc28e6683ba1fa3e57064a2707cff306b4"}
Oct 31 00:27:39.623483 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:39.623442    2572 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-hrsmn" event={"ID":"21df46e4-2158-4385-ab9a-4b1790b64b55","Type":"ContainerStarted","Data":"98823a5ef6a8f540d551a2646edf1a94844dabf41689d57381f677aef501aa73"}
Oct 31 00:27:39.735512 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:39.735403    2572 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k6fgp" podStartSLOduration=3.287841126 podStartE2EDuration="13.735388703s" totalImagesPullingTime="10.447547577s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-31 00:27:26 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:27:39.734819815 +0000 UTC m=+14.835362660" watchObservedRunningTime="2026-10-31 00:27:39.735388703 +0000 UTC m=+14.835931582"
Oct 31 00:27:39.851100 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:39.851041    2572 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-hrsmn" podStartSLOduration=2.627110138 podStartE2EDuration="13.851021446s" totalImagesPullingTime="11.223911308s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-31 00:27:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:27:39.850137458 +0000 UTC m=+14.950680291" watchObservedRunningTime="2026-10-31 00:27:39.851021446 +0000 UTC m=+14.951564279"
Oct 31 00:27:40.627788 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:40.627751    2572 generic.go:356] "Generic (PLEG): container finished" podID="41c4cc4b-b387-4c70-8a27-58255ba632b9" containerID="c67fca05dd53a48ed0934bfea1e1439af43446f05682b0c6b1c27cf7d265da07" exitCode=0
Oct 31 00:27:40.628448 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:40.627843    2572 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2t5fx" event={"ID":"41c4cc4b-b387-4c70-8a27-58255ba632b9","Type":"ContainerDied","Data":"c67fca05dd53a48ed0934bfea1e1439af43446f05682b0c6b1c27cf7d265da07"}
Oct 31 00:27:41.464846 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:41.464814    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jktkx"
Oct 31 00:27:41.464846 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:41.464845    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-plnkm"
Oct 31 00:27:41.465068 ip-10-0-132-173 kubenswrapper[2572]: E1031 00:27:41.464942    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jktkx" podUID="c4ce4bad-af31-4f2e-a684-b2123e9da265"
Oct 31 00:27:41.465068 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:41.465035    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-24sxq"
Oct 31 00:27:41.465068 ip-10-0-132-173 kubenswrapper[2572]: E1031 00:27:41.465034    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-plnkm" podUID="85e4b969-79f8-4954-8539-0905bd5db99d"
Oct 31 00:27:41.465245 ip-10-0-132-173 kubenswrapper[2572]: E1031 00:27:41.465111    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-24sxq" podUID="97410648-b7ae-4cf4-accf-3164c398b525"
Oct 31 00:27:42.635067 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:42.635034    2572 generic.go:356] "Generic (PLEG): container finished" podID="41c4cc4b-b387-4c70-8a27-58255ba632b9" containerID="6ac8ef00f2a5ff74e39d436adae7bf8734fd6dd81a2fb68129183b470058f957" exitCode=0
Oct 31 00:27:42.635851 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:42.635076    2572 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2t5fx" event={"ID":"41c4cc4b-b387-4c70-8a27-58255ba632b9","Type":"ContainerDied","Data":"6ac8ef00f2a5ff74e39d436adae7bf8734fd6dd81a2fb68129183b470058f957"}
Oct 31 00:27:43.152921 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:43.152881    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/97410648-b7ae-4cf4-accf-3164c398b525-original-pull-secret\") pod \"global-pull-secret-syncer-24sxq\" (UID: \"97410648-b7ae-4cf4-accf-3164c398b525\") " pod="kube-system/global-pull-secret-syncer-24sxq"
Oct 31 00:27:43.153115 ip-10-0-132-173 kubenswrapper[2572]: E1031 00:27:43.153008    2572 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 31 00:27:43.153115 ip-10-0-132-173 kubenswrapper[2572]: E1031 00:27:43.153068    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/97410648-b7ae-4cf4-accf-3164c398b525-original-pull-secret podName:97410648-b7ae-4cf4-accf-3164c398b525 nodeName:}" failed. No retries permitted until 2026-10-31 00:27:59.153049848 +0000 UTC m=+34.253592659 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/97410648-b7ae-4cf4-accf-3164c398b525-original-pull-secret") pod "global-pull-secret-syncer-24sxq" (UID: "97410648-b7ae-4cf4-accf-3164c398b525") : object "kube-system"/"original-pull-secret" not registered
Oct 31 00:27:43.253601 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:43.253557    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hxhdf\" (UniqueName: \"kubernetes.io/projected/c4ce4bad-af31-4f2e-a684-b2123e9da265-kube-api-access-hxhdf\") pod \"network-check-target-jktkx\" (UID: \"c4ce4bad-af31-4f2e-a684-b2123e9da265\") " pod="openshift-network-diagnostics/network-check-target-jktkx"
Oct 31 00:27:43.253601 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:43.253603    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/85e4b969-79f8-4954-8539-0905bd5db99d-metrics-certs\") pod \"network-metrics-daemon-plnkm\" (UID: \"85e4b969-79f8-4954-8539-0905bd5db99d\") " pod="openshift-multus/network-metrics-daemon-plnkm"
Oct 31 00:27:43.253832 ip-10-0-132-173 kubenswrapper[2572]: E1031 00:27:43.253721    2572 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 31 00:27:43.253832 ip-10-0-132-173 kubenswrapper[2572]: E1031 00:27:43.253744    2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 31 00:27:43.253832 ip-10-0-132-173 kubenswrapper[2572]: E1031 00:27:43.253769    2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 31 00:27:43.253832 ip-10-0-132-173 kubenswrapper[2572]: E1031 00:27:43.253777    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/85e4b969-79f8-4954-8539-0905bd5db99d-metrics-certs podName:85e4b969-79f8-4954-8539-0905bd5db99d nodeName:}" failed. No retries permitted until 2026-10-31 00:27:59.253763454 +0000 UTC m=+34.354306267 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/85e4b969-79f8-4954-8539-0905bd5db99d-metrics-certs") pod "network-metrics-daemon-plnkm" (UID: "85e4b969-79f8-4954-8539-0905bd5db99d") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 31 00:27:43.253832 ip-10-0-132-173 kubenswrapper[2572]: E1031 00:27:43.253782    2572 projected.go:196] Error preparing data for projected volume kube-api-access-hxhdf for pod openshift-network-diagnostics/network-check-target-jktkx: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 31 00:27:43.253832 ip-10-0-132-173 kubenswrapper[2572]: E1031 00:27:43.253832    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c4ce4bad-af31-4f2e-a684-b2123e9da265-kube-api-access-hxhdf podName:c4ce4bad-af31-4f2e-a684-b2123e9da265 nodeName:}" failed. No retries permitted until 2026-10-31 00:27:59.253816243 +0000 UTC m=+34.354359056 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-hxhdf" (UniqueName: "kubernetes.io/projected/c4ce4bad-af31-4f2e-a684-b2123e9da265-kube-api-access-hxhdf") pod "network-check-target-jktkx" (UID: "c4ce4bad-af31-4f2e-a684-b2123e9da265") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 31 00:27:43.464802 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:43.464772    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-plnkm"
Oct 31 00:27:43.464963 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:43.464783    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jktkx"
Oct 31 00:27:43.464963 ip-10-0-132-173 kubenswrapper[2572]: E1031 00:27:43.464897    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-plnkm" podUID="85e4b969-79f8-4954-8539-0905bd5db99d"
Oct 31 00:27:43.464963 ip-10-0-132-173 kubenswrapper[2572]: E1031 00:27:43.464953    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jktkx" podUID="c4ce4bad-af31-4f2e-a684-b2123e9da265"
Oct 31 00:27:43.465127 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:43.465005    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-24sxq"
Oct 31 00:27:43.465127 ip-10-0-132-173 kubenswrapper[2572]: E1031 00:27:43.465073    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-24sxq" podUID="97410648-b7ae-4cf4-accf-3164c398b525"
Oct 31 00:27:44.643334 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:44.643255    2572 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rrk77" event={"ID":"63bedd38-6ab7-493d-b94c-b3c288af3f48","Type":"ContainerStarted","Data":"d3ee6a7f6f1cc8054086f6282710f9b8379dfb80fd19a50925335572076d06a8"}
Oct 31 00:27:44.643807 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:44.643365    2572 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rrk77" event={"ID":"63bedd38-6ab7-493d-b94c-b3c288af3f48","Type":"ContainerStarted","Data":"5fe09b9cfbd5538c94e755f98c76923cef11cd52608b662abae389cc14366086"}
Oct 31 00:27:44.643807 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:44.643439    2572 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rrk77" event={"ID":"63bedd38-6ab7-493d-b94c-b3c288af3f48","Type":"ContainerStarted","Data":"d7db3b1becb5a78c6e72990a2b3003416edfc86fa720b25e45d928e79c311c1d"}
Oct 31 00:27:44.643807 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:44.643478    2572 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rrk77" event={"ID":"63bedd38-6ab7-493d-b94c-b3c288af3f48","Type":"ContainerStarted","Data":"677cbbf017f0ba2425083a3744dcd2af2a14745bd736e3d709f14fc4314ac999"}
Oct 31 00:27:44.643807 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:44.643525    2572 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rrk77" event={"ID":"63bedd38-6ab7-493d-b94c-b3c288af3f48","Type":"ContainerStarted","Data":"3c3ee44b3efabf793af741e67dd11a3fd298896c2cd69458c402c85144c19e1b"}
Oct 31 00:27:44.643807 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:44.643567    2572 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rrk77" event={"ID":"63bedd38-6ab7-493d-b94c-b3c288af3f48","Type":"ContainerStarted","Data":"38b73d8819bcc800aa461502d7df83118a4de4a8a88f4c254ca52377443c95a6"}
Oct 31 00:27:45.466555 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:45.466392    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-plnkm"
Oct 31 00:27:45.466692 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:45.466462    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jktkx"
Oct 31 00:27:45.466743 ip-10-0-132-173 kubenswrapper[2572]: E1031 00:27:45.466713    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jktkx" podUID="c4ce4bad-af31-4f2e-a684-b2123e9da265"
Oct 31 00:27:45.466743 ip-10-0-132-173 kubenswrapper[2572]: E1031 00:27:45.466633    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-plnkm" podUID="85e4b969-79f8-4954-8539-0905bd5db99d"
Oct 31 00:27:45.466743 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:45.466510    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-24sxq"
Oct 31 00:27:45.466854 ip-10-0-132-173 kubenswrapper[2572]: E1031 00:27:45.466795    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-24sxq" podUID="97410648-b7ae-4cf4-accf-3164c398b525"
Oct 31 00:27:45.648201 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:45.648168    2572 generic.go:356] "Generic (PLEG): container finished" podID="41c4cc4b-b387-4c70-8a27-58255ba632b9" containerID="8f2df9b116a2a10b7f5c0df12cc0a0f287da91deb1c919cae2edbe479ba9c642" exitCode=0
Oct 31 00:27:45.648572 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:45.648252    2572 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2t5fx" event={"ID":"41c4cc4b-b387-4c70-8a27-58255ba632b9","Type":"ContainerDied","Data":"8f2df9b116a2a10b7f5c0df12cc0a0f287da91deb1c919cae2edbe479ba9c642"}
Oct 31 00:27:46.653488 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:46.653453    2572 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rrk77" event={"ID":"63bedd38-6ab7-493d-b94c-b3c288af3f48","Type":"ContainerStarted","Data":"537db2ce2d15f17ae6c423d557e98293290b95762141745bc8759e7c4985d7c1"}
Oct 31 00:27:46.656157 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:46.656123    2572 generic.go:356] "Generic (PLEG): container finished" podID="41c4cc4b-b387-4c70-8a27-58255ba632b9" containerID="f9f499ecb0a7c7954769abbadf0e73f4107a33dbf973bd99b9718074d4fd14dc" exitCode=0
Oct 31 00:27:46.656269 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:46.656162    2572 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2t5fx" event={"ID":"41c4cc4b-b387-4c70-8a27-58255ba632b9","Type":"ContainerDied","Data":"f9f499ecb0a7c7954769abbadf0e73f4107a33dbf973bd99b9718074d4fd14dc"}
Oct 31 00:27:47.210954 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:47.210927    2572 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-k2sk2"
Oct 31 00:27:47.211096 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:47.211059    2572 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Oct 31 00:27:47.211465 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:47.211438    2572 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-k2sk2"
Oct 31 00:27:47.465913 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:47.465838    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jktkx"
Oct 31 00:27:47.465913 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:47.465897    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-24sxq"
Oct 31 00:27:47.465913 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:47.465897    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-plnkm"
Oct 31 00:27:47.466196 ip-10-0-132-173 kubenswrapper[2572]: E1031 00:27:47.466000    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jktkx" podUID="c4ce4bad-af31-4f2e-a684-b2123e9da265"
Oct 31 00:27:47.466196 ip-10-0-132-173 kubenswrapper[2572]: E1031 00:27:47.466107    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-24sxq" podUID="97410648-b7ae-4cf4-accf-3164c398b525"
Oct 31 00:27:47.466196 ip-10-0-132-173 kubenswrapper[2572]: E1031 00:27:47.466186    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-plnkm" podUID="85e4b969-79f8-4954-8539-0905bd5db99d"
Oct 31 00:27:47.661386 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:47.661354    2572 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2t5fx" event={"ID":"41c4cc4b-b387-4c70-8a27-58255ba632b9","Type":"ContainerStarted","Data":"d1f865632d324184483093bea2ae23defdfd9facd192295e0de7d99c0e7ef5e7"}
Oct 31 00:27:47.779031 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:47.778940    2572 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-2t5fx" podStartSLOduration=8.688920482 podStartE2EDuration="21.778927293s" totalImagesPullingTime="13.090006811s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-31 00:27:26 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:27:47.778521064 +0000 UTC m=+22.879063895" watchObservedRunningTime="2026-10-31 00:27:47.778927293 +0000 UTC m=+22.879470123"
Oct 31 00:27:47.883156 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:47.883114    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:27:48.081354 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:48.081263    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:27:48.284386 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:48.284318    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:27:48.432643 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:48.432594    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:27:48.668829 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:48.668790    2572 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rrk77" event={"ID":"63bedd38-6ab7-493d-b94c-b3c288af3f48","Type":"ContainerStarted","Data":"c615310ee4a7d405db400d51c346ba6aeea9ec9cf01e405b03ab88c2845298d7"}
Oct 31 00:27:49.464803 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:49.464777    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-plnkm"
Oct 31 00:27:49.464978 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:49.464813    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-24sxq"
Oct 31 00:27:49.464978 ip-10-0-132-173 kubenswrapper[2572]: E1031 00:27:49.464895    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-plnkm" podUID="85e4b969-79f8-4954-8539-0905bd5db99d"
Oct 31 00:27:49.464978 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:49.464777    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jktkx"
Oct 31 00:27:49.465078 ip-10-0-132-173 kubenswrapper[2572]: E1031 00:27:49.464990    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-24sxq" podUID="97410648-b7ae-4cf4-accf-3164c398b525"
Oct 31 00:27:49.465078 ip-10-0-132-173 kubenswrapper[2572]: E1031 00:27:49.465070    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jktkx" podUID="c4ce4bad-af31-4f2e-a684-b2123e9da265"
Oct 31 00:27:49.671374 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:49.671340    2572 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-rrk77"
Oct 31 00:27:49.687346 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:49.687324    2572 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-rrk77"
Oct 31 00:27:49.935123 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:49.935072    2572 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-rrk77" podStartSLOduration=7.596658096 podStartE2EDuration="23.935058947s" totalImagesPullingTime="16.338400851s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-31 00:27:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:27:49.887608112 +0000 UTC m=+24.988150944" watchObservedRunningTime="2026-10-31 00:27:49.935058947 +0000 UTC m=+25.035601776"
Oct 31 00:27:50.672931 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:50.672896    2572 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-rrk77"
Oct 31 00:27:50.673348 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:50.672973    2572 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-rrk77"
Oct 31 00:27:50.687183 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:50.687163    2572 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-rrk77"
Oct 31 00:27:51.372752 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:51.372508    2572 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-24sxq"]
Oct 31 00:27:51.372906 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:51.372820    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-24sxq"
Oct 31 00:27:51.372966 ip-10-0-132-173 kubenswrapper[2572]: E1031 00:27:51.372930    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-24sxq" podUID="97410648-b7ae-4cf4-accf-3164c398b525"
Oct 31 00:27:51.375968 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:51.375939    2572 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-jktkx"]
Oct 31 00:27:51.376105 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:51.376036    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jktkx"
Oct 31 00:27:51.376159 ip-10-0-132-173 kubenswrapper[2572]: E1031 00:27:51.376104    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jktkx" podUID="c4ce4bad-af31-4f2e-a684-b2123e9da265"
Oct 31 00:27:51.376455 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:51.376440    2572 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-plnkm"]
Oct 31 00:27:51.376520 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:51.376510    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-plnkm"
Oct 31 00:27:51.376586 ip-10-0-132-173 kubenswrapper[2572]: E1031 00:27:51.376571    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-plnkm" podUID="85e4b969-79f8-4954-8539-0905bd5db99d"
Oct 31 00:27:51.659972 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:51.659946    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:27:51.808002 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:51.807952    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:27:52.464753 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:52.464721    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jktkx"
Oct 31 00:27:52.464987 ip-10-0-132-173 kubenswrapper[2572]: E1031 00:27:52.464825    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jktkx" podUID="c4ce4bad-af31-4f2e-a684-b2123e9da265"
Oct 31 00:27:53.464870 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:53.464840    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-plnkm"
Oct 31 00:27:53.465311 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:53.464875    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-24sxq"
Oct 31 00:27:53.465311 ip-10-0-132-173 kubenswrapper[2572]: E1031 00:27:53.465019    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-24sxq" podUID="97410648-b7ae-4cf4-accf-3164c398b525"
Oct 31 00:27:53.465311 ip-10-0-132-173 kubenswrapper[2572]: E1031 00:27:53.465135    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-plnkm" podUID="85e4b969-79f8-4954-8539-0905bd5db99d"
Oct 31 00:27:54.064522 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:54.064448    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:27:54.234798 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:54.234767    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:27:54.388433 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:54.388358    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:27:54.465501 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:54.465462    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jktkx"
Oct 31 00:27:54.465869 ip-10-0-132-173 kubenswrapper[2572]: E1031 00:27:54.465572    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jktkx" podUID="c4ce4bad-af31-4f2e-a684-b2123e9da265"
Oct 31 00:27:54.535804 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:54.535774    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:27:55.465841 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:55.465808    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-plnkm"
Oct 31 00:27:55.466396 ip-10-0-132-173 kubenswrapper[2572]: E1031 00:27:55.465941    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-plnkm" podUID="85e4b969-79f8-4954-8539-0905bd5db99d"
Oct 31 00:27:55.466396 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:55.465997    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-24sxq"
Oct 31 00:27:55.466396 ip-10-0-132-173 kubenswrapper[2572]: E1031 00:27:55.466092    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-24sxq" podUID="97410648-b7ae-4cf4-accf-3164c398b525"
Oct 31 00:27:55.590521 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:55.590485    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:27:55.786424 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:55.786347    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:27:56.373868 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:56.373840    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:27:56.465311 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:56.465282    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jktkx"
Oct 31 00:27:56.465447 ip-10-0-132-173 kubenswrapper[2572]: E1031 00:27:56.465399    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jktkx" podUID="c4ce4bad-af31-4f2e-a684-b2123e9da265"
Oct 31 00:27:56.665559 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:56.665539    2572 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-173.us-west-2.compute.internal" event="NodeReady"
Oct 31 00:27:56.665879 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:56.665654    2572 kubelet_node_status.go:477] "Fast updating node status as it just became ready"
Oct 31 00:27:56.807909 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:56.807881    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:27:56.964432 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:56.964403    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:27:57.113491 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:57.113460    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:27:57.264997 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:57.264918    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:27:57.464740 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:57.464709    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-plnkm"
Oct 31 00:27:57.464919 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:57.464900    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-24sxq"
Oct 31 00:27:57.515557 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:57.515474    2572 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Oct 31 00:27:57.515557 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:57.515492    2572 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"global-pull-secret-syncer-dockercfg-dwmlr\""
Oct 31 00:27:57.545354 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:57.545323    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-zjb6f_2c9341ef-0860-4fb6-8a86-272ac063756a/dns-node-resolver/0.log"
Oct 31 00:27:57.545491 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:57.545475    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:27:57.562958 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:57.562933    2572 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Oct 31 00:27:57.563025 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:57.562990    2572 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-m2wdr\""
Oct 31 00:27:57.697583 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:57.697555    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:27:57.913949 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:57.913918    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:27:58.069498 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:58.069467    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:27:58.456616 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:58.456575    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:27:58.464784 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:58.464765    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jktkx"
Oct 31 00:27:58.470340 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:58.470323    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-qchks_61c60450-eb50-4181-8ab7-e69e864ffb51/node-ca/0.log"
Oct 31 00:27:58.525840 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:58.525817    2572 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Oct 31 00:27:58.530389 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:58.530369    2572 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Oct 31 00:27:58.573483 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:58.573459    2572 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-check-target-dockercfg-nwzth\""
Oct 31 00:27:58.896388 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:58.896301    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:27:59.170376 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:59.170349    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/97410648-b7ae-4cf4-accf-3164c398b525-original-pull-secret\") pod \"global-pull-secret-syncer-24sxq\" (UID: \"97410648-b7ae-4cf4-accf-3164c398b525\") " pod="kube-system/global-pull-secret-syncer-24sxq"
Oct 31 00:27:59.173875 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:59.173849    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/97410648-b7ae-4cf4-accf-3164c398b525-original-pull-secret\") pod \"global-pull-secret-syncer-24sxq\" (UID: \"97410648-b7ae-4cf4-accf-3164c398b525\") " pod="kube-system/global-pull-secret-syncer-24sxq"
Oct 31 00:27:59.214363 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:59.214337    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:27:59.271350 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:59.271324    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hxhdf\" (UniqueName: \"kubernetes.io/projected/c4ce4bad-af31-4f2e-a684-b2123e9da265-kube-api-access-hxhdf\") pod \"network-check-target-jktkx\" (UID: \"c4ce4bad-af31-4f2e-a684-b2123e9da265\") " pod="openshift-network-diagnostics/network-check-target-jktkx"
Oct 31 00:27:59.271511 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:59.271360    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/85e4b969-79f8-4954-8539-0905bd5db99d-metrics-certs\") pod \"network-metrics-daemon-plnkm\" (UID: \"85e4b969-79f8-4954-8539-0905bd5db99d\") " pod="openshift-multus/network-metrics-daemon-plnkm"
Oct 31 00:27:59.271511 ip-10-0-132-173 kubenswrapper[2572]: E1031 00:27:59.271464    2572 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Oct 31 00:27:59.271581 ip-10-0-132-173 kubenswrapper[2572]: E1031 00:27:59.271535    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/85e4b969-79f8-4954-8539-0905bd5db99d-metrics-certs podName:85e4b969-79f8-4954-8539-0905bd5db99d nodeName:}" failed. No retries permitted until 2026-10-31 00:28:31.271519572 +0000 UTC m=+66.372062397 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/85e4b969-79f8-4954-8539-0905bd5db99d-metrics-certs") pod "network-metrics-daemon-plnkm" (UID: "85e4b969-79f8-4954-8539-0905bd5db99d") : secret "metrics-daemon-secret" not found
Oct 31 00:27:59.273890 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:59.273873    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hxhdf\" (UniqueName: \"kubernetes.io/projected/c4ce4bad-af31-4f2e-a684-b2123e9da265-kube-api-access-hxhdf\") pod \"network-check-target-jktkx\" (UID: \"c4ce4bad-af31-4f2e-a684-b2123e9da265\") " pod="openshift-network-diagnostics/network-check-target-jktkx"
Oct 31 00:27:59.326361 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:59.326334    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-24sxq"
Oct 31 00:27:59.370819 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:59.370795    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:27:59.385878 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:59.385850    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jktkx"
Oct 31 00:27:59.628121 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:59.628087    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:27:59.755142 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:59.755072    2572 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-24sxq"]
Oct 31 00:27:59.757863 ip-10-0-132-173 kubenswrapper[2572]: W1031 00:27:59.757840    2572 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod97410648_b7ae_4cf4_accf_3164c398b525.slice/crio-00db36ffba9650250554710ed1cdb2e16a0a488cc757cc63aaee54fabdc19df2 WatchSource:0}: Error finding container 00db36ffba9650250554710ed1cdb2e16a0a488cc757cc63aaee54fabdc19df2: Status 404 returned error can't find the container with id 00db36ffba9650250554710ed1cdb2e16a0a488cc757cc63aaee54fabdc19df2
Oct 31 00:27:59.801629 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:59.801607    2572 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-jktkx"]
Oct 31 00:27:59.814425 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:59.814405    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:27:59.973181 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:27:59.973117    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:28:00.130273 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:28:00.130152    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:28:00.411459 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:28:00.411422    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:28:00.625589 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:28:00.625558    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:28:00.693806 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:28:00.693742    2572 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-jktkx" event={"ID":"c4ce4bad-af31-4f2e-a684-b2123e9da265","Type":"ContainerStarted","Data":"ba701dec65a813195d815f7857a177f17a6126298c996127a489ccb3c59f2870"}
Oct 31 00:28:00.694816 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:28:00.694755    2572 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-24sxq" event={"ID":"97410648-b7ae-4cf4-accf-3164c398b525","Type":"ContainerStarted","Data":"00db36ffba9650250554710ed1cdb2e16a0a488cc757cc63aaee54fabdc19df2"}
Oct 31 00:28:01.209596 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:28:01.209489    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:28:01.420270 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:28:01.420236    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:28:01.611435 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:28:01.611359    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:28:01.811428 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:28:01.811395    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:28:02.016595 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:28:02.016551    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:28:02.420162 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:28:02.420128    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:28:02.863585 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:28:02.863501    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:28:03.020532 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:28:03.020494    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:28:03.225811 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:28:03.225776    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:28:03.610714 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:28:03.610632    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:28:03.812205 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:28:03.812171    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:28:03.956580 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:28:03.956548    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:28:04.112020 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:28:04.111989    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:28:04.416702 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:28:04.416663    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:28:04.707849 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:28:04.707765    2572 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-jktkx" event={"ID":"c4ce4bad-af31-4f2e-a684-b2123e9da265","Type":"ContainerStarted","Data":"59e5cfb2a7c413b4dbd6261911a0c445e4f8af19a4d2c596f4a7875bd8cb524c"}
Oct 31 00:28:04.708311 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:28:04.707911    2572 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-jktkx"
Oct 31 00:28:04.818544 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:28:04.818488    2572 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-jktkx" podStartSLOduration=34.608199742 podStartE2EDuration="38.818470739s" totalImagesPullingTime="4.210270997s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 00:27:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:28:04.818319227 +0000 UTC m=+39.918862061" watchObservedRunningTime="2026-10-31 00:28:04.818470739 +0000 UTC m=+39.919013566"
Oct 31 00:28:05.217018 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:28:05.216986    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:28:05.612150 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:28:05.612126    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:28:05.711604 ip-10-0-132-173 kubenswrapper[2572]: I1031 00:28:05.711568    2572 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-24sxq" event={"ID":"97410648-b7ae-4cf4-accf-3164c398b525","Type":"ContainerStarted","Data":"79c81bfb3b007683efff944477aa0ce42f1c42f8e18e6b881ee1ccdacee8ff4f"}
Oct 31 00:28:05.806327 ip-10-0-132-173 systemd[1]: Stopping Kubernetes Kubelet...
Oct 31 00:28:05.813729 ip-10-0-132-173 systemd[1]: kubelet.service: Deactivated successfully.
Oct 31 00:28:05.813890 ip-10-0-132-173 systemd[1]: Stopped Kubernetes Kubelet.
Oct 31 00:28:05.813939 ip-10-0-132-173 systemd[1]: kubelet.service: Consumed 2.266s CPU time.
Oct 31 00:28:05.824837 ip-10-0-132-173 systemd[1]: Starting Kubernetes Kubelet...
Oct 31 00:28:05.882446 ip-10-0-132-173 kubenswrapper[4284]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 31 00:28:05.882446 ip-10-0-132-173 kubenswrapper[4284]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Oct 31 00:28:05.882446 ip-10-0-132-173 kubenswrapper[4284]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 31 00:28:05.882446 ip-10-0-132-173 kubenswrapper[4284]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 31 00:28:05.885149 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.885040    4284 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 31 00:28:05.885149 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.885148    4284 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 31 00:28:05.885149 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.885153    4284 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 31 00:28:05.885149 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.885159    4284 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 31 00:28:05.885369 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.885164    4284 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 31 00:28:05.885369 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.885169    4284 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 31 00:28:05.885369 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.885173    4284 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 31 00:28:05.885369 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.885178    4284 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 31 00:28:05.885369 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.885181    4284 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 31 00:28:05.885369 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.885187    4284 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 31 00:28:05.885369 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.885196    4284 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 31 00:28:05.885369 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.885200    4284 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 31 00:28:05.885369 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.885204    4284 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 31 00:28:05.885369 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.885225    4284 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 31 00:28:05.885369 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.885230    4284 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 31 00:28:05.885369 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.885234    4284 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 31 00:28:05.885369 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.885238    4284 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 31 00:28:05.885369 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.885244    4284 feature_gate.go:386] unrecognized feature gate: Example
Oct 31 00:28:05.885369 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.885248    4284 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 31 00:28:05.885369 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.885251    4284 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 31 00:28:05.885369 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.885254    4284 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 31 00:28:05.885369 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.885256    4284 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 31 00:28:05.885369 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.885259    4284 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 31 00:28:05.885369 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.885264    4284 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 31 00:28:05.885852 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.885268    4284 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 31 00:28:05.885852 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.885273    4284 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 31 00:28:05.885852 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.885276    4284 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 31 00:28:05.885852 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.885280    4284 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 31 00:28:05.885852 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.885283    4284 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 31 00:28:05.885852 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.885286    4284 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 31 00:28:05.885852 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.885289    4284 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 31 00:28:05.885852 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.885292    4284 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 31 00:28:05.885852 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.885295    4284 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 31 00:28:05.885852 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.885298    4284 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 31 00:28:05.885852 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.885301    4284 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 31 00:28:05.885852 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.885307    4284 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 31 00:28:05.885852 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.885310    4284 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 31 00:28:05.885852 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.885314    4284 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 31 00:28:05.885852 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.885318    4284 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 31 00:28:05.885852 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.885321    4284 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 31 00:28:05.885852 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.885324    4284 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 31 00:28:05.885852 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.885327    4284 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 31 00:28:05.886345 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.885331    4284 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 31 00:28:05.886345 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.885334    4284 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 31 00:28:05.886345 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.885337    4284 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 31 00:28:05.886345 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.885340    4284 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 31 00:28:05.886345 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.885344    4284 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 31 00:28:05.886345 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.885346    4284 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 31 00:28:05.886345 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.885351    4284 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 31 00:28:05.886345 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.885354    4284 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 31 00:28:05.886345 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.885357    4284 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 31 00:28:05.886345 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.885361    4284 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 31 00:28:05.886345 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.885364    4284 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 31 00:28:05.886345 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.885366    4284 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 31 00:28:05.886345 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.885369    4284 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 31 00:28:05.886345 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.885372    4284 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 31 00:28:05.886345 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.885374    4284 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 31 00:28:05.886345 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.885377    4284 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 31 00:28:05.886345 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.885380    4284 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 31 00:28:05.886345 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.885383    4284 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 31 00:28:05.886345 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.885389    4284 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 31 00:28:05.886345 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.885391    4284 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 31 00:28:05.886827 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.885393    4284 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 31 00:28:05.886827 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.885396    4284 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 31 00:28:05.886827 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.885399    4284 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 31 00:28:05.886827 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.885402    4284 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 31 00:28:05.886827 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.885406    4284 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 31 00:28:05.886827 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.885409    4284 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 31 00:28:05.886827 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.885412    4284 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 31 00:28:05.886827 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.885415    4284 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 31 00:28:05.886827 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.885417    4284 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 31 00:28:05.886827 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.885420    4284 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 31 00:28:05.886827 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.885425    4284 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 31 00:28:05.886827 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.885428    4284 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 31 00:28:05.886827 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.885430    4284 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 31 00:28:05.886827 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.885434    4284 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 31 00:28:05.886827 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.885436    4284 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 31 00:28:05.886827 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.885439    4284 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 31 00:28:05.886827 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.885442    4284 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 31 00:28:05.886827 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.885444    4284 feature_gate.go:386] unrecognized feature gate: Example2
Oct 31 00:28:05.886827 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.885447    4284 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 31 00:28:05.887304 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.885449    4284 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 31 00:28:05.887304 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.885452    4284 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 31 00:28:05.887304 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.885454    4284 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 31 00:28:05.887304 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.885457    4284 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 31 00:28:05.887304 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.885461    4284 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 31 00:28:05.887304 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.885464    4284 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 31 00:28:05.887304 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.885466    4284 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 31 00:28:05.887304 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.885469    4284 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 31 00:28:05.887304 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.885471    4284 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 31 00:28:05.887304 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.885473    4284 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 31 00:28:05.887304 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.885477    4284 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 31 00:28:05.887304 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.885479    4284 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 31 00:28:05.887304 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.885482    4284 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 31 00:28:05.887304 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.885485    4284 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 31 00:28:05.887304 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.885487    4284 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 31 00:28:05.887304 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.885490    4284 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 31 00:28:05.887304 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.885492    4284 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 31 00:28:05.887304 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.885497    4284 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 31 00:28:05.887304 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.885500    4284 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 31 00:28:05.887304 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.885503    4284 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 31 00:28:05.887814 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.885505    4284 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 31 00:28:05.887814 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.885508    4284 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 31 00:28:05.887814 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.885510    4284 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 31 00:28:05.887814 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.885512    4284 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 31 00:28:05.887814 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.885515    4284 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 31 00:28:05.887814 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.886812    4284 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 31 00:28:05.887814 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.886825    4284 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 31 00:28:05.887814 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.886829    4284 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 31 00:28:05.887814 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.886832    4284 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 31 00:28:05.887814 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.886835    4284 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 31 00:28:05.887814 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.886839    4284 feature_gate.go:386] unrecognized feature gate: Example
Oct 31 00:28:05.887814 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.886841    4284 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 31 00:28:05.887814 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.886844    4284 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 31 00:28:05.887814 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.886846    4284 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 31 00:28:05.887814 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.886849    4284 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 31 00:28:05.887814 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.886853    4284 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 31 00:28:05.887814 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.886858    4284 feature_gate.go:386] unrecognized feature gate: Example2
Oct 31 00:28:05.887814 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.886861    4284 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 31 00:28:05.887814 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.886864    4284 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 31 00:28:05.888351 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.886867    4284 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 31 00:28:05.888351 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.886870    4284 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 31 00:28:05.888351 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.886873    4284 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 31 00:28:05.888351 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.886875    4284 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 31 00:28:05.888351 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.886878    4284 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 31 00:28:05.888351 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.886881    4284 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 31 00:28:05.888351 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.886884    4284 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 31 00:28:05.888351 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.886886    4284 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 31 00:28:05.888351 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.886889    4284 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 31 00:28:05.888351 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.886891    4284 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 31 00:28:05.888351 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.886894    4284 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 31 00:28:05.888351 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.886897    4284 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 31 00:28:05.888351 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.886899    4284 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 31 00:28:05.888351 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.886902    4284 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 31 00:28:05.888351 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.886904    4284 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 31 00:28:05.888351 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.886907    4284 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 31 00:28:05.888351 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.886910    4284 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 31 00:28:05.888351 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.886914    4284 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 31 00:28:05.888351 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.886916    4284 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 31 00:28:05.888351 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.886918    4284 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 31 00:28:05.888845 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.886921    4284 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 31 00:28:05.888845 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.886923    4284 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 31 00:28:05.888845 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.886926    4284 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 31 00:28:05.888845 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.886928    4284 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 31 00:28:05.888845 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.886930    4284 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 31 00:28:05.888845 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.886933    4284 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 31 00:28:05.888845 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.886935    4284 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 31 00:28:05.888845 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.886938    4284 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 31 00:28:05.888845 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.886940    4284 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 31 00:28:05.888845 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.886943    4284 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 31 00:28:05.888845 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.886946    4284 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 31 00:28:05.888845 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.886948    4284 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 31 00:28:05.888845 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.886950    4284 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 31 00:28:05.888845 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.886953    4284 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 31 00:28:05.888845 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.886955    4284 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 31 00:28:05.888845 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.886958    4284 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 31 00:28:05.888845 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.886960    4284 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 31 00:28:05.888845 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.886962    4284 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 31 00:28:05.888845 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.886964    4284 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 31 00:28:05.889317 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.886967    4284 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 31 00:28:05.889317 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.886969    4284 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 31 00:28:05.889317 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.886972    4284 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 31 00:28:05.889317 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.886974    4284 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 31 00:28:05.889317 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.886976    4284 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 31 00:28:05.889317 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.886979    4284 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 31 00:28:05.889317 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.886981    4284 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 31 00:28:05.889317 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.887000    4284 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 31 00:28:05.889317 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.887004    4284 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 31 00:28:05.889317 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.887007    4284 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 31 00:28:05.889317 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.887010    4284 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 31 00:28:05.889317 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.887013    4284 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 31 00:28:05.889317 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.887016    4284 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 31 00:28:05.889317 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.887019    4284 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 31 00:28:05.889317 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.887021    4284 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 31 00:28:05.889317 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.887024    4284 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 31 00:28:05.889317 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.887027    4284 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 31 00:28:05.889317 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.887029    4284 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 31 00:28:05.889317 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.887032    4284 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 31 00:28:05.889790 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.887034    4284 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 31 00:28:05.889790 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.887037    4284 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 31 00:28:05.889790 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.887039    4284 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 31 00:28:05.889790 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.887042    4284 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 31 00:28:05.889790 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.887046    4284 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 31 00:28:05.889790 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.887049    4284 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 31 00:28:05.889790 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.887051    4284 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 31 00:28:05.889790 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.887054    4284 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 31 00:28:05.889790 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.887057    4284 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 31 00:28:05.889790 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.887060    4284 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 31 00:28:05.889790 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.887062    4284 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 31 00:28:05.889790 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.887064    4284 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 31 00:28:05.889790 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.887067    4284 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 31 00:28:05.889790 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.887069    4284 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 31 00:28:05.889790 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.887071    4284 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 31 00:28:05.889790 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.887074    4284 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 31 00:28:05.889790 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.887076    4284 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 31 00:28:05.889790 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.887079    4284 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 31 00:28:05.889790 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.887081    4284 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 31 00:28:05.889790 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.887084    4284 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 31 00:28:05.890286 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.887086    4284 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 31 00:28:05.890286 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.887089    4284 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 31 00:28:05.890286 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.887091    4284 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 31 00:28:05.890286 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.887095    4284 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 31 00:28:05.890286 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.887099    4284 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 31 00:28:05.890286 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.887102    4284 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 31 00:28:05.890286 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.887104    4284 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 31 00:28:05.890286 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.887107    4284 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 31 00:28:05.890286 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.887111    4284 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 31 00:28:05.890286 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.887113    4284 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 31 00:28:05.890286 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.887116    4284 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 31 00:28:05.890286 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.887118    4284 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 31 00:28:05.890286 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.887121    4284 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 31 00:28:05.890286 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.887123    4284 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 31 00:28:05.890286 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.887659    4284 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 31 00:28:05.890286 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.887666    4284 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 31 00:28:05.890286 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.887670    4284 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 31 00:28:05.890286 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.887674    4284 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 31 00:28:05.890286 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.887677    4284 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 31 00:28:05.890286 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.887680    4284 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 31 00:28:05.890771 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.887682    4284 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 31 00:28:05.890771 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.887685    4284 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 31 00:28:05.890771 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.887687    4284 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 31 00:28:05.890771 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.887690    4284 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 31 00:28:05.890771 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.887693    4284 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 31 00:28:05.890771 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.887696    4284 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 31 00:28:05.890771 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.887698    4284 feature_gate.go:386] unrecognized feature gate: Example
Oct 31 00:28:05.890771 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.887701    4284 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 31 00:28:05.890771 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.887704    4284 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 31 00:28:05.890771 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.887707    4284 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 31 00:28:05.890771 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.887710    4284 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 31 00:28:05.890771 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.887713    4284 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 31 00:28:05.890771 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.887715    4284 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 31 00:28:05.890771 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.887718    4284 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 31 00:28:05.890771 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.887720    4284 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 31 00:28:05.890771 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.887723    4284 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 31 00:28:05.890771 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.887725    4284 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 31 00:28:05.890771 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.887728    4284 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 31 00:28:05.890771 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.887730    4284 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 31 00:28:05.891226 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.887733    4284 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 31 00:28:05.891226 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.887736    4284 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 31 00:28:05.891226 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.887738    4284 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 31 00:28:05.891226 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.887741    4284 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 31 00:28:05.891226 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.887743    4284 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 31 00:28:05.891226 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.887746    4284 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 31 00:28:05.891226 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.887748    4284 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 31 00:28:05.891226 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.887751    4284 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 31 00:28:05.891226 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.887753    4284 feature_gate.go:386] unrecognized feature gate: Example2
Oct 31 00:28:05.891226 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.887756    4284 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 31 00:28:05.891226 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.887759    4284 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 31 00:28:05.891226 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.887761    4284 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 31 00:28:05.891226 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.887764    4284 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 31 00:28:05.891226 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.887766    4284 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 31 00:28:05.891226 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.887769    4284 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 31 00:28:05.891226 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.887771    4284 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 31 00:28:05.891226 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.887774    4284 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 31 00:28:05.891226 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.887776    4284 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 31 00:28:05.891226 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.887781    4284 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 31 00:28:05.891226 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.887784    4284 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 31 00:28:05.891795 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.887787    4284 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 31 00:28:05.891795 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.887789    4284 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 31 00:28:05.891795 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.887792    4284 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 31 00:28:05.891795 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.887796    4284 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 31 00:28:05.891795 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.887798    4284 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 31 00:28:05.891795 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.887801    4284 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 31 00:28:05.891795 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.887804    4284 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 31 00:28:05.891795 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.887806    4284 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 31 00:28:05.891795 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.887808    4284 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 31 00:28:05.891795 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.887811    4284 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 31 00:28:05.891795 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.887813    4284 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 31 00:28:05.891795 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.887816    4284 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 31 00:28:05.891795 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.887818    4284 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 31 00:28:05.891795 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.887821    4284 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 31 00:28:05.891795 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.887824    4284 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 31 00:28:05.891795 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.887827    4284 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 31 00:28:05.891795 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.887829    4284 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 31 00:28:05.891795 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.887832    4284 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 31 00:28:05.891795 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.887834    4284 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 31 00:28:05.891795 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.887837    4284 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 31 00:28:05.892313 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.887839    4284 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 31 00:28:05.892313 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.887842    4284 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 31 00:28:05.892313 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.887845    4284 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 31 00:28:05.892313 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.887848    4284 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 31 00:28:05.892313 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.887850    4284 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 31 00:28:05.892313 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.887853    4284 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 31 00:28:05.892313 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.887855    4284 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 31 00:28:05.892313 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.887858    4284 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 31 00:28:05.892313 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.887861    4284 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 31 00:28:05.892313 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.887863    4284 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 31 00:28:05.892313 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.887867    4284 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 31 00:28:05.892313 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.887869    4284 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 31 00:28:05.892313 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.887872    4284 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 31 00:28:05.892313 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.887874    4284 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 31 00:28:05.892313 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.887877    4284 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 31 00:28:05.892313 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.887881    4284 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 31 00:28:05.892313 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.887884    4284 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 31 00:28:05.892313 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.887887    4284 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 31 00:28:05.892313 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.887889    4284 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 31 00:28:05.892313 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.887892    4284 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 31 00:28:05.892812 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.887894    4284 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 31 00:28:05.892812 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.887897    4284 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 31 00:28:05.892812 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.887899    4284 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 31 00:28:05.892812 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.887903    4284 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 31 00:28:05.892812 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.887907    4284 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 31 00:28:05.892812 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.887916    4284 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 31 00:28:05.892812 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.887920    4284 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 31 00:28:05.892812 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.887922    4284 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 31 00:28:05.892812 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.887925    4284 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 31 00:28:05.892812 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.887929    4284 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 31 00:28:05.892812 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.887932    4284 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 31 00:28:05.892812 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.887935    4284 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 31 00:28:05.892812 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.887938    4284 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 31 00:28:05.892812 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.887940    4284 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 31 00:28:05.892812 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.887943    4284 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 31 00:28:05.892812 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.887945    4284 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 31 00:28:05.892812 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.887948    4284 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 31 00:28:05.892812 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.887951    4284 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 31 00:28:05.893279 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.887953    4284 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 31 00:28:05.893279 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.887956    4284 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 31 00:28:05.893279 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.887958    4284 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 31 00:28:05.893279 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.888050    4284 flags.go:64] FLAG: --address="0.0.0.0"
Oct 31 00:28:05.893279 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.888059    4284 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Oct 31 00:28:05.893279 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.888066    4284 flags.go:64] FLAG: --anonymous-auth="true"
Oct 31 00:28:05.893279 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.888070    4284 flags.go:64] FLAG: --application-metrics-count-limit="100"
Oct 31 00:28:05.893279 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.888075    4284 flags.go:64] FLAG: --authentication-token-webhook="false"
Oct 31 00:28:05.893279 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.888078    4284 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Oct 31 00:28:05.893279 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.888084    4284 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Oct 31 00:28:05.893279 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.888088    4284 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Oct 31 00:28:05.893279 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.888091    4284 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Oct 31 00:28:05.893279 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.888094    4284 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Oct 31 00:28:05.893279 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.888097    4284 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Oct 31 00:28:05.893279 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.888100    4284 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Oct 31 00:28:05.893279 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.888103    4284 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Oct 31 00:28:05.893279 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.888106    4284 flags.go:64] FLAG: --cgroup-root=""
Oct 31 00:28:05.893279 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.888109    4284 flags.go:64] FLAG: --cgroups-per-qos="true"
Oct 31 00:28:05.893279 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.888112    4284 flags.go:64] FLAG: --client-ca-file=""
Oct 31 00:28:05.893279 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.888115    4284 flags.go:64] FLAG: --cloud-provider="external"
Oct 31 00:28:05.893279 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.888118    4284 flags.go:64] FLAG: --cluster-dns="[]"
Oct 31 00:28:05.893279 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.888122    4284 flags.go:64] FLAG: --cluster-domain=""
Oct 31 00:28:05.893279 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.888124    4284 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Oct 31 00:28:05.893812 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.888127    4284 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Oct 31 00:28:05.893812 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.888130    4284 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Oct 31 00:28:05.893812 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.888134    4284 flags.go:64] FLAG: --container-log-max-files="5"
Oct 31 00:28:05.893812 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.888138    4284 flags.go:64] FLAG: --container-log-max-size="10Mi"
Oct 31 00:28:05.893812 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.888140    4284 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Oct 31 00:28:05.893812 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.888155    4284 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Oct 31 00:28:05.893812 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.888158    4284 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Oct 31 00:28:05.893812 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.888161    4284 flags.go:64] FLAG: --contention-profiling="false"
Oct 31 00:28:05.893812 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.888165    4284 flags.go:64] FLAG: --cpu-cfs-quota="true"
Oct 31 00:28:05.893812 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.888168    4284 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Oct 31 00:28:05.893812 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.888171    4284 flags.go:64] FLAG: --cpu-manager-policy="none"
Oct 31 00:28:05.893812 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.888174    4284 flags.go:64] FLAG: --cpu-manager-policy-options=""
Oct 31 00:28:05.893812 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.888178    4284 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Oct 31 00:28:05.893812 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.888182    4284 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Oct 31 00:28:05.893812 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.888185    4284 flags.go:64] FLAG: --enable-debugging-handlers="true"
Oct 31 00:28:05.893812 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.888188    4284 flags.go:64] FLAG: --enable-load-reader="false"
Oct 31 00:28:05.893812 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.888191    4284 flags.go:64] FLAG: --enable-server="true"
Oct 31 00:28:05.893812 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.888194    4284 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Oct 31 00:28:05.893812 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.888199    4284 flags.go:64] FLAG: --event-burst="100"
Oct 31 00:28:05.893812 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.888203    4284 flags.go:64] FLAG: --event-qps="50"
Oct 31 00:28:05.893812 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.888206    4284 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Oct 31 00:28:05.893812 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.888220    4284 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Oct 31 00:28:05.893812 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.888223    4284 flags.go:64] FLAG: --eviction-hard=""
Oct 31 00:28:05.893812 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.888227    4284 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Oct 31 00:28:05.894401 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.888230    4284 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Oct 31 00:28:05.894401 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.888233    4284 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Oct 31 00:28:05.894401 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.888237    4284 flags.go:64] FLAG: --eviction-soft=""
Oct 31 00:28:05.894401 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.888240    4284 flags.go:64] FLAG: --eviction-soft-grace-period=""
Oct 31 00:28:05.894401 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.888243    4284 flags.go:64] FLAG: --exit-on-lock-contention="false"
Oct 31 00:28:05.894401 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.888246    4284 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Oct 31 00:28:05.894401 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.888249    4284 flags.go:64] FLAG: --experimental-mounter-path=""
Oct 31 00:28:05.894401 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.888252    4284 flags.go:64] FLAG: --fail-cgroupv1="true"
Oct 31 00:28:05.894401 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.888254    4284 flags.go:64] FLAG: --fail-swap-on="true"
Oct 31 00:28:05.894401 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.888257    4284 flags.go:64] FLAG: --feature-gates=""
Oct 31 00:28:05.894401 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.888261    4284 flags.go:64] FLAG: --file-check-frequency="20s"
Oct 31 00:28:05.894401 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.888264    4284 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Oct 31 00:28:05.894401 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.888267    4284 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Oct 31 00:28:05.894401 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.888270    4284 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Oct 31 00:28:05.894401 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.888273    4284 flags.go:64] FLAG: --healthz-port="10248"
Oct 31 00:28:05.894401 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.888276    4284 flags.go:64] FLAG: --help="false"
Oct 31 00:28:05.894401 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.888279    4284 flags.go:64] FLAG: --hostname-override="ip-10-0-132-173.us-west-2.compute.internal"
Oct 31 00:28:05.894401 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.888283    4284 flags.go:64] FLAG: --housekeeping-interval="10s"
Oct 31 00:28:05.894401 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.888286    4284 flags.go:64] FLAG: --http-check-frequency="20s"
Oct 31 00:28:05.894401 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.888289    4284 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Oct 31 00:28:05.894401 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.888292    4284 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Oct 31 00:28:05.894401 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.888297    4284 flags.go:64] FLAG: --image-gc-high-threshold="85"
Oct 31 00:28:05.894401 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.888299    4284 flags.go:64] FLAG: --image-gc-low-threshold="80"
Oct 31 00:28:05.894935 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.888302    4284 flags.go:64] FLAG: --image-service-endpoint=""
Oct 31 00:28:05.894935 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.888305    4284 flags.go:64] FLAG: --kernel-memcg-notification="false"
Oct 31 00:28:05.894935 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.888308    4284 flags.go:64] FLAG: --kube-api-burst="100"
Oct 31 00:28:05.894935 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.888312    4284 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Oct 31 00:28:05.894935 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.888315    4284 flags.go:64] FLAG: --kube-api-qps="50"
Oct 31 00:28:05.894935 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.888318    4284 flags.go:64] FLAG: --kube-reserved=""
Oct 31 00:28:05.894935 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.888321    4284 flags.go:64] FLAG: --kube-reserved-cgroup=""
Oct 31 00:28:05.894935 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.888324    4284 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Oct 31 00:28:05.894935 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.888327    4284 flags.go:64] FLAG: --kubelet-cgroups=""
Oct 31 00:28:05.894935 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.888329    4284 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Oct 31 00:28:05.894935 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.888332    4284 flags.go:64] FLAG: --lock-file=""
Oct 31 00:28:05.894935 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.888335    4284 flags.go:64] FLAG: --log-cadvisor-usage="false"
Oct 31 00:28:05.894935 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.888338    4284 flags.go:64] FLAG: --log-flush-frequency="5s"
Oct 31 00:28:05.894935 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.888341    4284 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Oct 31 00:28:05.894935 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.888347    4284 flags.go:64] FLAG: --log-json-split-stream="false"
Oct 31 00:28:05.894935 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.888350    4284 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Oct 31 00:28:05.894935 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.888353    4284 flags.go:64] FLAG: --log-text-split-stream="false"
Oct 31 00:28:05.894935 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.888356    4284 flags.go:64] FLAG: --logging-format="text"
Oct 31 00:28:05.894935 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.888359    4284 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Oct 31 00:28:05.894935 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.888362    4284 flags.go:64] FLAG: --make-iptables-util-chains="true"
Oct 31 00:28:05.894935 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.888365    4284 flags.go:64] FLAG: --manifest-url=""
Oct 31 00:28:05.894935 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.888368    4284 flags.go:64] FLAG: --manifest-url-header=""
Oct 31 00:28:05.894935 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.888372    4284 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Oct 31 00:28:05.894935 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.888375    4284 flags.go:64] FLAG: --max-open-files="1000000"
Oct 31 00:28:05.894935 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.888379    4284 flags.go:64] FLAG: --max-pods="110"
Oct 31 00:28:05.895573 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.888382    4284 flags.go:64] FLAG: --maximum-dead-containers="-1"
Oct 31 00:28:05.895573 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.888385    4284 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Oct 31 00:28:05.895573 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.888388    4284 flags.go:64] FLAG: --memory-manager-policy="None"
Oct 31 00:28:05.895573 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.888391    4284 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Oct 31 00:28:05.895573 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.888394    4284 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Oct 31 00:28:05.895573 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.888398    4284 flags.go:64] FLAG: --node-ip="0.0.0.0"
Oct 31 00:28:05.895573 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.888401    4284 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Oct 31 00:28:05.895573 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.888408    4284 flags.go:64] FLAG: --node-status-max-images="50"
Oct 31 00:28:05.895573 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.888411    4284 flags.go:64] FLAG: --node-status-update-frequency="10s"
Oct 31 00:28:05.895573 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.888414    4284 flags.go:64] FLAG: --oom-score-adj="-999"
Oct 31 00:28:05.895573 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.888418    4284 flags.go:64] FLAG: --pod-cidr=""
Oct 31 00:28:05.895573 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.888420    4284 flags.go:64] FLAG: --pod-manifest-path=""
Oct 31 00:28:05.895573 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.888424    4284 flags.go:64] FLAG: --pod-max-pids="-1"
Oct 31 00:28:05.895573 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.888427    4284 flags.go:64] FLAG: --pods-per-core="0"
Oct 31 00:28:05.895573 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.888430    4284 flags.go:64] FLAG: --port="10250"
Oct 31 00:28:05.895573 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.888433    4284 flags.go:64] FLAG: --protect-kernel-defaults="false"
Oct 31 00:28:05.895573 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.888435    4284 flags.go:64] FLAG: --provider-id="aws:///us-west-2a/i-00e6ca61ed28a7c40"
Oct 31 00:28:05.895573 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.888438    4284 flags.go:64] FLAG: --qos-reserved=""
Oct 31 00:28:05.895573 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.888441    4284 flags.go:64] FLAG: --read-only-port="10255"
Oct 31 00:28:05.895573 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.888444    4284 flags.go:64] FLAG: --register-node="true"
Oct 31 00:28:05.895573 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.888447    4284 flags.go:64] FLAG: --register-with-taints=""
Oct 31 00:28:05.895573 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.888451    4284 flags.go:64] FLAG: --registry-burst="10"
Oct 31 00:28:05.895573 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.888454    4284 flags.go:64] FLAG: --registry-qps="5"
Oct 31 00:28:05.895573 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.888456    4284 flags.go:64] FLAG: --reserved-cpus=""
Oct 31 00:28:05.895573 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.888459    4284 flags.go:64] FLAG: --reserved-memory=""
Oct 31 00:28:05.895573 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.888463    4284 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Oct 31 00:28:05.896258 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.888466    4284 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Oct 31 00:28:05.896258 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.888469    4284 flags.go:64] FLAG: --rotate-certificates="false"
Oct 31 00:28:05.896258 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.888472    4284 flags.go:64] FLAG: --rotate-server-certificates="false"
Oct 31 00:28:05.896258 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.888475    4284 flags.go:64] FLAG: --runonce="false"
Oct 31 00:28:05.896258 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.888478    4284 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Oct 31 00:28:05.896258 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.888481    4284 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Oct 31 00:28:05.896258 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.888484    4284 flags.go:64] FLAG: --seccomp-default="false"
Oct 31 00:28:05.896258 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.888486    4284 flags.go:64] FLAG: --serialize-image-pulls="true"
Oct 31 00:28:05.896258 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.888489    4284 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Oct 31 00:28:05.896258 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.888493    4284 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Oct 31 00:28:05.896258 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.888496    4284 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Oct 31 00:28:05.896258 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.888501    4284 flags.go:64] FLAG: --storage-driver-password="root"
Oct 31 00:28:05.896258 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.888503    4284 flags.go:64] FLAG: --storage-driver-secure="false"
Oct 31 00:28:05.896258 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.888506    4284 flags.go:64] FLAG: --storage-driver-table="stats"
Oct 31 00:28:05.896258 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.888509    4284 flags.go:64] FLAG: --storage-driver-user="root"
Oct 31 00:28:05.896258 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.888512    4284 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Oct 31 00:28:05.896258 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.888515    4284 flags.go:64] FLAG: --sync-frequency="1m0s"
Oct 31 00:28:05.896258 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.888518    4284 flags.go:64] FLAG: --system-cgroups=""
Oct 31 00:28:05.896258 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.888520    4284 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Oct 31 00:28:05.896258 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.888526    4284 flags.go:64] FLAG: --system-reserved-cgroup=""
Oct 31 00:28:05.896258 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.888528    4284 flags.go:64] FLAG: --tls-cert-file=""
Oct 31 00:28:05.896258 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.888531    4284 flags.go:64] FLAG: --tls-cipher-suites="[]"
Oct 31 00:28:05.896258 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.888535    4284 flags.go:64] FLAG: --tls-min-version=""
Oct 31 00:28:05.896258 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.888538    4284 flags.go:64] FLAG: --tls-private-key-file=""
Oct 31 00:28:05.896258 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.888540    4284 flags.go:64] FLAG: --topology-manager-policy="none"
Oct 31 00:28:05.896877 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.888543    4284 flags.go:64] FLAG: --topology-manager-policy-options=""
Oct 31 00:28:05.896877 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.888546    4284 flags.go:64] FLAG: --topology-manager-scope="container"
Oct 31 00:28:05.896877 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.888549    4284 flags.go:64] FLAG: --v="2"
Oct 31 00:28:05.896877 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.888553    4284 flags.go:64] FLAG: --version="false"
Oct 31 00:28:05.896877 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.888558    4284 flags.go:64] FLAG: --vmodule=""
Oct 31 00:28:05.896877 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.888562    4284 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Oct 31 00:28:05.896877 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.888577    4284 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Oct 31 00:28:05.896877 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.888714    4284 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 31 00:28:05.896877 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.888718    4284 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 31 00:28:05.896877 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.888721    4284 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 31 00:28:05.896877 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.888723    4284 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 31 00:28:05.896877 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.888726    4284 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 31 00:28:05.896877 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.888729    4284 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 31 00:28:05.896877 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.888731    4284 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 31 00:28:05.896877 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.888734    4284 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 31 00:28:05.896877 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.888737    4284 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 31 00:28:05.896877 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.888739    4284 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 31 00:28:05.896877 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.888742    4284 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 31 00:28:05.896877 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.888746    4284 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 31 00:28:05.896877 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.888748    4284 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 31 00:28:05.896877 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.888751    4284 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 31 00:28:05.897388 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.888753    4284 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 31 00:28:05.897388 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.888756    4284 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 31 00:28:05.897388 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.888758    4284 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 31 00:28:05.897388 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.888761    4284 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 31 00:28:05.897388 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.888763    4284 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 31 00:28:05.897388 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.888765    4284 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 31 00:28:05.897388 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.888768    4284 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 31 00:28:05.897388 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.888770    4284 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 31 00:28:05.897388 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.888773    4284 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 31 00:28:05.897388 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.888775    4284 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 31 00:28:05.897388 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.888778    4284 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 31 00:28:05.897388 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.888780    4284 feature_gate.go:386] unrecognized feature gate: Example2
Oct 31 00:28:05.897388 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.888782    4284 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 31 00:28:05.897388 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.888785    4284 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 31 00:28:05.897388 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.888787    4284 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 31 00:28:05.897388 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.888790    4284 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 31 00:28:05.897388 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.888792    4284 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 31 00:28:05.897388 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.888794    4284 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 31 00:28:05.897388 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.888797    4284 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 31 00:28:05.897388 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.888799    4284 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 31 00:28:05.897933 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.888801    4284 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 31 00:28:05.897933 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.888804    4284 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 31 00:28:05.897933 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.888806    4284 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 31 00:28:05.897933 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.888808    4284 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 31 00:28:05.897933 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.888811    4284 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 31 00:28:05.897933 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.888813    4284 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 31 00:28:05.897933 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.888816    4284 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 31 00:28:05.897933 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.888818    4284 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 31 00:28:05.897933 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.888821    4284 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 31 00:28:05.897933 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.888828    4284 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 31 00:28:05.897933 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.888833    4284 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 31 00:28:05.897933 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.888835    4284 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 31 00:28:05.897933 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.888838    4284 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 31 00:28:05.897933 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.888841    4284 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 31 00:28:05.897933 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.888843    4284 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 31 00:28:05.897933 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.888846    4284 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 31 00:28:05.897933 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.888848    4284 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 31 00:28:05.897933 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.888851    4284 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 31 00:28:05.897933 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.888853    4284 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 31 00:28:05.898611 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.888856    4284 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 31 00:28:05.898611 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.888858    4284 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 31 00:28:05.898611 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.888861    4284 feature_gate.go:386] unrecognized feature gate: Example
Oct 31 00:28:05.898611 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.888863    4284 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 31 00:28:05.898611 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.888866    4284 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 31 00:28:05.898611 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.888868    4284 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 31 00:28:05.898611 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.888871    4284 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 31 00:28:05.898611 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.888873    4284 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 31 00:28:05.898611 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.888876    4284 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 31 00:28:05.898611 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.888879    4284 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 31 00:28:05.898611 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.888883    4284 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 31 00:28:05.898611 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.888886    4284 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 31 00:28:05.898611 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.888889    4284 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 31 00:28:05.898611 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.888891    4284 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 31 00:28:05.898611 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.888893    4284 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 31 00:28:05.898611 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.888896    4284 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 31 00:28:05.898611 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.888900    4284 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 31 00:28:05.898611 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.888903    4284 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 31 00:28:05.898611 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.888907    4284 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 31 00:28:05.899112 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.888909    4284 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 31 00:28:05.899112 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.888912    4284 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 31 00:28:05.899112 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.888914    4284 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 31 00:28:05.899112 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.888918    4284 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 31 00:28:05.899112 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.888923    4284 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 31 00:28:05.899112 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.888925    4284 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 31 00:28:05.899112 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.888928    4284 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 31 00:28:05.899112 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.888930    4284 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 31 00:28:05.899112 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.888932    4284 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 31 00:28:05.899112 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.888935    4284 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 31 00:28:05.899112 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.888938    4284 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 31 00:28:05.899112 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.888940    4284 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 31 00:28:05.899112 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.888943    4284 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 31 00:28:05.899112 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.888945    4284 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 31 00:28:05.899112 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.888948    4284 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 31 00:28:05.899112 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.888950    4284 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 31 00:28:05.899112 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.888952    4284 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 31 00:28:05.899112 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.888955    4284 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 31 00:28:05.899112 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.888957    4284 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 31 00:28:05.899112 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.888960    4284 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 31 00:28:05.899888 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.888962    4284 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 31 00:28:05.899888 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.888965    4284 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 31 00:28:05.899888 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.888967    4284 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 31 00:28:05.899888 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.888969    4284 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 31 00:28:05.899888 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.888972    4284 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 31 00:28:05.899888 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.888974    4284 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 31 00:28:05.899888 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.888977    4284 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 31 00:28:05.899888 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.888979    4284 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 31 00:28:05.899888 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.888982    4284 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 31 00:28:05.899888 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.888984    4284 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 31 00:28:05.899888 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.888986    4284 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 31 00:28:05.899888 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.888989    4284 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 31 00:28:05.899888 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.888991    4284 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 31 00:28:05.899888 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.888993    4284 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 31 00:28:05.899888 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.889018    4284 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 31 00:28:05.899888 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.889064    4284 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 31 00:28:05.900607 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.889070    4284 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 31 00:28:05.900607 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.889073    4284 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 31 00:28:05.900607 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.889077    4284 feature_gate.go:386] unrecognized feature gate: Example
Oct 31 00:28:05.900607 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.889079    4284 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 31 00:28:05.900607 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.889082    4284 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 31 00:28:05.900607 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.889084    4284 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 31 00:28:05.900607 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.889086    4284 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 31 00:28:05.900607 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.889089    4284 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 31 00:28:05.900607 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.889091    4284 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 31 00:28:05.900607 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.889095    4284 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 31 00:28:05.900607 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.889099    4284 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 31 00:28:05.900607 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.889102    4284 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 31 00:28:05.900607 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.889104    4284 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 31 00:28:05.900607 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.889107    4284 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 31 00:28:05.900607 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.889109    4284 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 31 00:28:05.900607 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.889112    4284 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 31 00:28:05.900607 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.889115    4284 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 31 00:28:05.900607 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.889119    4284 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 31 00:28:05.900607 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.889122    4284 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 31 00:28:05.901081 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.889124    4284 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 31 00:28:05.901081 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.889127    4284 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 31 00:28:05.901081 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.889129    4284 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 31 00:28:05.901081 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.889131    4284 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 31 00:28:05.901081 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.889134    4284 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 31 00:28:05.901081 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.889136    4284 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 31 00:28:05.901081 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.889139    4284 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 31 00:28:05.901081 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.889141    4284 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 31 00:28:05.901081 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.889143    4284 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 31 00:28:05.901081 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.889146    4284 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 31 00:28:05.901081 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.889148    4284 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 31 00:28:05.901081 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.889151    4284 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 31 00:28:05.901081 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.889153    4284 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 31 00:28:05.901081 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.889158    4284 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 31 00:28:05.901081 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.889161    4284 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 31 00:28:05.901081 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.889164    4284 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 31 00:28:05.901081 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.889166    4284 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 31 00:28:05.901081 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.889169    4284 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 31 00:28:05.901081 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.889171    4284 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 31 00:28:05.901081 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.889174    4284 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 31 00:28:05.901803 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.889176    4284 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 31 00:28:05.901803 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.889179    4284 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 31 00:28:05.901803 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.889181    4284 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 31 00:28:05.901803 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.889183    4284 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 31 00:28:05.901803 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.889186    4284 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 31 00:28:05.901803 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.889188    4284 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 31 00:28:05.901803 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.889191    4284 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 31 00:28:05.901803 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.889193    4284 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 31 00:28:05.901803 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.889196    4284 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 31 00:28:05.901803 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.889198    4284 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 31 00:28:05.901803 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.889201    4284 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 31 00:28:05.901803 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.889203    4284 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 31 00:28:05.901803 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.889205    4284 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 31 00:28:05.901803 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.889223    4284 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 31 00:28:05.901803 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.889226    4284 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 31 00:28:05.901803 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.889229    4284 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 31 00:28:05.901803 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.889231    4284 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 31 00:28:05.901803 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.889234    4284 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 31 00:28:05.901803 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.889236    4284 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 31 00:28:05.902483 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.889239    4284 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 31 00:28:05.902483 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.889242    4284 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 31 00:28:05.902483 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.889244    4284 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 31 00:28:05.902483 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.889247    4284 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 31 00:28:05.902483 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.889249    4284 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 31 00:28:05.902483 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.889252    4284 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 31 00:28:05.902483 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.889256    4284 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 31 00:28:05.902483 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.889259    4284 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 31 00:28:05.902483 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.889262    4284 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 31 00:28:05.902483 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.889265    4284 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 31 00:28:05.902483 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.889267    4284 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 31 00:28:05.902483 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.889270    4284 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 31 00:28:05.902483 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.889272    4284 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 31 00:28:05.902483 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.889275    4284 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 31 00:28:05.902483 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.889278    4284 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 31 00:28:05.902483 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.889280    4284 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 31 00:28:05.902483 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.889283    4284 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 31 00:28:05.902483 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.889285    4284 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 31 00:28:05.902483 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.889288    4284 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 31 00:28:05.902940 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.889290    4284 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 31 00:28:05.902940 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.889292    4284 feature_gate.go:386] unrecognized feature gate: Example2
Oct 31 00:28:05.902940 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.889295    4284 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 31 00:28:05.902940 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.889297    4284 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 31 00:28:05.902940 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.889299    4284 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 31 00:28:05.902940 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.889302    4284 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 31 00:28:05.902940 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.889304    4284 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 31 00:28:05.902940 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.889307    4284 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 31 00:28:05.902940 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.889309    4284 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 31 00:28:05.902940 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.889311    4284 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 31 00:28:05.902940 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.889314    4284 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 31 00:28:05.902940 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.889316    4284 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 31 00:28:05.902940 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.889319    4284 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 31 00:28:05.902940 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.889322    4284 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 31 00:28:05.902940 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.889324    4284 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 31 00:28:05.902940 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.889327    4284 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 31 00:28:05.902940 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.889329    4284 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 31 00:28:05.902940 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.889332    4284 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 31 00:28:05.902940 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.889334    4284 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 31 00:28:05.902940 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.889339    4284 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 31 00:28:05.903750 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.889341    4284 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 31 00:28:05.903750 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.889344    4284 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 31 00:28:05.903750 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.889346    4284 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 31 00:28:05.903750 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.889350    4284 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 31 00:28:05.903750 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.889352    4284 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 31 00:28:05.903750 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.889355    4284 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 31 00:28:05.903750 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.889357    4284 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 31 00:28:05.903750 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.889359    4284 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 31 00:28:05.903750 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.893192    4284 server.go:525] "Kubelet version" kubeletVersion="v1.35.9"
Oct 31 00:28:05.903750 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.893224    4284 server.go:527] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Oct 31 00:28:05.903750 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.893238    4284 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Oct 31 00:28:05.903750 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.893243    4284 watchdog_linux.go:138] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Oct 31 00:28:05.903750 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.893307    4284 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 31 00:28:05.903750 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.893313    4284 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 31 00:28:05.903750 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.893316    4284 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 31 00:28:05.903750 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.893319    4284 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 31 00:28:05.903750 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.893321    4284 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 31 00:28:05.903750 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.893324    4284 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 31 00:28:05.903750 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.893326    4284 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 31 00:28:05.904420 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.893329    4284 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 31 00:28:05.904420 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.893331    4284 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 31 00:28:05.904420 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.893334    4284 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 31 00:28:05.904420 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.893336    4284 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 31 00:28:05.904420 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.893339    4284 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 31 00:28:05.904420 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.893341    4284 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 31 00:28:05.904420 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.893344    4284 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 31 00:28:05.904420 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.893346    4284 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 31 00:28:05.904420 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.893349    4284 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 31 00:28:05.904420 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.893351    4284 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 31 00:28:05.904420 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.893354    4284 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 31 00:28:05.904420 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.893356    4284 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 31 00:28:05.904420 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.893359    4284 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 31 00:28:05.904420 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.893361    4284 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 31 00:28:05.904420 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.893364    4284 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 31 00:28:05.904420 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.893366    4284 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 31 00:28:05.904420 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.893368    4284 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 31 00:28:05.904420 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.893371    4284 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 31 00:28:05.904420 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.893373    4284 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 31 00:28:05.904420 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.893376    4284 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 31 00:28:05.904912 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.893378    4284 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 31 00:28:05.904912 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.893381    4284 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 31 00:28:05.904912 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.893385    4284 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 31 00:28:05.904912 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.893388    4284 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 31 00:28:05.904912 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.893391    4284 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 31 00:28:05.904912 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.893393    4284 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 31 00:28:05.904912 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.893396    4284 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 31 00:28:05.904912 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.893398    4284 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 31 00:28:05.904912 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.893401    4284 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 31 00:28:05.904912 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.893405    4284 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 31 00:28:05.904912 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.893408    4284 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 31 00:28:05.904912 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.893411    4284 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 31 00:28:05.904912 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.893414    4284 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 31 00:28:05.904912 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.893417    4284 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 31 00:28:05.904912 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.893420    4284 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 31 00:28:05.904912 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.893422    4284 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 31 00:28:05.904912 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.893425    4284 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 31 00:28:05.904912 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.893427    4284 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 31 00:28:05.904912 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.893429    4284 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 31 00:28:05.905378 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.893432    4284 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 31 00:28:05.905378 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.893434    4284 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 31 00:28:05.905378 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.893437    4284 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 31 00:28:05.905378 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.893439    4284 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 31 00:28:05.905378 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.893442    4284 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 31 00:28:05.905378 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.893444    4284 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 31 00:28:05.905378 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.893446    4284 feature_gate.go:386] unrecognized feature gate: Example
Oct 31 00:28:05.905378 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.893449    4284 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 31 00:28:05.905378 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.893451    4284 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 31 00:28:05.905378 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.893454    4284 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 31 00:28:05.905378 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.893457    4284 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 31 00:28:05.905378 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.893459    4284 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 31 00:28:05.905378 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.893462    4284 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 31 00:28:05.905378 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.893464    4284 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 31 00:28:05.905378 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.893467    4284 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 31 00:28:05.905378 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.893470    4284 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 31 00:28:05.905378 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.893478    4284 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 31 00:28:05.905378 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.893481    4284 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 31 00:28:05.905378 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.893484    4284 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 31 00:28:05.905821 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.893487    4284 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 31 00:28:05.905821 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.893489    4284 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 31 00:28:05.905821 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.893492    4284 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 31 00:28:05.905821 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.893494    4284 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 31 00:28:05.905821 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.893497    4284 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 31 00:28:05.905821 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.893499    4284 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 31 00:28:05.905821 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.893502    4284 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 31 00:28:05.905821 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.893504    4284 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 31 00:28:05.905821 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.893506    4284 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 31 00:28:05.905821 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.893509    4284 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 31 00:28:05.905821 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.893511    4284 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 31 00:28:05.905821 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.893514    4284 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 31 00:28:05.905821 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.893516    4284 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 31 00:28:05.905821 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.893519    4284 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 31 00:28:05.905821 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.893521    4284 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 31 00:28:05.905821 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.893524    4284 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 31 00:28:05.905821 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.893526    4284 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 31 00:28:05.905821 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.893529    4284 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 31 00:28:05.905821 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.893531    4284 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 31 00:28:05.905821 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.893533    4284 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 31 00:28:05.906315 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.893536    4284 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 31 00:28:05.906315 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.893539    4284 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 31 00:28:05.906315 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.893541    4284 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 31 00:28:05.906315 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.893543    4284 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 31 00:28:05.906315 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.893546    4284 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 31 00:28:05.906315 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.893548    4284 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 31 00:28:05.906315 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.893551    4284 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 31 00:28:05.906315 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.893553    4284 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 31 00:28:05.906315 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.893556    4284 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 31 00:28:05.906315 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.893559    4284 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 31 00:28:05.906315 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.893568    4284 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 31 00:28:05.906315 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.893570    4284 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 31 00:28:05.906315 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.893572    4284 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 31 00:28:05.906315 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.893575    4284 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 31 00:28:05.906315 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.893579    4284 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 31 00:28:05.906315 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.893583    4284 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 31 00:28:05.906315 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.893585    4284 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 31 00:28:05.906315 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.893588    4284 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 31 00:28:05.906315 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.893591    4284 feature_gate.go:386] unrecognized feature gate: Example2
Oct 31 00:28:05.906768 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.893593    4284 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 31 00:28:05.906768 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.893596    4284 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 31 00:28:05.906768 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.893621    4284 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 31 00:28:05.906768 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.893790    4284 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 31 00:28:05.906768 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.893794    4284 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 31 00:28:05.906768 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.893797    4284 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 31 00:28:05.906768 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.893800    4284 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 31 00:28:05.906768 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.893802    4284 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 31 00:28:05.906768 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.893805    4284 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 31 00:28:05.906768 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.893807    4284 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 31 00:28:05.906768 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.893809    4284 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 31 00:28:05.906768 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.893812    4284 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 31 00:28:05.906768 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.893815    4284 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 31 00:28:05.906768 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.893817    4284 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 31 00:28:05.906768 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.893820    4284 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 31 00:28:05.906768 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.893823    4284 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 31 00:28:05.907144 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.893825    4284 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 31 00:28:05.907144 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.893828    4284 feature_gate.go:386] unrecognized feature gate: Example
Oct 31 00:28:05.907144 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.893830    4284 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 31 00:28:05.907144 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.893833    4284 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 31 00:28:05.907144 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.893835    4284 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 31 00:28:05.907144 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.893837    4284 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 31 00:28:05.907144 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.893840    4284 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 31 00:28:05.907144 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.893843    4284 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 31 00:28:05.907144 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.893850    4284 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 31 00:28:05.907144 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.893853    4284 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 31 00:28:05.907144 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.893855    4284 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 31 00:28:05.907144 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.893858    4284 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 31 00:28:05.907144 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.893860    4284 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 31 00:28:05.907144 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.893862    4284 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 31 00:28:05.907144 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.893865    4284 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 31 00:28:05.907144 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.893867    4284 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 31 00:28:05.907144 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.893869    4284 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 31 00:28:05.907144 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.893873    4284 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 31 00:28:05.907144 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.893877    4284 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 31 00:28:05.907790 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.893880    4284 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 31 00:28:05.907790 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.893882    4284 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 31 00:28:05.907790 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.893885    4284 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 31 00:28:05.907790 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.893887    4284 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 31 00:28:05.907790 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.893890    4284 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 31 00:28:05.907790 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.893893    4284 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 31 00:28:05.907790 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.893896    4284 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 31 00:28:05.907790 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.893898    4284 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 31 00:28:05.907790 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.893901    4284 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 31 00:28:05.907790 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.893903    4284 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 31 00:28:05.907790 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.893906    4284 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 31 00:28:05.907790 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.893908    4284 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 31 00:28:05.907790 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.893910    4284 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 31 00:28:05.907790 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.893913    4284 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 31 00:28:05.907790 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.893915    4284 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 31 00:28:05.907790 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.893918    4284 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 31 00:28:05.907790 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.893920    4284 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 31 00:28:05.907790 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.893923    4284 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 31 00:28:05.907790 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.893925    4284 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 31 00:28:05.907790 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.893928    4284 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 31 00:28:05.908330 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.893930    4284 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 31 00:28:05.908330 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.893932    4284 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 31 00:28:05.908330 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.893940    4284 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 31 00:28:05.908330 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.893943    4284 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 31 00:28:05.908330 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.893945    4284 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 31 00:28:05.908330 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.893948    4284 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 31 00:28:05.908330 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.893950    4284 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 31 00:28:05.908330 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.893952    4284 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 31 00:28:05.908330 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.893955    4284 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 31 00:28:05.908330 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.893957    4284 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 31 00:28:05.908330 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.893960    4284 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 31 00:28:05.908330 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.893962    4284 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 31 00:28:05.908330 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.893965    4284 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 31 00:28:05.908330 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.893967    4284 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 31 00:28:05.908330 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.893969    4284 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 31 00:28:05.908330 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.893972    4284 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 31 00:28:05.908330 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.893974    4284 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 31 00:28:05.908330 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.893977    4284 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 31 00:28:05.908330 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.893979    4284 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 31 00:28:05.908330 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.893981    4284 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 31 00:28:05.908823 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.893984    4284 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 31 00:28:05.908823 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.893986    4284 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 31 00:28:05.908823 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.893988    4284 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 31 00:28:05.908823 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.893991    4284 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 31 00:28:05.908823 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.893993    4284 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 31 00:28:05.908823 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.893995    4284 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 31 00:28:05.908823 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.893998    4284 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 31 00:28:05.908823 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.894000    4284 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 31 00:28:05.908823 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.894003    4284 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 31 00:28:05.908823 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.894005    4284 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 31 00:28:05.908823 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.894008    4284 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 31 00:28:05.908823 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.894010    4284 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 31 00:28:05.908823 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.894012    4284 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 31 00:28:05.908823 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.894015    4284 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 31 00:28:05.908823 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.894017    4284 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 31 00:28:05.908823 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.894025    4284 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 31 00:28:05.908823 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.894028    4284 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 31 00:28:05.908823 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.894030    4284 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 31 00:28:05.908823 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.894032    4284 feature_gate.go:386] unrecognized feature gate: Example2
Oct 31 00:28:05.909285 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.894035    4284 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 31 00:28:05.909285 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.894037    4284 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 31 00:28:05.909285 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.894040    4284 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 31 00:28:05.909285 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.894042    4284 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 31 00:28:05.909285 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.894045    4284 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 31 00:28:05.909285 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.894047    4284 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 31 00:28:05.909285 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.894049    4284 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 31 00:28:05.909285 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.894066    4284 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 31 00:28:05.909285 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.894073    4284 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 31 00:28:05.909285 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.894076    4284 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 31 00:28:05.909285 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.894079    4284 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 31 00:28:05.909285 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.894082    4284 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 31 00:28:05.909285 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.894085    4284 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 31 00:28:05.909285 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.894087    4284 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 31 00:28:05.909285 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.894089    4284 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 31 00:28:05.909285 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.894112    4284 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 31 00:28:05.909654 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.894153    4284 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 31 00:28:05.909654 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.894157    4284 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 31 00:28:05.909654 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.894160    4284 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 31 00:28:05.909654 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.894162    4284 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 31 00:28:05.909654 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.894166    4284 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 31 00:28:05.909654 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.894169    4284 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 31 00:28:05.909654 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.894171    4284 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 31 00:28:05.909654 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.894174    4284 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 31 00:28:05.909654 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.894176    4284 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 31 00:28:05.909654 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.894179    4284 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 31 00:28:05.909654 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.894182    4284 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 31 00:28:05.909654 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.894184    4284 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 31 00:28:05.909654 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.894186    4284 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 31 00:28:05.909654 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.894198    4284 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 31 00:28:05.909654 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.894201    4284 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 31 00:28:05.909654 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.894203    4284 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 31 00:28:05.909654 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.894205    4284 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 31 00:28:05.909654 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.894222    4284 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 31 00:28:05.909654 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.894225    4284 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 31 00:28:05.909654 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.894233    4284 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 31 00:28:05.910123 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.894236    4284 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 31 00:28:05.910123 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.894238    4284 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 31 00:28:05.910123 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.894241    4284 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 31 00:28:05.910123 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.894243    4284 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 31 00:28:05.910123 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.894246    4284 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 31 00:28:05.910123 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.894249    4284 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 31 00:28:05.910123 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.894252    4284 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 31 00:28:05.910123 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.894254    4284 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 31 00:28:05.910123 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.894256    4284 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 31 00:28:05.910123 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.894259    4284 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 31 00:28:05.910123 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.894261    4284 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 31 00:28:05.910123 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.894264    4284 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 31 00:28:05.910123 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.894266    4284 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 31 00:28:05.910123 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.894269    4284 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 31 00:28:05.910123 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.894271    4284 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 31 00:28:05.910123 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.894274    4284 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 31 00:28:05.910123 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.894277    4284 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 31 00:28:05.910123 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.894280    4284 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 31 00:28:05.910123 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.894283    4284 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 31 00:28:05.910624 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.894286    4284 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 31 00:28:05.910624 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.894288    4284 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 31 00:28:05.910624 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.894290    4284 feature_gate.go:386] unrecognized feature gate: Example2
Oct 31 00:28:05.910624 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.894293    4284 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 31 00:28:05.910624 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.894295    4284 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 31 00:28:05.910624 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.894298    4284 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 31 00:28:05.910624 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.894300    4284 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 31 00:28:05.910624 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.894308    4284 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 31 00:28:05.910624 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.894311    4284 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 31 00:28:05.910624 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.894313    4284 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 31 00:28:05.910624 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.894315    4284 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 31 00:28:05.910624 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.894318    4284 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 31 00:28:05.910624 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.894320    4284 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 31 00:28:05.910624 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.894323    4284 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 31 00:28:05.910624 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.894325    4284 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 31 00:28:05.910624 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.894328    4284 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 31 00:28:05.910624 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.894330    4284 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 31 00:28:05.910624 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.894332    4284 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 31 00:28:05.910624 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.894336    4284 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 31 00:28:05.911066 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.894339    4284 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 31 00:28:05.911066 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.894341    4284 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 31 00:28:05.911066 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.894344    4284 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 31 00:28:05.911066 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.894346    4284 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 31 00:28:05.911066 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.894348    4284 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 31 00:28:05.911066 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.894351    4284 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 31 00:28:05.911066 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.894353    4284 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 31 00:28:05.911066 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.894356    4284 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 31 00:28:05.911066 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.894358    4284 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 31 00:28:05.911066 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.894360    4284 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 31 00:28:05.911066 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.894364    4284 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 31 00:28:05.911066 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.894366    4284 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 31 00:28:05.911066 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.894369    4284 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 31 00:28:05.911066 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.894371    4284 feature_gate.go:386] unrecognized feature gate: Example
Oct 31 00:28:05.911066 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.894374    4284 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 31 00:28:05.911066 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.894376    4284 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 31 00:28:05.911066 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.894378    4284 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 31 00:28:05.911066 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.894381    4284 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 31 00:28:05.911066 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.894383    4284 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 31 00:28:05.911066 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.894385    4284 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 31 00:28:05.911625 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.894388    4284 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 31 00:28:05.911625 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.894395    4284 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 31 00:28:05.911625 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.894398    4284 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 31 00:28:05.911625 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.894400    4284 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 31 00:28:05.911625 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.894403    4284 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 31 00:28:05.911625 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.894405    4284 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 31 00:28:05.911625 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.894408    4284 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 31 00:28:05.911625 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.894410    4284 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 31 00:28:05.911625 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.894413    4284 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 31 00:28:05.911625 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.894417    4284 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 31 00:28:05.911625 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.894420    4284 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 31 00:28:05.911625 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.894424    4284 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 31 00:28:05.911625 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.894427    4284 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 31 00:28:05.911625 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.894430    4284 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 31 00:28:05.911625 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.894432    4284 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 31 00:28:05.911625 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.894434    4284 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 31 00:28:05.911625 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.894439    4284 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 31 00:28:05.911625 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.894442    4284 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 31 00:28:05.911625 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.894445    4284 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 31 00:28:05.912087 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.894448    4284 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 31 00:28:05.912087 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.894450    4284 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 31 00:28:05.912087 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.894453    4284 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 31 00:28:05.912087 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.894456    4284 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 31 00:28:05.912087 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.894459    4284 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 31 00:28:05.912087 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.894461    4284 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 31 00:28:05.912087 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.894463    4284 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 31 00:28:05.912087 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.894466    4284 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 31 00:28:05.912087 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:05.894469    4284 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 31 00:28:05.912087 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.894525    4284 server.go:957] "Client rotation is on, will bootstrap in background"
Oct 31 00:28:05.912087 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.895541    4284 bootstrap.go:85] "Current kubeconfig file contents are still valid, no bootstrap necessary"
Oct 31 00:28:05.912087 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.895588    4284 certificate_store.go:147] "Loading cert/key pair from a file" filePath="/var/lib/kubelet/pki/kubelet-client-current.pem"
Oct 31 00:28:05.912087 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.895796    4284 server.go:1014] "Starting client certificate rotation"
Oct 31 00:28:05.912087 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.895939    4284 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Oct 31 00:28:05.912087 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.895993    4284 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-10-30 00:22:26 +0000 UTC" deadline="2028-04-16 20:10:46.958122562 +0000 UTC"
Oct 31 00:28:05.912087 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.896035    4284 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12811h42m41.062089884s"
Oct 31 00:28:05.912661 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.896322    4284 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Oct 31 00:28:05.912661 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.897612    4284 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Oct 31 00:28:05.912661 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.899501    4284 log.go:25] "Validated CRI v1 runtime API"
Oct 31 00:28:05.912661 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.901185    4284 log.go:25] "Validated CRI v1 image API"
Oct 31 00:28:05.912661 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.901950    4284 server.go:1424] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Oct 31 00:28:05.912661 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.904594    4284 fs.go:135] Filesystem UUIDs: map[08366a60-f410-4403-861b-20a69f500d7d:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 b31872fd-a5d9-412e-b841-083f2dc08025:/dev/nvme0n1p3]
Oct 31 00:28:05.912827 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.904611    4284 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:24 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:26 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/00db36ffba9650250554710ed1cdb2e16a0a488cc757cc63aaee54fabdc19df2/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/00db36ffba9650250554710ed1cdb2e16a0a488cc757cc63aaee54fabdc19df2/userdata/shm major:0 minor:201 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/1188e094eabb027d1abb7b9851edccf1d8c7f701d4ac9866b902992f255fdb47/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/1188e094eabb027d1abb7b9851edccf1d8c7f701d4ac9866b902992f255fdb47/userdata/shm major:0 minor:76 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/320bed655f31eb0479e8e1af7d7a9978e83b9bb1f2756fa39f548f8954a6c756/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/320bed655f31eb0479e8e1af7d7a9978e83b9bb1f2756fa39f548f8954a6c756/userdata/shm major:0 minor:47 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/6044eeab0fcad7ea4b7fc0c1977044cd6744c491b566b1cd496034659c3f2490/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/6044eeab0fcad7ea4b7fc0c1977044cd6744c491b566b1cd496034659c3f2490/userdata/shm major:0 minor:77 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/8f0c1b598b3f2bc9b83fd8a0d75ad2e544d28447ea8afcd04398bee142bd99db/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/8f0c1b598b3f2bc9b83fd8a0d75ad2e544d28447ea8afcd04398bee142bd99db/userdata/shm major:0 minor:94 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/959d55d527ec7a6c7178ba7b1926ee7fb817f7975b3031a70bd9a576e2e3b1ce/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/959d55d527ec7a6c7178ba7b1926ee7fb817f7975b3031a70bd9a576e2e3b1ce/userdata/shm major:0 minor:51 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/9c4c014bdeff186fadbbba83b7498c289b19af588e40e2bf964ae400fb92f5eb/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/9c4c014bdeff186fadbbba83b7498c289b19af588e40e2bf964ae400fb92f5eb/userdata/shm major:0 minor:70 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/a17b6e11450f47a75f14e2684c8652ce4cc819ae80934ec6121b0c93e6af2407/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/a17b6e11450f47a75f14e2684c8652ce4cc819ae80934ec6121b0c93e6af2407/userdata/shm major:0 minor:68 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/ba701dec65a813195d815f7857a177f17a6126298c996127a489ccb3c59f2870/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/ba701dec65a813195d815f7857a177f17a6126298c996127a489ccb3c59f2870/userdata/shm major:0 minor:203 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/cf5491ca921b7277d410f4b081d5adf249e326fbdb3c5f39211ac00a72bfd7fb/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/cf5491ca921b7277d410f4b081d5adf249e326fbdb3c5f39211ac00a72bfd7fb/userdata/shm major:0 minor:98 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/e0c303a73e7bc56cbce47921d95d76eeabf37b5c2954b84d3cf1cb7f2d06787d/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/e0c303a73e7bc56cbce47921d95d76eeabf37b5c2954b84d3cf1cb7f2d06787d/userdata/shm major:0 minor:83 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/eb65fa9a666c91c9ac2ffe7057788f9538e79d38857116a648ae44c11f219605/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/eb65fa9a666c91c9ac2ffe7057788f9538e79d38857116a648ae44c11f219605/userdata/shm major:0 minor:88 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:32 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/1a72f876-1157-4447-845c-923cdfd5aa5e/volumes/kubernetes.io~projected/kube-api-access-xhp7k:{mountpoint:/var/lib/kubelet/pods/1a72f876-1157-4447-845c-923cdfd5aa5e/volumes/kubernetes.io~projected/kube-api-access-xhp7k major:0 minor:62 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/21df46e4-2158-4385-ab9a-4b1790b64b55/volumes/kubernetes.io~empty-dir/etc-tuned:{mountpoint:/var/lib/kubelet/pods/21df46e4-2158-4385-ab9a-4b1790b64b55/volumes/kubernetes.io~empty-dir/etc-tuned major:0 minor:57 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/21df46e4-2158-4385-ab9a-4b1790b64b55/volumes/kubernetes.io~empty-dir/tmp:{mountpoint:/var/lib/kubelet/pods/21df46e4-2158-4385-ab9a-4b1790b64b55/volumes/kubernetes.io~empty-dir/tmp major:0 minor:56 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/21df46e4-2158-4385-ab9a-4b1790b64b55/volumes/kubernetes.io~projected/kube-api-access-gm472:{mountpoint:/var/lib/kubelet/pods/21df46e4-2158-4385-ab9a-4b1790b64b55/volumes/kubernetes.io~projected/kube-api-access-gm472 major:0 minor:60 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/2c9341ef-0860-4fb6-8a86-272ac063756a/volumes/kubernetes.io~projected/kube-api-access-x6bgk:{mountpoint:/var/lib/kubelet/pods/2c9341ef-0860-4fb6-8a86-272ac063756a/volumes/kubernetes.io~projected/kube-api-access-x6bgk major:0 minor:59 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/41c4cc4b-b387-4c70-8a27-58255ba632b9/volumes/kubernetes.io~projected/kube-api-access-vrldg:{mountpoint:/var/lib/kubelet/pods/41c4cc4b-b387-4c70-8a27-58255ba632b9/volumes/kubernetes.io~projected/kube-api-access-vrldg major:0 minor:64 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/61c60450-eb50-4181-8ab7-e69e864ffb51/volumes/kubernetes.io~projected/kube-api-access-sslf9:{mountpoint:/var/lib/kubelet/pods/61c60450-eb50-4181-8ab7-e69e864ffb51/volumes/kubernetes.io~projected/kube-api-access-sslf9 major:0 minor:65 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/63bedd38-6ab7-493d-b94c-b3c288af3f48/volume-subpaths/run-systemd/ovnkube-controller/6:{mountpoint:/var/lib/kubelet/pods/63bedd38-6ab7-493d-b94c-b3c288af3f48/volume-subpaths/run-systemd/ovnkube-controller/6 major:0 minor:26 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/63bedd38-6ab7-493d-b94c-b3c288af3f48/volumes/kubernetes.io~projected/kube-api-access-vp9kn:{mountpoint:/var/lib/kubelet/pods/63bedd38-6ab7-493d-b94c-b3c288af3f48/volumes/kubernetes.io~projected/kube-api-access-vp9kn major:0 minor:63 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/63bedd38-6ab7-493d-b94c-b3c288af3f48/volumes/kubernetes.io~secret/ovn-node-metrics-cert:{mountpoint:/var/lib/kubelet/pods/63bedd38-6ab7-493d-b94c-b3c288af3f48/volumes/kubernetes.io~secret/ovn-node-metrics-cert major:0 minor:55 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/6926e9b8-7c44-413e-bc7e-63cfff593066/volumes/kubernetes.io~projected/kube-api-access-m2phw:{mountpoint:/var/lib/kubelet/pods/6926e9b8-7c44-413e-bc7e-63cfff593066/volumes/kubernetes.io~projected/kube-api-access-m2phw major:0 minor:66 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/85e4b969-79f8-4954-8539-0905bd5db99d/volumes/kubernetes.io~projected/kube-api-access-kplpm:{mountpoint:/var/lib/kubelet/pods/85e4b969-79f8-4954-8539-0905bd5db99d/volumes/kubernetes.io~projected/kube-api-access-kplpm major:0 minor:67 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/97410648-b7ae-4cf4-accf-3164c398b525/volumes/kubernetes.io~secret/original-pull-secret:{mountpoint:/var/lib/kubelet/pods/97410648-b7ae-4cf4-accf-3164c398b525/volumes/kubernetes.io~secret/original-pull-secret major:0 minor:45 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/9daae1e9-e437-437c-ba8b-3d9c206b64a3/volumes/kubernetes.io~projected/kube-api-access-4xxhd:{mountpoint:/var/lib/kubelet/pods/9daae1e9-e437-437c-ba8b-3d9c206b64a3/volumes/kubernetes.io~projected/kube-api-access-4xxhd major:0 minor:61 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/c4ce4bad-af31-4f2e-a684-b2123e9da265/volumes/kubernetes.io~projected/kube-api-access-hxhdf:{mountpoint:/var/lib/kubelet/pods/c4ce4bad-af31-4f2e-a684-b2123e9da265/volumes/kubernetes.io~projected/kube-api-access-hxhdf major:0 minor:197 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/f93a119c-fb17-44bd-973c-699a3b87523b/volumes/kubernetes.io~secret/agent-certs:{mountpoint:/var/lib/kubelet/pods/f93a119c-fb17-44bd-973c-699a3b87523b/volumes/kubernetes.io~secret/agent-certs major:0 minor:58 fsType:tmpfs blockSize:0} composefs_0-34:{mountpoint:/ major:0 minor:34 fsType:overlay blockSize:0} overlay_0-100:{mountpoint:/var/lib/containers/storage/overlay/121e99b2501abf38face52df33b6e7380994fade5a8820acce4de4447da70aa4/merged major:0 minor:100 fsType:overlay blockSize:0} overlay_0-102:{mountpoint:/var/lib/containers/storage/overlay/7c047af1ee53feeccab8f4bd50fea6bd952faf25aaafd8db8cab963e6571c10c/merged major:0 minor:102 fsType:overlay blockSize:0} overlay_0-105:{mountpoint:/var/lib/containers/storage/overlay/c4652c636bcfee1445e0b89b0c34fed57df6d0dd8cc3409673b82c2278cb2bea/merged major:0 minor:105 fsType:overlay blockSize:0} overlay_0-106:{mountpoint:/var/lib/containers/storage/overlay/5f8a88975d6784ee7d783737cfd79d1f79631bdb254fdae02877b6137357c857/merged major:0 minor:106 fsType:overlay blockSize:0} overlay_0-111:{mountpoint:/var/lib/containers/storage/overlay/a68940cb8045d093bb1dfc68c59c566aab8fec2506b3d24fad418cf8e671fc21/merged major:0 minor:111 fsType:overlay blockSize:0} overlay_0-119:{mountpoint:/var/lib/containers/storage/overlay/b47f2c52b9639b6859795f6d1a7e130cc349409515b13ec30260a5ae636acec9/merged major:0 minor:119 fsType:overlay blockSize:0} overlay_0-124:{mountpoint:/var/lib/containers/storage/overlay/703919f8e989df416a146b464bf07a25b27916e736b48be2ab661bbce4c0cb84/merged major:0 minor:124 fsType:overlay blockSize:0} overlay_0-127:{mountpoint:/var/lib/containers/storage/overlay/0138cf1d1668f01d3880a957b060fb6806c1299e3c3f68626eb5af42e9e9d8ba/merged major:0 minor:127 fsType:overlay blockSize:0} overlay_0-132:{mountpoint:/var/lib/containers/storage/overlay/9777baf56d1d26d08323d2f177f75f17fe2f8a71157df6353144ec74fc670a86/merged major:0 minor:132 fsType:overlay blockSize:0} overlay_0-134:{mountpoint:/var/lib/containers/storage/overlay/891c81ecb355dd6cf26eee9679e967508eb742a2693690cd0eddd7db5833f440/merged major:0 minor:134 fsType:overlay blockSize:0} overlay_0-144:{mountpoint:/var/lib/containers/storage/overlay/84b88d7ed07a63f14778ddd1c689be37c4a8be7df556dddc3484c2d888b49bbc/merged major:0 minor:144 fsType:overlay blockSize:0} overlay_0-147:{mountpoint:/var/lib/containers/storage/overlay/e30e47664e9cbfcc222ae94c4a84beb2c339966c62134f3ca509f5b61dda0a15/merged major:0 minor:147 fsType:overlay blockSize:0} overlay_0-152:{mountpoint:/var/lib/containers/storage/overlay/de65f40b1c97a29e5bb9b0b86ff28519e5128cd07e6c300b1bbde3457a3b0133/merged major:0 minor:152 fsType:overlay blockSize:0} overlay_0-157:{mountpoint:/var/lib/containers/storage/overlay/27f8c91ac3851f9239a5d6f5092ae5218d2f7344ed0bc057ba590003453a0a60/merged major:0 minor:157 fsType:overlay blockSize:0} overlay_0-162:{mountpoint:/var/lib/containers/storage/overlay/fc1b9cf08844c12f45cbb4b0366bf71c1059500a259280c1827ff6da2b916f07/merged major:0 minor:162 fsType:overlay blockSize:0} overlay_0-167:{mountpoint:/var/lib/containers/storage/overlay/5c2d58b7f7f0feaa4e09acb2d78db34e2cbcc04c703dac6064808ee589b7192c/merged major:0 minor:167 fsType:overlay blockSize:0} overlay_0-172:{mountpoint:/var/lib/containers/storage/overlay/b157ad4a3d62d0696fce04cb140f5133cb6e65afdba4f61994aeabdac2ad6193/merged major:0 minor:172 fsType:overlay blockSize:0} overlay_0-177:{mountpoint:/var/lib/containers/storage/overlay/826c3156a98834aff97e2823ee6b23371dfbc592f7680e44a5e7f1b52cbbbc92/merged major:0 minor:177 fsType:overlay blockSize:0} overlay_0-182:{mountpoint:/var/lib/containers/storage/overlay/1a1e08c0c50f132891b52e74c5db64a6a66c336f0af3abfb780fe9b0a3334d06/merged major:0 minor:182 fsType:overlay blockSize:0} overlay_0-187:{mountpoint:/var/lib/containers/storage/overlay/fcbcb99b2866b3b64e12d863497b65d373670a9d3ca562646f4db189d9c20c68/merged major:0 minor:187 fsType:overlay blockSize:0} overlay_0-192:{mountpoint:/var/lib/containers/storage/overlay/dfc27478dfdf5530d24a36a50739de0f6a71225b44d65e21a5e546e22d0cd950/merged major:0 minor:192 fsType:overlay blockSize:0} overlay_0-205:{mountpoint:/var/lib/containers/storage/overlay/e1eefc71982d9544388717707db91f463af7d64467f3397cbc0ddfa44560c102/merged major:0 minor:205 fsType:overlay blockSize:0} overlay_0-207:{mountpoint:/var/lib/containers/storage/overlay/74147208167f804ff55267ac363ec89543de34d9fe3bc8856f21e8af4adeafbe/merged major:0 minor:207 fsType:overlay blockSize:0} overlay_0-209:{mountpoint:/var/lib/containers/storage/overlay/c1d79942d3f33504b540c3d20b405d81c728ccd0a3f46eb70ef31a7e865dc865/merged major:0 minor:209 fsType:overlay blockSize:0} overlay_0-211:{mountpoint:/var/lib/containers/storage/overlay/9d48a7135617b3b76a4051be0c72a26727fe680569cca8ac14a63d31de3816bb/merged major:0 minor:211 fsType:overlay blockSize:0} overlay_0-49:{mountpoint:/var/lib/containers/storage/overlay/bbe1cf1fb10fc7c7161f15fc80bfcb437986c8b511027637a6550167c63471c8/merged major:0 minor:49 fsType:overlay blockSize:0} overlay_0-53:{mountpoint:/var/lib/containers/storage/overlay/fb563082935d4a45f69a99fa04dc02d2cb30c04b6ba109717b0380de48dba1e6/merged major:0 minor:53 fsType:overlay blockSize:0} overlay_0-71:{mountpoint:/var/lib/containers/storage/overlay/556da675dc4e37f2251974cea5e5302f575e6984aba9399fa2a2da4a6847d0a2/merged major:0 minor:71 fsType:overlay blockSize:0} overlay_0-74:{mountpoint:/var/lib/containers/storage/overlay/0f309294f99d5f47869478d13ae98ae1243d1074121de8e0379d20d5be14ebba/merged major:0 minor:74 fsType:overlay blockSize:0} overlay_0-80:{mountpoint:/var/lib/containers/storage/overlay/9eab2a829838560ec65212df6cb4e0067cbe2c88f2c2133373ba4fe0950d4b53/merged major:0 minor:80 fsType:overlay blockSize:0} overlay_0-82:{mountpoint:/var/lib/containers/storage/overlay/5ca443db00b26694aebabe19fffe1846f8f4ea05827902b671ea254affaa59bc/merged major:0 minor:82 fsType:overlay blockSize:0} overlay_0-86:{mountpoint:/var/lib/containers/storage/overlay/0cd444eede985c9b41fb8a00779aff7ab2eb6f518bc3d0219b4f9c028774c4fc/merged major:0 minor:86 fsType:overlay blockSize:0} overlay_0-90:{mountpoint:/var/lib/containers/storage/overlay/c462f489088f6325177c50bbdd6dc6e3739266a51a28b05f5cc776b54120cf81/merged major:0 minor:90 fsType:overlay blockSize:0} overlay_0-92:{mountpoint:/var/lib/containers/storage/overlay/d0f88e81d5d3814a2a9cb5c6f78dcd62b1a0924c0590d7765df31a19edf4211d/merged major:0 minor:92 fsType:overlay blockSize:0} overlay_0-96:{mountpoint:/var/lib/containers/storage/overlay/2a2da977aced3d25ccf40a018a6a05672c43fe5016aa96b772178efc101ec06a/merged major:0 minor:96 fsType:overlay blockSize:0}]
Oct 31 00:28:05.912827 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.910230    4284 manager.go:217] Machine: {Timestamp:2026-10-31 00:28:05.909777708 +0000 UTC m=+0.054784614 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3126128 MemoryCapacity:32812019712 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2bd4a8a49e2a5ad51eb8832cab50c2 SystemUUID:ec2bd4a8-a49e-2a5a-d51e-b8832cab50c2 BootID:251962ca-cb1d-4df9-9f05-f29568131ac4 Filesystems:[{Device:overlay_0-49 DeviceMajor:0 DeviceMinor:49 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/21df46e4-2158-4385-ab9a-4b1790b64b55/volumes/kubernetes.io~empty-dir/tmp DeviceMajor:0 DeviceMinor:56 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/9daae1e9-e437-437c-ba8b-3d9c206b64a3/volumes/kubernetes.io~projected/kube-api-access-4xxhd DeviceMajor:0 DeviceMinor:61 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/eb65fa9a666c91c9ac2ffe7057788f9538e79d38857116a648ae44c11f219605/userdata/shm DeviceMajor:0 DeviceMinor:88 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/8f0c1b598b3f2bc9b83fd8a0d75ad2e544d28447ea8afcd04398bee142bd99db/userdata/shm DeviceMajor:0 DeviceMinor:94 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-147 DeviceMajor:0 DeviceMinor:147 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-167 DeviceMajor:0 DeviceMinor:167 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/63bedd38-6ab7-493d-b94c-b3c288af3f48/volumes/kubernetes.io~secret/ovn-node-metrics-cert DeviceMajor:0 DeviceMinor:55 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-71 DeviceMajor:0 DeviceMinor:71 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-144 DeviceMajor:0 DeviceMinor:144 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:32 Capacity:16406011904 Type:vfs Inodes:1048576 HasInodes:true} {Device:/run/containers/storage/overlay-containers/959d55d527ec7a6c7178ba7b1926ee7fb817f7975b3031a70bd9a576e2e3b1ce/userdata/shm DeviceMajor:0 DeviceMinor:51 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-96 DeviceMajor:0 DeviceMinor:96 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-100 DeviceMajor:0 DeviceMinor:100 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-207 DeviceMajor:0 DeviceMinor:207 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-34 DeviceMajor:0 DeviceMinor:34 Capacity:6422528 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/var/lib/kubelet/pods/2c9341ef-0860-4fb6-8a86-272ac063756a/volumes/kubernetes.io~projected/kube-api-access-x6bgk DeviceMajor:0 DeviceMinor:59 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-127 DeviceMajor:0 DeviceMinor:127 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-132 DeviceMajor:0 DeviceMinor:132 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-162 DeviceMajor:0 DeviceMinor:162 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-177 DeviceMajor:0 DeviceMinor:177 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/c4ce4bad-af31-4f2e-a684-b2123e9da265/volumes/kubernetes.io~projected/kube-api-access-hxhdf DeviceMajor:0 DeviceMinor:197 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-211 DeviceMajor:0 DeviceMinor:211 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:overlay_0-111 DeviceMajor:0 DeviceMinor:111 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-205 DeviceMajor:0 DeviceMinor:205 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/21df46e4-2158-4385-ab9a-4b1790b64b55/volumes/kubernetes.io~projected/kube-api-access-gm472 DeviceMajor:0 DeviceMinor:60 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/1a72f876-1157-4447-845c-923cdfd5aa5e/volumes/kubernetes.io~projected/kube-api-access-xhp7k DeviceMajor:0 DeviceMinor:62 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-172 DeviceMajor:0 DeviceMinor:172 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:24 Capacity:16406007808 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/320bed655f31eb0479e8e1af7d7a9978e83b9bb1f2756fa39f548f8954a6c756/userdata/shm DeviceMajor:0 DeviceMinor:47 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-74 DeviceMajor:0 DeviceMinor:74 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-90 DeviceMajor:0 DeviceMinor:90 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-192 DeviceMajor:0 DeviceMinor:192 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/6044eeab0fcad7ea4b7fc0c1977044cd6744c491b566b1cd496034659c3f2490/userdata/shm DeviceMajor:0 DeviceMinor:77 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/e0c303a73e7bc56cbce47921d95d76eeabf37b5c2954b84d3cf1cb7f2d06787d/userdata/shm DeviceMajor:0 DeviceMinor:83 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-187 DeviceMajor:0 DeviceMinor:187 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/ba701dec65a813195d815f7857a177f17a6126298c996127a489ccb3c59f2870/userdata/shm DeviceMajor:0 DeviceMinor:203 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:26 Capacity:6562406400 Type:vfs Inodes:819200 HasInodes:true} {Device:/var/lib/kubelet/pods/21df46e4-2158-4385-ab9a-4b1790b64b55/volumes/kubernetes.io~empty-dir/etc-tuned DeviceMajor:0 DeviceMinor:57 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-80 DeviceMajor:0 DeviceMinor:80 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-82 DeviceMajor:0 DeviceMinor:82 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-86 DeviceMajor:0 DeviceMinor:86 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-92 DeviceMajor:0 DeviceMinor:92 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-105 DeviceMajor:0 DeviceMinor:105 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-182 DeviceMajor:0 DeviceMinor:182 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-102 DeviceMajor:0 DeviceMinor:102 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-124 DeviceMajor:0 DeviceMinor:124 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/61c60450-eb50-4181-8ab7-e69e864ffb51/volumes/kubernetes.io~projected/kube-api-access-sslf9 DeviceMajor:0 DeviceMinor:65 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/85e4b969-79f8-4954-8539-0905bd5db99d/volumes/kubernetes.io~projected/kube-api-access-kplpm DeviceMajor:0 DeviceMinor:67 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-119 DeviceMajor:0 DeviceMinor:119 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-53 DeviceMajor:0 DeviceMinor:53 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/f93a119c-fb17-44bd-973c-699a3b87523b/volumes/kubernetes.io~secret/agent-certs DeviceMajor:0 DeviceMinor:58 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/41c4cc4b-b387-4c70-8a27-58255ba632b9/volumes/kubernetes.io~projected/kube-api-access-vrldg DeviceMajor:0 DeviceMinor:64 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/a17b6e11450f47a75f14e2684c8652ce4cc819ae80934ec6121b0c93e6af2407/userdata/shm DeviceMajor:0 DeviceMinor:68 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/cf5491ca921b7277d410f4b081d5adf249e326fbdb3c5f39211ac00a72bfd7fb/userdata/shm DeviceMajor:0 DeviceMinor:98 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-152 DeviceMajor:0 DeviceMinor:152 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/63bedd38-6ab7-493d-b94c-b3c288af3f48/volumes/kubernetes.io~projected/kube-api-access-vp9kn DeviceMajor:0 DeviceMinor:63 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/6926e9b8-7c44-413e-bc7e-63cfff593066/volumes/kubernetes.io~projected/kube-api-access-m2phw DeviceMajor:0 DeviceMinor:66 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-106 DeviceMajor:0 DeviceMinor:106 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-157 DeviceMajor:0 DeviceMinor:157 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/1188e094eabb027d1abb7b9851edccf1d8c7f701d4ac9866b902992f255fdb47/userdata/shm DeviceMajor:0 DeviceMinor:76 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/63bedd38-6ab7-493d-b94c-b3c288af3f48/volume-subpaths/run-systemd/ovnkube-controller/6 DeviceMajor:0 DeviceMinor:26 Capacity:6562406400 Type:vfs Inodes:819200 HasInodes:true} {Device:/run/containers/storage/overlay-containers/00db36ffba9650250554710ed1cdb2e16a0a488cc757cc63aaee54fabdc19df2/userdata/shm DeviceMajor:0 DeviceMinor:201 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/9c4c014bdeff186fadbbba83b7498c289b19af588e40e2bf964ae400fb92f5eb/userdata/shm DeviceMajor:0 DeviceMinor:70 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-134 DeviceMajor:0 DeviceMinor:134 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/97410648-b7ae-4cf4-accf-3164c398b525/volumes/kubernetes.io~secret/original-pull-secret DeviceMajor:0 DeviceMinor:45 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-209 DeviceMajor:0 DeviceMinor:209 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:00db36ffba96502 MacAddress:ca:c4:2a:ff:5a:ac Speed:10000 Mtu:8901} {Name:ba701dec65a8131 MacAddress:66:90:97:ee:d7:79 Speed:10000 Mtu:8901} {Name:br-ex MacAddress:06:ff:cc:06:45:87 Speed:0 Mtu:9001} {Name:br-int MacAddress:8a:8e:a0:2f:b0:a7 Speed:0 Mtu:8901} {Name:ens5 MacAddress:06:ff:cc:06:45:87 Speed:0 Mtu:9001} {Name:genev_sys_6081 MacAddress:c6:93:cc:bd:9b:bf Speed:0 Mtu:65000} {Name:ovn-k8s-mp0 MacAddress:0a:58:0a:86:00:02 Speed:0 Mtu:8901} {Name:ovs-system MacAddress:3a:b4:36:ce:b8:78 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812019712 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Oct 31 00:28:05.912827 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.910437    4284 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Oct 31 00:28:05.912827 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.910480    4284 manager.go:233] Version: {KernelVersion:5.14.0-687.60.1.el9_8.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.8.20261023-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Oct 31 00:28:05.912827 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.910741    4284 container_manager_linux.go:275] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Oct 31 00:28:05.912827 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.910760    4284 container_manager_linux.go:280] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-132-173.us-west-2.compute.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"/system.slice","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{},"system-reserved-compressible":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Oct 31 00:28:05.912827 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.910891    4284 topology_manager.go:143] "Creating topology manager with none policy"
Oct 31 00:28:05.912827 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.910898    4284 container_manager_linux.go:311] "Creating device plugin manager"
Oct 31 00:28:05.912827 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.910909    4284 manager.go:144] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Oct 31 00:28:05.912827 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.910922    4284 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Oct 31 00:28:05.912827 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.910994    4284 container_manager_linux.go:320] "Creating Dynamic Resource Allocation (DRA) manager"
Oct 31 00:28:05.912827 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.911161    4284 state_mem.go:41] "Initialized" logger="CPUManager state memory"
Oct 31 00:28:05.912827 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.911285    4284 server.go:1235] "Using root directory" path="/var/lib/kubelet"
Oct 31 00:28:05.912827 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.911320    4284 kubelet.go:493] "Attempting to sync node with API server"
Oct 31 00:28:05.912827 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.911332    4284 kubelet.go:394] "Adding static pod path" path="/etc/kubernetes/manifests"
Oct 31 00:28:05.913450 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.911342    4284 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Oct 31 00:28:05.913450 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.911351    4284 kubelet.go:405] "Adding apiserver pod source"
Oct 31 00:28:05.913450 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.911359    4284 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Oct 31 00:28:05.913450 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.911954    4284 state_checkpoint.go:81] "State checkpoint: restored pod resource state from checkpoint"
Oct 31 00:28:05.913450 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.911968    4284 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Oct 31 00:28:05.913450 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.912524    4284 kuberuntime_manager.go:304] "Container runtime initialized" containerRuntime="cri-o" version="1.35.10-3.rhaos4.22.git9383502.el9" apiVersion="v1"
Oct 31 00:28:05.915453 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.915431    4284 state_checkpoint.go:81] "State checkpoint: restored pod resource state from checkpoint"
Oct 31 00:28:05.915514 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.915456    4284 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Oct 31 00:28:05.915616 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.915603    4284 certificate_store.go:147] "Loading cert/key pair from a file" filePath="/var/lib/kubelet/pki/kubelet-server-current.pem"
Oct 31 00:28:05.915935 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.915921    4284 kubelet.go:963] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Oct 31 00:28:05.915978 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.915968    4284 kubelet.go:990] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Oct 31 00:28:05.916133 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.916119    4284 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Oct 31 00:28:05.916164 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.916146    4284 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Oct 31 00:28:05.916164 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.916161    4284 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Oct 31 00:28:05.916225 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.916169    4284 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Oct 31 00:28:05.916225 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.916178    4284 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Oct 31 00:28:05.916225 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.916186    4284 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Oct 31 00:28:05.916225 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.916196    4284 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Oct 31 00:28:05.916376 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.916205    4284 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Oct 31 00:28:05.916376 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.916237    4284 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Oct 31 00:28:05.916376 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.916247    4284 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Oct 31 00:28:05.916376 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.916268    4284 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Oct 31 00:28:05.916376 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.916305    4284 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Oct 31 00:28:05.916376 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.916354    4284 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Oct 31 00:28:05.916376 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.916363    4284 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Oct 31 00:28:05.919606 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.919594    4284 server.go:1263] "Started kubelet"
Oct 31 00:28:05.920014 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.919676    4284 server.go:182] "Starting to listen" address="0.0.0.0" port=10250
Oct 31 00:28:05.920014 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.919708    4284 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Oct 31 00:28:05.920014 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.919796    4284 server_v1.go:49] "podresources" method="list" useActivePods=true
Oct 31 00:28:05.920113 ip-10-0-132-173 systemd[1]: Started Kubernetes Kubelet.
Oct 31 00:28:05.921364 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.921338    4284 server.go:254] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Oct 31 00:28:05.921590 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.921570    4284 server.go:317] "Adding debug handlers to kubelet server"
Oct 31 00:28:05.925672 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.923539    4284 fs_resource_analyzer.go:69] "Starting FS ResourceAnalyzer"
Oct 31 00:28:05.925672 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.924076    4284 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Oct 31 00:28:05.925672 ip-10-0-132-173 kubenswrapper[4284]: E1031 00:28:05.924368    4284 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-132-173.us-west-2.compute.internal\" not found"
Oct 31 00:28:05.925672 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.924413    4284 volume_manager.go:309] "The desired_state_of_world populator starts"
Oct 31 00:28:05.925672 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.924430    4284 volume_manager.go:311] "Starting Kubelet Volume Manager"
Oct 31 00:28:05.925672 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.924448    4284 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-10-30 00:22:28 +0000 UTC" deadline="2028-07-01 03:29:45.233185884 +0000 UTC"
Oct 31 00:28:05.925672 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.924486    4284 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14619h1m39.308703566s"
Oct 31 00:28:05.925672 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.924518    4284 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Oct 31 00:28:05.925672 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.925196    4284 factory.go:55] Registering systemd factory
Oct 31 00:28:05.925672 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.925249    4284 factory.go:223] Registration of the systemd container factory successfully
Oct 31 00:28:05.925672 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.925581    4284 factory.go:153] Registering CRI-O factory
Oct 31 00:28:05.925672 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.925625    4284 factory.go:223] Registration of the crio container factory successfully
Oct 31 00:28:05.926291 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.925724    4284 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Oct 31 00:28:05.926291 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.925762    4284 factory.go:103] Registering Raw factory
Oct 31 00:28:05.926291 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.925788    4284 manager.go:1199] Started watching for new ooms in manager
Oct 31 00:28:05.927117 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.927083    4284 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="6926e9b8-7c44-413e-bc7e-63cfff593066" volumeName="kubernetes.io/projected/6926e9b8-7c44-413e-bc7e-63cfff593066-kube-api-access-m2phw" seLinuxMountContext=""
Oct 31 00:28:05.927117 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.927118    4284 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="97410648-b7ae-4cf4-accf-3164c398b525" volumeName="kubernetes.io/secret/97410648-b7ae-4cf4-accf-3164c398b525-original-pull-secret" seLinuxMountContext=""
Oct 31 00:28:05.927305 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.927128    4284 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="9daae1e9-e437-437c-ba8b-3d9c206b64a3" volumeName="kubernetes.io/projected/9daae1e9-e437-437c-ba8b-3d9c206b64a3-kube-api-access-4xxhd" seLinuxMountContext=""
Oct 31 00:28:05.927305 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.927136    4284 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="63bedd38-6ab7-493d-b94c-b3c288af3f48" volumeName="kubernetes.io/configmap/63bedd38-6ab7-493d-b94c-b3c288af3f48-ovnkube-script-lib" seLinuxMountContext=""
Oct 31 00:28:05.927305 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.927148    4284 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="21df46e4-2158-4385-ab9a-4b1790b64b55" volumeName="kubernetes.io/empty-dir/21df46e4-2158-4385-ab9a-4b1790b64b55-etc-tuned" seLinuxMountContext=""
Oct 31 00:28:05.927305 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.927161    4284 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="41c4cc4b-b387-4c70-8a27-58255ba632b9" volumeName="kubernetes.io/configmap/41c4cc4b-b387-4c70-8a27-58255ba632b9-cni-binary-copy" seLinuxMountContext=""
Oct 31 00:28:05.927305 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.927172    4284 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="41c4cc4b-b387-4c70-8a27-58255ba632b9" volumeName="kubernetes.io/projected/41c4cc4b-b387-4c70-8a27-58255ba632b9-kube-api-access-vrldg" seLinuxMountContext=""
Oct 31 00:28:05.927305 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.927182    4284 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="61c60450-eb50-4181-8ab7-e69e864ffb51" volumeName="kubernetes.io/configmap/61c60450-eb50-4181-8ab7-e69e864ffb51-serviceca" seLinuxMountContext=""
Oct 31 00:28:05.927305 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.927204    4284 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="85e4b969-79f8-4954-8539-0905bd5db99d" volumeName="kubernetes.io/projected/85e4b969-79f8-4954-8539-0905bd5db99d-kube-api-access-kplpm" seLinuxMountContext=""
Oct 31 00:28:05.927305 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.927272    4284 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="f93a119c-fb17-44bd-973c-699a3b87523b" volumeName="kubernetes.io/secret/f93a119c-fb17-44bd-973c-699a3b87523b-agent-certs" seLinuxMountContext=""
Oct 31 00:28:05.927305 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.927283    4284 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="21df46e4-2158-4385-ab9a-4b1790b64b55" volumeName="kubernetes.io/empty-dir/21df46e4-2158-4385-ab9a-4b1790b64b55-tmp" seLinuxMountContext=""
Oct 31 00:28:05.927305 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.927295    4284 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="63bedd38-6ab7-493d-b94c-b3c288af3f48" volumeName="kubernetes.io/configmap/63bedd38-6ab7-493d-b94c-b3c288af3f48-ovnkube-config" seLinuxMountContext=""
Oct 31 00:28:05.927819 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.927305    4284 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="6926e9b8-7c44-413e-bc7e-63cfff593066" volumeName="kubernetes.io/configmap/6926e9b8-7c44-413e-bc7e-63cfff593066-multus-daemon-config" seLinuxMountContext=""
Oct 31 00:28:05.927819 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.927338    4284 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="9daae1e9-e437-437c-ba8b-3d9c206b64a3" volumeName="kubernetes.io/configmap/9daae1e9-e437-437c-ba8b-3d9c206b64a3-iptables-alerter-script" seLinuxMountContext=""
Oct 31 00:28:05.927819 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.927355    4284 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="c4ce4bad-af31-4f2e-a684-b2123e9da265" volumeName="kubernetes.io/projected/c4ce4bad-af31-4f2e-a684-b2123e9da265-kube-api-access-hxhdf" seLinuxMountContext=""
Oct 31 00:28:05.927819 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.927366    4284 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="f93a119c-fb17-44bd-973c-699a3b87523b" volumeName="kubernetes.io/configmap/f93a119c-fb17-44bd-973c-699a3b87523b-konnectivity-ca" seLinuxMountContext=""
Oct 31 00:28:05.927819 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.927377    4284 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="61c60450-eb50-4181-8ab7-e69e864ffb51" volumeName="kubernetes.io/projected/61c60450-eb50-4181-8ab7-e69e864ffb51-kube-api-access-sslf9" seLinuxMountContext=""
Oct 31 00:28:05.927819 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.927387    4284 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1a72f876-1157-4447-845c-923cdfd5aa5e" volumeName="kubernetes.io/projected/1a72f876-1157-4447-845c-923cdfd5aa5e-kube-api-access-xhp7k" seLinuxMountContext=""
Oct 31 00:28:05.927819 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.927399    4284 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="21df46e4-2158-4385-ab9a-4b1790b64b55" volumeName="kubernetes.io/projected/21df46e4-2158-4385-ab9a-4b1790b64b55-kube-api-access-gm472" seLinuxMountContext=""
Oct 31 00:28:05.927819 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.927410    4284 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="2c9341ef-0860-4fb6-8a86-272ac063756a" volumeName="kubernetes.io/projected/2c9341ef-0860-4fb6-8a86-272ac063756a-kube-api-access-x6bgk" seLinuxMountContext=""
Oct 31 00:28:05.927819 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.927422    4284 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="41c4cc4b-b387-4c70-8a27-58255ba632b9" volumeName="kubernetes.io/configmap/41c4cc4b-b387-4c70-8a27-58255ba632b9-cni-sysctl-allowlist" seLinuxMountContext=""
Oct 31 00:28:05.927819 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.927433    4284 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="63bedd38-6ab7-493d-b94c-b3c288af3f48" volumeName="kubernetes.io/projected/63bedd38-6ab7-493d-b94c-b3c288af3f48-kube-api-access-vp9kn" seLinuxMountContext=""
Oct 31 00:28:05.927819 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.927450    4284 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="63bedd38-6ab7-493d-b94c-b3c288af3f48" volumeName="kubernetes.io/secret/63bedd38-6ab7-493d-b94c-b3c288af3f48-ovn-node-metrics-cert" seLinuxMountContext=""
Oct 31 00:28:05.927819 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.927461    4284 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="6926e9b8-7c44-413e-bc7e-63cfff593066" volumeName="kubernetes.io/configmap/6926e9b8-7c44-413e-bc7e-63cfff593066-cni-binary-copy" seLinuxMountContext=""
Oct 31 00:28:05.927819 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.927487    4284 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="2c9341ef-0860-4fb6-8a86-272ac063756a" volumeName="kubernetes.io/empty-dir/2c9341ef-0860-4fb6-8a86-272ac063756a-tmp-dir" seLinuxMountContext=""
Oct 31 00:28:05.927819 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.927492    4284 manager.go:319] Starting recovery of all containers
Oct 31 00:28:05.927819 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.927516    4284 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="41c4cc4b-b387-4c70-8a27-58255ba632b9" volumeName="kubernetes.io/configmap/41c4cc4b-b387-4c70-8a27-58255ba632b9-whereabouts-flatfile-configmap" seLinuxMountContext=""
Oct 31 00:28:05.927819 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.927529    4284 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="63bedd38-6ab7-493d-b94c-b3c288af3f48" volumeName="kubernetes.io/configmap/63bedd38-6ab7-493d-b94c-b3c288af3f48-env-overrides" seLinuxMountContext=""
Oct 31 00:28:05.927819 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.927541    4284 reconstruct.go:97] "Volume reconstruction finished"
Oct 31 00:28:05.927819 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.927548    4284 reconciler.go:29] "Reconciler: start to sync state"
Oct 31 00:28:05.953870 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.953822    4284 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Oct 31 00:28:05.955337 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.955314    4284 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Oct 31 00:28:05.955337 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.955336    4284 status_manager.go:249] "Starting to sync pod status with apiserver"
Oct 31 00:28:05.955523 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.955356    4284 kubelet.go:2521] "Starting kubelet main sync loop"
Oct 31 00:28:05.955523 ip-10-0-132-173 kubenswrapper[4284]: E1031 00:28:05.955393    4284 kubelet.go:2545] "Skipping pod synchronization" err="[container runtime status check may not have completed yet, PLEG is not healthy: pleg has yet to be successful]"
Oct 31 00:28:05.971568 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.971534    4284 generic.go:356] "Generic (PLEG): container finished" podID="41c4cc4b-b387-4c70-8a27-58255ba632b9" containerID="f9f499ecb0a7c7954769abbadf0e73f4107a33dbf973bd99b9718074d4fd14dc" exitCode=0
Oct 31 00:28:05.971568 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.971564    4284 generic.go:356] "Generic (PLEG): container finished" podID="41c4cc4b-b387-4c70-8a27-58255ba632b9" containerID="8f2df9b116a2a10b7f5c0df12cc0a0f287da91deb1c919cae2edbe479ba9c642" exitCode=0
Oct 31 00:28:05.971724 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.971574    4284 generic.go:356] "Generic (PLEG): container finished" podID="41c4cc4b-b387-4c70-8a27-58255ba632b9" containerID="6ac8ef00f2a5ff74e39d436adae7bf8734fd6dd81a2fb68129183b470058f957" exitCode=0
Oct 31 00:28:05.971724 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.971585    4284 generic.go:356] "Generic (PLEG): container finished" podID="41c4cc4b-b387-4c70-8a27-58255ba632b9" containerID="c67fca05dd53a48ed0934bfea1e1439af43446f05682b0c6b1c27cf7d265da07" exitCode=0
Oct 31 00:28:05.971724 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.971591    4284 generic.go:356] "Generic (PLEG): container finished" podID="41c4cc4b-b387-4c70-8a27-58255ba632b9" containerID="3e3f91b116ad245d40596238a421239406c46c83c681aee7049eafd3ed15156c" exitCode=0
Oct 31 00:28:05.971724 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.971597    4284 generic.go:356] "Generic (PLEG): container finished" podID="41c4cc4b-b387-4c70-8a27-58255ba632b9" containerID="508a002da3205d4f14eae542dc1b75634c31b15fce4118dd41d9dc7c8d8d2e06" exitCode=0
Oct 31 00:28:05.983399 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.983362    4284 generic.go:356] "Generic (PLEG): container finished" podID="6b005c94812deb7bd218c8f0cf290c09" containerID="5b2371580e711da02ae94dfbe659fb42391bd9cb84cc78e18838a4bc8c55ae17" exitCode=0
Oct 31 00:28:05.984270 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.984251    4284 manager.go:324] Recovery completed
Oct 31 00:28:05.995090 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.995076    4284 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 31 00:28:05.997361 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.997344    4284 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-173.us-west-2.compute.internal" event="NodeHasSufficientMemory"
Oct 31 00:28:05.997430 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.997378    4284 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-173.us-west-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 31 00:28:05.997430 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.997392    4284 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-173.us-west-2.compute.internal" event="NodeHasSufficientPID"
Oct 31 00:28:05.998554 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.998539    4284 cpu_manager.go:226] "Starting" policy="none"
Oct 31 00:28:05.998554 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.998550    4284 cpu_manager.go:227] "Reconciling" reconcilePeriod="10s"
Oct 31 00:28:05.998644 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.998565    4284 state_mem.go:41] "Initialized" logger="CPUManager state checkpoint.CPUManager state memory"
Oct 31 00:28:05.998644 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.998551    4284 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 31 00:28:05.998708 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.998670    4284 state_mem.go:94] "Updated default CPUSet" logger="CPUManager state checkpoint.CPUManager state memory" cpuSet=""
Oct 31 00:28:05.998708 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.998680    4284 state_mem.go:102] "Updated CPUSet assignments" logger="CPUManager state checkpoint.CPUManager state memory" assignments={}
Oct 31 00:28:05.998708 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.998695    4284 state_checkpoint.go:142] "restored state from checkpoint" logger="CPUManager state checkpoint"
Oct 31 00:28:05.998708 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.998700    4284 state_checkpoint.go:143] "defaultCPUSet" logger="CPUManager state checkpoint" defaultCpuSet=""
Oct 31 00:28:05.998708 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.998704    4284 policy_none.go:50] "Start"
Oct 31 00:28:05.998913 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.998712    4284 memory_manager.go:187] "Starting memorymanager" policy="None"
Oct 31 00:28:05.998913 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.998720    4284 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Oct 31 00:28:05.998913 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.998838    4284 state_mem.go:77] "Updated machine memory state" logger="Memory Manager state checkpoint"
Oct 31 00:28:05.998913 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.998845    4284 state_checkpoint.go:85] "State checkpoint: restored state from checkpoint" logger="Memory Manager state checkpoint"
Oct 31 00:28:05.998913 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.998851    4284 policy_none.go:44] "Start"
Oct 31 00:28:05.999623 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.999607    4284 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-173.us-west-2.compute.internal" event="NodeHasSufficientMemory"
Oct 31 00:28:05.999699 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.999639    4284 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-173.us-west-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 31 00:28:05.999699 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:05.999654    4284 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-173.us-west-2.compute.internal" event="NodeHasSufficientPID"
Oct 31 00:28:06.002490 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:06.002464    4284 node_container_manager_linux.go:130] "Enforcing system reserved compressible on cgroup" cgroupName="/system.slice" limits={"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"}
Oct 31 00:28:06.007090 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:06.007077    4284 manager.go:341] "Starting Device Plugin manager"
Oct 31 00:28:06.007243 ip-10-0-132-173 kubenswrapper[4284]: E1031 00:28:06.007104    4284 manager.go:525] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Oct 31 00:28:06.007243 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:06.007115    4284 server.go:87] "Starting device plugin registration server"
Oct 31 00:28:06.007421 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:06.007407    4284 eviction_manager.go:194] "Eviction manager: starting control loop"
Oct 31 00:28:06.007502 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:06.007426    4284 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Oct 31 00:28:06.007551 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:06.007509    4284 plugin_watcher.go:53] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Oct 31 00:28:06.007628 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:06.007590    4284 plugin_watcher.go:201] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Oct 31 00:28:06.007628 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:06.007632    4284 plugin_manager.go:119] "The desired_state_of_world populator (plugin watcher) starts"
Oct 31 00:28:06.007628 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:06.007638    4284 plugin_manager.go:121] "Starting Kubelet Plugin Manager"
Oct 31 00:28:06.007836 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:06.007678    4284 reconciler.go:167] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-10-31T00:28:06.007621664Z","UUID":"0e79a700-b726-4dca-a1b4-611fbf5f64eb","Handler":null,"Name":"","Endpoint":""}
Oct 31 00:28:06.008684 ip-10-0-132-173 kubenswrapper[4284]: E1031 00:28:06.008668    4284 eviction_manager.go:272] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Oct 31 00:28:06.008756 ip-10-0-132-173 kubenswrapper[4284]: E1031 00:28:06.008699    4284 eviction_manager.go:297] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-132-173.us-west-2.compute.internal\" not found"
Oct 31 00:28:06.009107 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:06.009090    4284 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Oct 31 00:28:06.009107 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:06.009115    4284 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Oct 31 00:28:06.056907 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:06.056406    4284 kubelet.go:2608] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-173.us-west-2.compute.internal","kube-system/kube-apiserver-proxy-ip-10-0-132-173.us-west-2.compute.internal"]
Oct 31 00:28:06.057042 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:06.056963    4284 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 31 00:28:06.057989 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:06.057972    4284 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-173.us-west-2.compute.internal" event="NodeHasSufficientMemory"
Oct 31 00:28:06.058089 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:06.058005    4284 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-173.us-west-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 31 00:28:06.058089 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:06.058019    4284 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-173.us-west-2.compute.internal" event="NodeHasSufficientPID"
Oct 31 00:28:06.058327 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:06.058313    4284 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 31 00:28:06.058465 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:06.058449    4284 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 31 00:28:06.059292 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:06.059274    4284 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-173.us-west-2.compute.internal" event="NodeHasSufficientMemory"
Oct 31 00:28:06.059371 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:06.059305    4284 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-173.us-west-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 31 00:28:06.059371 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:06.059316    4284 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-173.us-west-2.compute.internal" event="NodeHasSufficientPID"
Oct 31 00:28:06.059371 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:06.059280    4284 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-173.us-west-2.compute.internal" event="NodeHasSufficientMemory"
Oct 31 00:28:06.059473 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:06.059379    4284 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-173.us-west-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 31 00:28:06.059473 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:06.059396    4284 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-173.us-west-2.compute.internal" event="NodeHasSufficientPID"
Oct 31 00:28:06.059473 ip-10-0-132-173 kubenswrapper[4284]: E1031 00:28:06.059466    4284 kubelet.go:3364] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-132-173.us-west-2.compute.internal\" not found" node="ip-10-0-132-173.us-west-2.compute.internal"
Oct 31 00:28:06.059651 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:06.059624    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-173.us-west-2.compute.internal" event={"ID":"6c9ceace9ec30530fd2581e47cd10afe","Type":"ContainerStarted","Data":"3d715c11efa2b4718cdc3a5a05a77be833001e9aeba541a56f95fecb8aada745"}
Oct 31 00:28:06.059683 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:06.059659    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-173.us-west-2.compute.internal" event={"ID":"6c9ceace9ec30530fd2581e47cd10afe","Type":"ContainerStarted","Data":"959d55d527ec7a6c7178ba7b1926ee7fb817f7975b3031a70bd9a576e2e3b1ce"}
Oct 31 00:28:06.059683 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:06.059671    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-173.us-west-2.compute.internal" event={"ID":"6b005c94812deb7bd218c8f0cf290c09","Type":"ContainerStarted","Data":"43fa13eef47facac1d98a81cb1264617f9a2a72a6daaaec3cd2f9ea064206fc7"}
Oct 31 00:28:06.059744 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:06.059685    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-173.us-west-2.compute.internal" event={"ID":"6b005c94812deb7bd218c8f0cf290c09","Type":"ContainerDied","Data":"5b2371580e711da02ae94dfbe659fb42391bd9cb84cc78e18838a4bc8c55ae17"}
Oct 31 00:28:06.059744 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:06.059692    4284 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 31 00:28:06.059744 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:06.059697    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-173.us-west-2.compute.internal" event={"ID":"6b005c94812deb7bd218c8f0cf290c09","Type":"ContainerStarted","Data":"320bed655f31eb0479e8e1af7d7a9978e83b9bb1f2756fa39f548f8954a6c756"}
Oct 31 00:28:06.061086 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:06.061072    4284 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-173.us-west-2.compute.internal" event="NodeHasSufficientMemory"
Oct 31 00:28:06.061135 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:06.061099    4284 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-173.us-west-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 31 00:28:06.061135 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:06.061109    4284 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-173.us-west-2.compute.internal" event="NodeHasSufficientPID"
Oct 31 00:28:06.061284 ip-10-0-132-173 kubenswrapper[4284]: E1031 00:28:06.061272    4284 kubelet.go:3364] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-132-173.us-west-2.compute.internal\" not found" node="ip-10-0-132-173.us-west-2.compute.internal"
Oct 31 00:28:06.107611 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:06.107578    4284 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 31 00:28:06.108717 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:06.108697    4284 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-173.us-west-2.compute.internal" event="NodeHasSufficientMemory"
Oct 31 00:28:06.108826 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:06.108733    4284 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-173.us-west-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 31 00:28:06.108826 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:06.108748    4284 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-173.us-west-2.compute.internal" event="NodeHasSufficientPID"
Oct 31 00:28:06.108826 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:06.108771    4284 kubelet_node_status.go:77] "Attempting to register node" node="ip-10-0-132-173.us-west-2.compute.internal"
Oct 31 00:28:06.115986 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:06.115963    4284 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 31 00:28:06.116120 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:06.115973    4284 reconstruct.go:205] "DevicePaths of reconstructed volumes updated"
Oct 31 00:28:06.161917 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:06.161887    4284 reflector.go:446] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 31 00:28:06.162053 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:06.161980    4284 reflector.go:446] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 31 00:28:06.164518 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:06.164498    4284 reflector.go:446] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 31 00:28:06.216913 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:06.216888    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/6c9ceace9ec30530fd2581e47cd10afe-config\") pod \"kube-apiserver-proxy-ip-10-0-132-173.us-west-2.compute.internal\" (UID: \"6c9ceace9ec30530fd2581e47cd10afe\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-173.us-west-2.compute.internal"
Oct 31 00:28:06.216992 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:06.216943    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/6b005c94812deb7bd218c8f0cf290c09-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-173.us-west-2.compute.internal\" (UID: \"6b005c94812deb7bd218c8f0cf290c09\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-173.us-west-2.compute.internal"
Oct 31 00:28:06.216992 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:06.216965    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6b005c94812deb7bd218c8f0cf290c09-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-173.us-west-2.compute.internal\" (UID: \"6b005c94812deb7bd218c8f0cf290c09\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-173.us-west-2.compute.internal"
Oct 31 00:28:06.225039 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:06.225014    4284 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-173.us-west-2.compute.internal"
Oct 31 00:28:06.259650 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:06.259616    4284 nodeinfomanager.go:471] existing CSINode "ip-10-0-132-173.us-west-2.compute.internal" is owned by different node (oldNodeID="7345ec66-7a03-4ba4-b099-c29b95dac717", newNodeID=""), cleaning up...
Oct 31 00:28:06.259801 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:06.259768    4284 kubelet_node_status.go:126] "Node was previously registered" node="ip-10-0-132-173.us-west-2.compute.internal"
Oct 31 00:28:06.259844 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:06.259834    4284 kubelet_node_status.go:80] "Successfully registered node" node="ip-10-0-132-173.us-west-2.compute.internal"
Oct 31 00:28:06.260442 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:06.260428    4284 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-173.us-west-2.compute.internal" event="NodeNotReady"
Oct 31 00:28:06.260501 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:06.260457    4284 setters.go:546] "Node became not ready" node="ip-10-0-132-173.us-west-2.compute.internal" condition={"type":"Ready","status":"False","lastHeartbeatTime":"2026-10-31T00:28:06Z","lastTransitionTime":"2026-10-31T00:28:06Z","reason":"KubeletNotReady","message":"CSINode is not yet initialized"}
Oct 31 00:28:06.281619 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:06.281602    4284 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 31 00:28:06.281666 ip-10-0-132-173 kubenswrapper[4284]: E1031 00:28:06.281636    4284 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-apiserver-proxy-ip-10-0-132-173.us-west-2.compute.internal\" already exists" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-173.us-west-2.compute.internal"
Oct 31 00:28:06.281666 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:06.281648    4284 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-173.us-west-2.compute.internal"
Oct 31 00:28:06.317742 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:06.317683    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/6c9ceace9ec30530fd2581e47cd10afe-config\") pod \"kube-apiserver-proxy-ip-10-0-132-173.us-west-2.compute.internal\" (UID: \"6c9ceace9ec30530fd2581e47cd10afe\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-173.us-west-2.compute.internal"
Oct 31 00:28:06.317742 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:06.317715    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/6b005c94812deb7bd218c8f0cf290c09-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-173.us-west-2.compute.internal\" (UID: \"6b005c94812deb7bd218c8f0cf290c09\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-173.us-west-2.compute.internal"
Oct 31 00:28:06.317742 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:06.317732    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6b005c94812deb7bd218c8f0cf290c09-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-173.us-west-2.compute.internal\" (UID: \"6b005c94812deb7bd218c8f0cf290c09\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-173.us-west-2.compute.internal"
Oct 31 00:28:06.317891 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:06.317788    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/6b005c94812deb7bd218c8f0cf290c09-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-173.us-west-2.compute.internal\" (UID: \"6b005c94812deb7bd218c8f0cf290c09\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-173.us-west-2.compute.internal"
Oct 31 00:28:06.317891 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:06.317802    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/6c9ceace9ec30530fd2581e47cd10afe-config\") pod \"kube-apiserver-proxy-ip-10-0-132-173.us-west-2.compute.internal\" (UID: \"6c9ceace9ec30530fd2581e47cd10afe\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-173.us-west-2.compute.internal"
Oct 31 00:28:06.317891 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:06.317832    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6b005c94812deb7bd218c8f0cf290c09-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-173.us-west-2.compute.internal\" (UID: \"6b005c94812deb7bd218c8f0cf290c09\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-173.us-west-2.compute.internal"
Oct 31 00:28:06.385626 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:06.385595    4284 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 31 00:28:06.385782 ip-10-0-132-173 kubenswrapper[4284]: E1031 00:28:06.385642    4284 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-rbac-proxy-crio-ip-10-0-132-173.us-west-2.compute.internal\" already exists" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-173.us-west-2.compute.internal"
Oct 31 00:28:06.527681 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:06.527658    4284 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-173.us-west-2.compute.internal" event="NodeReady"
Oct 31 00:28:06.527837 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:06.527729    4284 kubelet_node_status.go:477] "Fast updating node status as it just became ready"
Oct 31 00:28:06.911897 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:06.911851    4284 apiserver.go:52] "Watching apiserver"
Oct 31 00:28:06.987527 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:06.987505    4284 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-173.us-west-2.compute.internal"
Oct 31 00:28:06.987698 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:06.987592    4284 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-173.us-west-2.compute.internal"
Oct 31 00:28:07.010649 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.010619    4284 reflector.go:446] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Oct 31 00:28:07.010960 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.010944    4284 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-multus/multus-d7msb","kube-system/global-pull-secret-syncer-24sxq","openshift-network-operator/iptables-alerter-p4txz","openshift-multus/multus-additional-cni-plugins-2t5fx","openshift-multus/network-metrics-daemon-plnkm","openshift-cluster-node-tuning-operator/tuned-hrsmn","openshift-dns/node-resolver-zjb6f","openshift-network-diagnostics/network-check-target-jktkx","openshift-ovn-kubernetes/ovnkube-node-rrk77","kube-system/konnectivity-agent-k2sk2","kube-system/kube-apiserver-proxy-ip-10-0-132-173.us-west-2.compute.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k6fgp","openshift-image-registry/node-ca-qchks","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-173.us-west-2.compute.internal"]
Oct 31 00:28:07.014886 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.014722    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-plnkm"
Oct 31 00:28:07.026228 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.026193    4284 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Oct 31 00:28:07.044535 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.044518    4284 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 31 00:28:07.044598 ip-10-0-132-173 kubenswrapper[4284]: E1031 00:28:07.044560    4284 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-apiserver-proxy-ip-10-0-132-173.us-west-2.compute.internal\" already exists" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-173.us-west-2.compute.internal"
Oct 31 00:28:07.045190 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.045179    4284 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 31 00:28:07.045242 ip-10-0-132-173 kubenswrapper[4284]: E1031 00:28:07.045204    4284 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-rbac-proxy-crio-ip-10-0-132-173.us-west-2.compute.internal\" already exists" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-173.us-west-2.compute.internal"
Oct 31 00:28:07.063688 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.063667    4284 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-jsqnl\""
Oct 31 00:28:07.063977 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.063962    4284 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Oct 31 00:28:07.064952 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.064924    4284 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Oct 31 00:28:07.064952 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.064943    4284 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Oct 31 00:28:07.065097 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.064957    4284 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Oct 31 00:28:07.065097 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.065020    4284 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Oct 31 00:28:07.065189 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.065114    4284 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Oct 31 00:28:07.065301 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.065284    4284 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Oct 31 00:28:07.065382 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.065356    4284 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Oct 31 00:28:07.065485 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.065471    4284 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Oct 31 00:28:07.065534 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.065476    4284 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Oct 31 00:28:07.065581 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.065532    4284 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Oct 31 00:28:07.066390 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.066373    4284 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-46b97\""
Oct 31 00:28:07.066467 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.066448    4284 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-v6gdh\""
Oct 31 00:28:07.066519 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.066468    4284 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-p2znq\""
Oct 31 00:28:07.067087 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.067071    4284 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-cdw8b\""
Oct 31 00:28:07.067155 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.067088    4284 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Oct 31 00:28:07.067155 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.067114    4284 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-m2wdr\""
Oct 31 00:28:07.067284 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.067271    4284 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-node-identity-dockercfg-lwt4s\""
Oct 31 00:28:07.067415 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.067395    4284 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-check-target-dockercfg-nwzth\""
Oct 31 00:28:07.067415 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.067401    4284 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Oct 31 00:28:07.067561 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.067403    4284 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"global-pull-secret-syncer-dockercfg-dwmlr\""
Oct 31 00:28:07.067561 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.067429    4284 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-rw8bx\""
Oct 31 00:28:07.067561 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.067413    4284 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Oct 31 00:28:07.067561 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.067440    4284 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Oct 31 00:28:07.067561 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.067460    4284 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent-dockercfg-78fsr\""
Oct 31 00:28:07.067561 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.067493    4284 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Oct 31 00:28:07.067868 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.067849    4284 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Oct 31 00:28:07.067868 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.067852    4284 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Oct 31 00:28:07.067978 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.067858    4284 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Oct 31 00:28:07.067978 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.067865    4284 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Oct 31 00:28:07.067978 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.067902    4284 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Oct 31 00:28:07.068070 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.068023    4284 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Oct 31 00:28:07.068104 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.068075    4284 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Oct 31 00:28:07.068156 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.068144    4284 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Oct 31 00:28:07.068508 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.068496    4284 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Oct 31 00:28:07.068882 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.068867    4284 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Oct 31 00:28:07.068964 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.068952    4284 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Oct 31 00:28:07.069012 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.069007    4284 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Oct 31 00:28:07.085169 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.085153    4284 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Oct 31 00:28:07.085317 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.085306    4284 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Oct 31 00:28:07.121625 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.121602    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/41c4cc4b-b387-4c70-8a27-58255ba632b9-tuning-conf-dir\") pod \"multus-additional-cni-plugins-2t5fx\" (UID: \"41c4cc4b-b387-4c70-8a27-58255ba632b9\") " pod="openshift-multus/multus-additional-cni-plugins-2t5fx"
Oct 31 00:28:07.121689 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.121632    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kplpm\" (UniqueName: \"kubernetes.io/projected/85e4b969-79f8-4954-8539-0905bd5db99d-kube-api-access-kplpm\") pod \"network-metrics-daemon-plnkm\" (UID: \"85e4b969-79f8-4954-8539-0905bd5db99d\") " pod="openshift-multus/network-metrics-daemon-plnkm"
Oct 31 00:28:07.121689 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.121648    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/63bedd38-6ab7-493d-b94c-b3c288af3f48-host-run-netns\") pod \"ovnkube-node-rrk77\" (UID: \"63bedd38-6ab7-493d-b94c-b3c288af3f48\") " pod="openshift-ovn-kubernetes/ovnkube-node-rrk77"
Oct 31 00:28:07.121689 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.121666    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/21df46e4-2158-4385-ab9a-4b1790b64b55-etc-modprobe-d\") pod \"tuned-hrsmn\" (UID: \"21df46e4-2158-4385-ab9a-4b1790b64b55\") " pod="openshift-cluster-node-tuning-operator/tuned-hrsmn"
Oct 31 00:28:07.121782 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.121690    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6926e9b8-7c44-413e-bc7e-63cfff593066-cnibin\") pod \"multus-d7msb\" (UID: \"6926e9b8-7c44-413e-bc7e-63cfff593066\") " pod="openshift-multus/multus-d7msb"
Oct 31 00:28:07.121782 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.121747    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/97410648-b7ae-4cf4-accf-3164c398b525-dbus\") pod \"global-pull-secret-syncer-24sxq\" (UID: \"97410648-b7ae-4cf4-accf-3164c398b525\") " pod="kube-system/global-pull-secret-syncer-24sxq"
Oct 31 00:28:07.121839 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.121798    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/21df46e4-2158-4385-ab9a-4b1790b64b55-etc-systemd\") pod \"tuned-hrsmn\" (UID: \"21df46e4-2158-4385-ab9a-4b1790b64b55\") " pod="openshift-cluster-node-tuning-operator/tuned-hrsmn"
Oct 31 00:28:07.121839 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.121828    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/21df46e4-2158-4385-ab9a-4b1790b64b55-etc-tuned\") pod \"tuned-hrsmn\" (UID: \"21df46e4-2158-4385-ab9a-4b1790b64b55\") " pod="openshift-cluster-node-tuning-operator/tuned-hrsmn"
Oct 31 00:28:07.121919 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.121906    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/6926e9b8-7c44-413e-bc7e-63cfff593066-hostroot\") pod \"multus-d7msb\" (UID: \"6926e9b8-7c44-413e-bc7e-63cfff593066\") " pod="openshift-multus/multus-d7msb"
Oct 31 00:28:07.121952 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.121933    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/21df46e4-2158-4385-ab9a-4b1790b64b55-etc-tuned\") pod \"tuned-hrsmn\" (UID: \"21df46e4-2158-4385-ab9a-4b1790b64b55\") " pod="openshift-cluster-node-tuning-operator/tuned-hrsmn"
Oct 31 00:28:07.121983 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.121976    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/6926e9b8-7c44-413e-bc7e-63cfff593066-host-run-multus-certs\") pod \"multus-d7msb\" (UID: \"6926e9b8-7c44-413e-bc7e-63cfff593066\") " pod="openshift-multus/multus-d7msb"
Oct 31 00:28:07.122020 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.122010    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hxhdf\" (UniqueName: \"kubernetes.io/projected/c4ce4bad-af31-4f2e-a684-b2123e9da265-kube-api-access-hxhdf\") pod \"network-check-target-jktkx\" (UID: \"c4ce4bad-af31-4f2e-a684-b2123e9da265\") " pod="openshift-network-diagnostics/network-check-target-jktkx"
Oct 31 00:28:07.122049 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.122040    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/21df46e4-2158-4385-ab9a-4b1790b64b55-host\") pod \"tuned-hrsmn\" (UID: \"21df46e4-2158-4385-ab9a-4b1790b64b55\") " pod="openshift-cluster-node-tuning-operator/tuned-hrsmn"
Oct 31 00:28:07.122138 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.122124    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/85e4b969-79f8-4954-8539-0905bd5db99d-metrics-certs\") pod \"network-metrics-daemon-plnkm\" (UID: \"85e4b969-79f8-4954-8539-0905bd5db99d\") " pod="openshift-multus/network-metrics-daemon-plnkm"
Oct 31 00:28:07.122182 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.122166    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/63bedd38-6ab7-493d-b94c-b3c288af3f48-host-kubelet\") pod \"ovnkube-node-rrk77\" (UID: \"63bedd38-6ab7-493d-b94c-b3c288af3f48\") " pod="openshift-ovn-kubernetes/ovnkube-node-rrk77"
Oct 31 00:28:07.122233 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.122197    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/63bedd38-6ab7-493d-b94c-b3c288af3f48-systemd-units\") pod \"ovnkube-node-rrk77\" (UID: \"63bedd38-6ab7-493d-b94c-b3c288af3f48\") " pod="openshift-ovn-kubernetes/ovnkube-node-rrk77"
Oct 31 00:28:07.122267 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.122249    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2c9341ef-0860-4fb6-8a86-272ac063756a-tmp-dir\") pod \"node-resolver-zjb6f\" (UID: \"2c9341ef-0860-4fb6-8a86-272ac063756a\") " pod="openshift-dns/node-resolver-zjb6f"
Oct 31 00:28:07.122300 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.122275    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/21df46e4-2158-4385-ab9a-4b1790b64b55-etc-sysctl-conf\") pod \"tuned-hrsmn\" (UID: \"21df46e4-2158-4385-ab9a-4b1790b64b55\") " pod="openshift-cluster-node-tuning-operator/tuned-hrsmn"
Oct 31 00:28:07.122360 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.122348    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2c9341ef-0860-4fb6-8a86-272ac063756a-tmp-dir\") pod \"node-resolver-zjb6f\" (UID: \"2c9341ef-0860-4fb6-8a86-272ac063756a\") " pod="openshift-dns/node-resolver-zjb6f"
Oct 31 00:28:07.122390 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.122362    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/21df46e4-2158-4385-ab9a-4b1790b64b55-lib-modules\") pod \"tuned-hrsmn\" (UID: \"21df46e4-2158-4385-ab9a-4b1790b64b55\") " pod="openshift-cluster-node-tuning-operator/tuned-hrsmn"
Oct 31 00:28:07.122421 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.122394    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/1a72f876-1157-4447-845c-923cdfd5aa5e-kubelet-dir\") pod \"aws-ebs-csi-driver-node-k6fgp\" (UID: \"1a72f876-1157-4447-845c-923cdfd5aa5e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k6fgp"
Oct 31 00:28:07.122421 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.122408    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/61c60450-eb50-4181-8ab7-e69e864ffb51-host\") pod \"node-ca-qchks\" (UID: \"61c60450-eb50-4181-8ab7-e69e864ffb51\") " pod="openshift-image-registry/node-ca-qchks"
Oct 31 00:28:07.122482 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.122422    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/63bedd38-6ab7-493d-b94c-b3c288af3f48-run-openvswitch\") pod \"ovnkube-node-rrk77\" (UID: \"63bedd38-6ab7-493d-b94c-b3c288af3f48\") " pod="openshift-ovn-kubernetes/ovnkube-node-rrk77"
Oct 31 00:28:07.122482 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.122435    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/21df46e4-2158-4385-ab9a-4b1790b64b55-run\") pod \"tuned-hrsmn\" (UID: \"21df46e4-2158-4385-ab9a-4b1790b64b55\") " pod="openshift-cluster-node-tuning-operator/tuned-hrsmn"
Oct 31 00:28:07.122536 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.122480    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/6926e9b8-7c44-413e-bc7e-63cfff593066-host-run-k8s-cni-cncf-io\") pod \"multus-d7msb\" (UID: \"6926e9b8-7c44-413e-bc7e-63cfff593066\") " pod="openshift-multus/multus-d7msb"
Oct 31 00:28:07.122536 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.122517    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/1a72f876-1157-4447-845c-923cdfd5aa5e-socket-dir\") pod \"aws-ebs-csi-driver-node-k6fgp\" (UID: \"1a72f876-1157-4447-845c-923cdfd5aa5e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k6fgp"
Oct 31 00:28:07.122588 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.122536    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/63bedd38-6ab7-493d-b94c-b3c288af3f48-run-ovn\") pod \"ovnkube-node-rrk77\" (UID: \"63bedd38-6ab7-493d-b94c-b3c288af3f48\") " pod="openshift-ovn-kubernetes/ovnkube-node-rrk77"
Oct 31 00:28:07.122588 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.122566    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/63bedd38-6ab7-493d-b94c-b3c288af3f48-ovnkube-config\") pod \"ovnkube-node-rrk77\" (UID: \"63bedd38-6ab7-493d-b94c-b3c288af3f48\") " pod="openshift-ovn-kubernetes/ovnkube-node-rrk77"
Oct 31 00:28:07.122646 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.122591    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/21df46e4-2158-4385-ab9a-4b1790b64b55-etc-kubernetes\") pod \"tuned-hrsmn\" (UID: \"21df46e4-2158-4385-ab9a-4b1790b64b55\") " pod="openshift-cluster-node-tuning-operator/tuned-hrsmn"
Oct 31 00:28:07.122703 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.122690    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6926e9b8-7c44-413e-bc7e-63cfff593066-cni-binary-copy\") pod \"multus-d7msb\" (UID: \"6926e9b8-7c44-413e-bc7e-63cfff593066\") " pod="openshift-multus/multus-d7msb"
Oct 31 00:28:07.122736 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.122719    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6926e9b8-7c44-413e-bc7e-63cfff593066-etc-kubernetes\") pod \"multus-d7msb\" (UID: \"6926e9b8-7c44-413e-bc7e-63cfff593066\") " pod="openshift-multus/multus-d7msb"
Oct 31 00:28:07.122767 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.122733    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/63bedd38-6ab7-493d-b94c-b3c288af3f48-host-slash\") pod \"ovnkube-node-rrk77\" (UID: \"63bedd38-6ab7-493d-b94c-b3c288af3f48\") " pod="openshift-ovn-kubernetes/ovnkube-node-rrk77"
Oct 31 00:28:07.122795 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.122776    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/63bedd38-6ab7-493d-b94c-b3c288af3f48-node-log\") pod \"ovnkube-node-rrk77\" (UID: \"63bedd38-6ab7-493d-b94c-b3c288af3f48\") " pod="openshift-ovn-kubernetes/ovnkube-node-rrk77"
Oct 31 00:28:07.122795 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.122787    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/63bedd38-6ab7-493d-b94c-b3c288af3f48-ovnkube-config\") pod \"ovnkube-node-rrk77\" (UID: \"63bedd38-6ab7-493d-b94c-b3c288af3f48\") " pod="openshift-ovn-kubernetes/ovnkube-node-rrk77"
Oct 31 00:28:07.122923 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.122912    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6926e9b8-7c44-413e-bc7e-63cfff593066-host-var-lib-cni-bin\") pod \"multus-d7msb\" (UID: \"6926e9b8-7c44-413e-bc7e-63cfff593066\") " pod="openshift-multus/multus-d7msb"
Oct 31 00:28:07.122952 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.122933    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/97410648-b7ae-4cf4-accf-3164c398b525-original-pull-secret\") pod \"global-pull-secret-syncer-24sxq\" (UID: \"97410648-b7ae-4cf4-accf-3164c398b525\") " pod="kube-system/global-pull-secret-syncer-24sxq"
Oct 31 00:28:07.122952 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.122948    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/63bedd38-6ab7-493d-b94c-b3c288af3f48-env-overrides\") pod \"ovnkube-node-rrk77\" (UID: \"63bedd38-6ab7-493d-b94c-b3c288af3f48\") " pod="openshift-ovn-kubernetes/ovnkube-node-rrk77"
Oct 31 00:28:07.123003 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.122962    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/9daae1e9-e437-437c-ba8b-3d9c206b64a3-iptables-alerter-script\") pod \"iptables-alerter-p4txz\" (UID: \"9daae1e9-e437-437c-ba8b-3d9c206b64a3\") " pod="openshift-network-operator/iptables-alerter-p4txz"
Oct 31 00:28:07.123003 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.122976    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/41c4cc4b-b387-4c70-8a27-58255ba632b9-cnibin\") pod \"multus-additional-cni-plugins-2t5fx\" (UID: \"41c4cc4b-b387-4c70-8a27-58255ba632b9\") " pod="openshift-multus/multus-additional-cni-plugins-2t5fx"
Oct 31 00:28:07.123003 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.122988    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6926e9b8-7c44-413e-bc7e-63cfff593066-cni-binary-copy\") pod \"multus-d7msb\" (UID: \"6926e9b8-7c44-413e-bc7e-63cfff593066\") " pod="openshift-multus/multus-d7msb"
Oct 31 00:28:07.123003 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.122992    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/63bedd38-6ab7-493d-b94c-b3c288af3f48-etc-openvswitch\") pod \"ovnkube-node-rrk77\" (UID: \"63bedd38-6ab7-493d-b94c-b3c288af3f48\") " pod="openshift-ovn-kubernetes/ovnkube-node-rrk77"
Oct 31 00:28:07.123136 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.123017    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/2c9341ef-0860-4fb6-8a86-272ac063756a-hosts-file\") pod \"node-resolver-zjb6f\" (UID: \"2c9341ef-0860-4fb6-8a86-272ac063756a\") " pod="openshift-dns/node-resolver-zjb6f"
Oct 31 00:28:07.123136 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.123042    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/6926e9b8-7c44-413e-bc7e-63cfff593066-multus-socket-dir-parent\") pod \"multus-d7msb\" (UID: \"6926e9b8-7c44-413e-bc7e-63cfff593066\") " pod="openshift-multus/multus-d7msb"
Oct 31 00:28:07.123136 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.123059    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6926e9b8-7c44-413e-bc7e-63cfff593066-multus-conf-dir\") pod \"multus-d7msb\" (UID: \"6926e9b8-7c44-413e-bc7e-63cfff593066\") " pod="openshift-multus/multus-d7msb"
Oct 31 00:28:07.123136 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.123091    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/61c60450-eb50-4181-8ab7-e69e864ffb51-serviceca\") pod \"node-ca-qchks\" (UID: \"61c60450-eb50-4181-8ab7-e69e864ffb51\") " pod="openshift-image-registry/node-ca-qchks"
Oct 31 00:28:07.123136 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.123114    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/63bedd38-6ab7-493d-b94c-b3c288af3f48-env-overrides\") pod \"ovnkube-node-rrk77\" (UID: \"63bedd38-6ab7-493d-b94c-b3c288af3f48\") " pod="openshift-ovn-kubernetes/ovnkube-node-rrk77"
Oct 31 00:28:07.123363 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.123163    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/63bedd38-6ab7-493d-b94c-b3c288af3f48-log-socket\") pod \"ovnkube-node-rrk77\" (UID: \"63bedd38-6ab7-493d-b94c-b3c288af3f48\") " pod="openshift-ovn-kubernetes/ovnkube-node-rrk77"
Oct 31 00:28:07.123363 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.123271    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/97410648-b7ae-4cf4-accf-3164c398b525-original-pull-secret\") pod \"global-pull-secret-syncer-24sxq\" (UID: \"97410648-b7ae-4cf4-accf-3164c398b525\") " pod="kube-system/global-pull-secret-syncer-24sxq"
Oct 31 00:28:07.123363 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.123355    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/21df46e4-2158-4385-ab9a-4b1790b64b55-sys\") pod \"tuned-hrsmn\" (UID: \"21df46e4-2158-4385-ab9a-4b1790b64b55\") " pod="openshift-cluster-node-tuning-operator/tuned-hrsmn"
Oct 31 00:28:07.123508 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.123367    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/9daae1e9-e437-437c-ba8b-3d9c206b64a3-iptables-alerter-script\") pod \"iptables-alerter-p4txz\" (UID: \"9daae1e9-e437-437c-ba8b-3d9c206b64a3\") " pod="openshift-network-operator/iptables-alerter-p4txz"
Oct 31 00:28:07.123508 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.123398    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/21df46e4-2158-4385-ab9a-4b1790b64b55-var-lib-kubelet\") pod \"tuned-hrsmn\" (UID: \"21df46e4-2158-4385-ab9a-4b1790b64b55\") " pod="openshift-cluster-node-tuning-operator/tuned-hrsmn"
Oct 31 00:28:07.123508 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.123418    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6926e9b8-7c44-413e-bc7e-63cfff593066-system-cni-dir\") pod \"multus-d7msb\" (UID: \"6926e9b8-7c44-413e-bc7e-63cfff593066\") " pod="openshift-multus/multus-d7msb"
Oct 31 00:28:07.123508 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.123435    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/1a72f876-1157-4447-845c-923cdfd5aa5e-registration-dir\") pod \"aws-ebs-csi-driver-node-k6fgp\" (UID: \"1a72f876-1157-4447-845c-923cdfd5aa5e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k6fgp"
Oct 31 00:28:07.123508 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.123469    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xhp7k\" (UniqueName: \"kubernetes.io/projected/1a72f876-1157-4447-845c-923cdfd5aa5e-kube-api-access-xhp7k\") pod \"aws-ebs-csi-driver-node-k6fgp\" (UID: \"1a72f876-1157-4447-845c-923cdfd5aa5e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k6fgp"
Oct 31 00:28:07.123508 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.123505    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/9daae1e9-e437-437c-ba8b-3d9c206b64a3-host-slash\") pod \"iptables-alerter-p4txz\" (UID: \"9daae1e9-e437-437c-ba8b-3d9c206b64a3\") " pod="openshift-network-operator/iptables-alerter-p4txz"
Oct 31 00:28:07.123782 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.123552    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/41c4cc4b-b387-4c70-8a27-58255ba632b9-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-2t5fx\" (UID: \"41c4cc4b-b387-4c70-8a27-58255ba632b9\") " pod="openshift-multus/multus-additional-cni-plugins-2t5fx"
Oct 31 00:28:07.123782 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.123571    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x6bgk\" (UniqueName: \"kubernetes.io/projected/2c9341ef-0860-4fb6-8a86-272ac063756a-kube-api-access-x6bgk\") pod \"node-resolver-zjb6f\" (UID: \"2c9341ef-0860-4fb6-8a86-272ac063756a\") " pod="openshift-dns/node-resolver-zjb6f"
Oct 31 00:28:07.123782 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.123592    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/21df46e4-2158-4385-ab9a-4b1790b64b55-tmp\") pod \"tuned-hrsmn\" (UID: \"21df46e4-2158-4385-ab9a-4b1790b64b55\") " pod="openshift-cluster-node-tuning-operator/tuned-hrsmn"
Oct 31 00:28:07.123782 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.123657    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6926e9b8-7c44-413e-bc7e-63cfff593066-multus-cni-dir\") pod \"multus-d7msb\" (UID: \"6926e9b8-7c44-413e-bc7e-63cfff593066\") " pod="openshift-multus/multus-d7msb"
Oct 31 00:28:07.123782 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.123685    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/21df46e4-2158-4385-ab9a-4b1790b64b55-tmp\") pod \"tuned-hrsmn\" (UID: \"21df46e4-2158-4385-ab9a-4b1790b64b55\") " pod="openshift-cluster-node-tuning-operator/tuned-hrsmn"
Oct 31 00:28:07.123782 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.123686    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/1a72f876-1157-4447-845c-923cdfd5aa5e-device-dir\") pod \"aws-ebs-csi-driver-node-k6fgp\" (UID: \"1a72f876-1157-4447-845c-923cdfd5aa5e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k6fgp"
Oct 31 00:28:07.123782 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.123722    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/1a72f876-1157-4447-845c-923cdfd5aa5e-etc-selinux\") pod \"aws-ebs-csi-driver-node-k6fgp\" (UID: \"1a72f876-1157-4447-845c-923cdfd5aa5e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k6fgp"
Oct 31 00:28:07.123782 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.123750    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/41c4cc4b-b387-4c70-8a27-58255ba632b9-system-cni-dir\") pod \"multus-additional-cni-plugins-2t5fx\" (UID: \"41c4cc4b-b387-4c70-8a27-58255ba632b9\") " pod="openshift-multus/multus-additional-cni-plugins-2t5fx"
Oct 31 00:28:07.123782 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.123777    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/41c4cc4b-b387-4c70-8a27-58255ba632b9-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-2t5fx\" (UID: \"41c4cc4b-b387-4c70-8a27-58255ba632b9\") " pod="openshift-multus/multus-additional-cni-plugins-2t5fx"
Oct 31 00:28:07.124300 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.123779    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/41c4cc4b-b387-4c70-8a27-58255ba632b9-cni-binary-copy\") pod \"multus-additional-cni-plugins-2t5fx\" (UID: \"41c4cc4b-b387-4c70-8a27-58255ba632b9\") " pod="openshift-multus/multus-additional-cni-plugins-2t5fx"
Oct 31 00:28:07.124300 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.123814    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/6926e9b8-7c44-413e-bc7e-63cfff593066-host-var-lib-cni-multus\") pod \"multus-d7msb\" (UID: \"6926e9b8-7c44-413e-bc7e-63cfff593066\") " pod="openshift-multus/multus-d7msb"
Oct 31 00:28:07.124300 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.123844    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m2phw\" (UniqueName: \"kubernetes.io/projected/6926e9b8-7c44-413e-bc7e-63cfff593066-kube-api-access-m2phw\") pod \"multus-d7msb\" (UID: \"6926e9b8-7c44-413e-bc7e-63cfff593066\") " pod="openshift-multus/multus-d7msb"
Oct 31 00:28:07.124300 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.123883    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/1a72f876-1157-4447-845c-923cdfd5aa5e-sys-fs\") pod \"aws-ebs-csi-driver-node-k6fgp\" (UID: \"1a72f876-1157-4447-845c-923cdfd5aa5e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k6fgp"
Oct 31 00:28:07.124300 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.123888    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/61c60450-eb50-4181-8ab7-e69e864ffb51-serviceca\") pod \"node-ca-qchks\" (UID: \"61c60450-eb50-4181-8ab7-e69e864ffb51\") " pod="openshift-image-registry/node-ca-qchks"
Oct 31 00:28:07.124300 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.123940    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/63bedd38-6ab7-493d-b94c-b3c288af3f48-run-systemd\") pod \"ovnkube-node-rrk77\" (UID: \"63bedd38-6ab7-493d-b94c-b3c288af3f48\") " pod="openshift-ovn-kubernetes/ovnkube-node-rrk77"
Oct 31 00:28:07.124300 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.123943    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/41c4cc4b-b387-4c70-8a27-58255ba632b9-cni-binary-copy\") pod \"multus-additional-cni-plugins-2t5fx\" (UID: \"41c4cc4b-b387-4c70-8a27-58255ba632b9\") " pod="openshift-multus/multus-additional-cni-plugins-2t5fx"
Oct 31 00:28:07.124300 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.123983    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/63bedd38-6ab7-493d-b94c-b3c288af3f48-var-lib-openvswitch\") pod \"ovnkube-node-rrk77\" (UID: \"63bedd38-6ab7-493d-b94c-b3c288af3f48\") " pod="openshift-ovn-kubernetes/ovnkube-node-rrk77"
Oct 31 00:28:07.124300 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.124017    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/63bedd38-6ab7-493d-b94c-b3c288af3f48-host-cni-netd\") pod \"ovnkube-node-rrk77\" (UID: \"63bedd38-6ab7-493d-b94c-b3c288af3f48\") " pod="openshift-ovn-kubernetes/ovnkube-node-rrk77"
Oct 31 00:28:07.124300 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.124045    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/21df46e4-2158-4385-ab9a-4b1790b64b55-etc-sysconfig\") pod \"tuned-hrsmn\" (UID: \"21df46e4-2158-4385-ab9a-4b1790b64b55\") " pod="openshift-cluster-node-tuning-operator/tuned-hrsmn"
Oct 31 00:28:07.124300 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.124072    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4xxhd\" (UniqueName: \"kubernetes.io/projected/9daae1e9-e437-437c-ba8b-3d9c206b64a3-kube-api-access-4xxhd\") pod \"iptables-alerter-p4txz\" (UID: \"9daae1e9-e437-437c-ba8b-3d9c206b64a3\") " pod="openshift-network-operator/iptables-alerter-p4txz"
Oct 31 00:28:07.124300 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.124101    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/63bedd38-6ab7-493d-b94c-b3c288af3f48-host-cni-bin\") pod \"ovnkube-node-rrk77\" (UID: \"63bedd38-6ab7-493d-b94c-b3c288af3f48\") " pod="openshift-ovn-kubernetes/ovnkube-node-rrk77"
Oct 31 00:28:07.124300 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.124130    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/63bedd38-6ab7-493d-b94c-b3c288af3f48-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-rrk77\" (UID: \"63bedd38-6ab7-493d-b94c-b3c288af3f48\") " pod="openshift-ovn-kubernetes/ovnkube-node-rrk77"
Oct 31 00:28:07.124300 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.124158    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/63bedd38-6ab7-493d-b94c-b3c288af3f48-ovn-node-metrics-cert\") pod \"ovnkube-node-rrk77\" (UID: \"63bedd38-6ab7-493d-b94c-b3c288af3f48\") " pod="openshift-ovn-kubernetes/ovnkube-node-rrk77"
Oct 31 00:28:07.124300 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.124231    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/63bedd38-6ab7-493d-b94c-b3c288af3f48-ovnkube-script-lib\") pod \"ovnkube-node-rrk77\" (UID: \"63bedd38-6ab7-493d-b94c-b3c288af3f48\") " pod="openshift-ovn-kubernetes/ovnkube-node-rrk77"
Oct 31 00:28:07.124300 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.124259    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vp9kn\" (UniqueName: \"kubernetes.io/projected/63bedd38-6ab7-493d-b94c-b3c288af3f48-kube-api-access-vp9kn\") pod \"ovnkube-node-rrk77\" (UID: \"63bedd38-6ab7-493d-b94c-b3c288af3f48\") " pod="openshift-ovn-kubernetes/ovnkube-node-rrk77"
Oct 31 00:28:07.124300 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.124285    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/21df46e4-2158-4385-ab9a-4b1790b64b55-etc-sysctl-d\") pod \"tuned-hrsmn\" (UID: \"21df46e4-2158-4385-ab9a-4b1790b64b55\") " pod="openshift-cluster-node-tuning-operator/tuned-hrsmn"
Oct 31 00:28:07.125002 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.124469    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6926e9b8-7c44-413e-bc7e-63cfff593066-host-var-lib-kubelet\") pod \"multus-d7msb\" (UID: \"6926e9b8-7c44-413e-bc7e-63cfff593066\") " pod="openshift-multus/multus-d7msb"
Oct 31 00:28:07.125002 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.124486    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/63bedd38-6ab7-493d-b94c-b3c288af3f48-ovn-node-metrics-cert\") pod \"ovnkube-node-rrk77\" (UID: \"63bedd38-6ab7-493d-b94c-b3c288af3f48\") " pod="openshift-ovn-kubernetes/ovnkube-node-rrk77"
Oct 31 00:28:07.125002 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.124492    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/41c4cc4b-b387-4c70-8a27-58255ba632b9-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-2t5fx\" (UID: \"41c4cc4b-b387-4c70-8a27-58255ba632b9\") " pod="openshift-multus/multus-additional-cni-plugins-2t5fx"
Oct 31 00:28:07.125002 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.124518    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/63bedd38-6ab7-493d-b94c-b3c288af3f48-host-run-ovn-kubernetes\") pod \"ovnkube-node-rrk77\" (UID: \"63bedd38-6ab7-493d-b94c-b3c288af3f48\") " pod="openshift-ovn-kubernetes/ovnkube-node-rrk77"
Oct 31 00:28:07.125002 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.124532    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/63bedd38-6ab7-493d-b94c-b3c288af3f48-ovnkube-script-lib\") pod \"ovnkube-node-rrk77\" (UID: \"63bedd38-6ab7-493d-b94c-b3c288af3f48\") " pod="openshift-ovn-kubernetes/ovnkube-node-rrk77"
Oct 31 00:28:07.125002 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.124541    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6926e9b8-7c44-413e-bc7e-63cfff593066-host-run-netns\") pod \"multus-d7msb\" (UID: \"6926e9b8-7c44-413e-bc7e-63cfff593066\") " pod="openshift-multus/multus-d7msb"
Oct 31 00:28:07.125002 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.124627    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/6926e9b8-7c44-413e-bc7e-63cfff593066-multus-daemon-config\") pod \"multus-d7msb\" (UID: \"6926e9b8-7c44-413e-bc7e-63cfff593066\") " pod="openshift-multus/multus-d7msb"
Oct 31 00:28:07.125002 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.124665    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/97410648-b7ae-4cf4-accf-3164c398b525-kubelet-config\") pod \"global-pull-secret-syncer-24sxq\" (UID: \"97410648-b7ae-4cf4-accf-3164c398b525\") " pod="kube-system/global-pull-secret-syncer-24sxq"
Oct 31 00:28:07.125002 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.124691    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sslf9\" (UniqueName: \"kubernetes.io/projected/61c60450-eb50-4181-8ab7-e69e864ffb51-kube-api-access-sslf9\") pod \"node-ca-qchks\" (UID: \"61c60450-eb50-4181-8ab7-e69e864ffb51\") " pod="openshift-image-registry/node-ca-qchks"
Oct 31 00:28:07.125002 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.124707    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/41c4cc4b-b387-4c70-8a27-58255ba632b9-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-2t5fx\" (UID: \"41c4cc4b-b387-4c70-8a27-58255ba632b9\") " pod="openshift-multus/multus-additional-cni-plugins-2t5fx"
Oct 31 00:28:07.125002 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.124717    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vrldg\" (UniqueName: \"kubernetes.io/projected/41c4cc4b-b387-4c70-8a27-58255ba632b9-kube-api-access-vrldg\") pod \"multus-additional-cni-plugins-2t5fx\" (UID: \"41c4cc4b-b387-4c70-8a27-58255ba632b9\") " pod="openshift-multus/multus-additional-cni-plugins-2t5fx"
Oct 31 00:28:07.125002 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.124760    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gm472\" (UniqueName: \"kubernetes.io/projected/21df46e4-2158-4385-ab9a-4b1790b64b55-kube-api-access-gm472\") pod \"tuned-hrsmn\" (UID: \"21df46e4-2158-4385-ab9a-4b1790b64b55\") " pod="openshift-cluster-node-tuning-operator/tuned-hrsmn"
Oct 31 00:28:07.125002 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.124780    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/f93a119c-fb17-44bd-973c-699a3b87523b-agent-certs\") pod \"konnectivity-agent-k2sk2\" (UID: \"f93a119c-fb17-44bd-973c-699a3b87523b\") " pod="kube-system/konnectivity-agent-k2sk2"
Oct 31 00:28:07.125002 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.124794    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/f93a119c-fb17-44bd-973c-699a3b87523b-konnectivity-ca\") pod \"konnectivity-agent-k2sk2\" (UID: \"f93a119c-fb17-44bd-973c-699a3b87523b\") " pod="kube-system/konnectivity-agent-k2sk2"
Oct 31 00:28:07.125002 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.124818    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/6926e9b8-7c44-413e-bc7e-63cfff593066-multus-daemon-config\") pod \"multus-d7msb\" (UID: \"6926e9b8-7c44-413e-bc7e-63cfff593066\") " pod="openshift-multus/multus-d7msb"
Oct 31 00:28:07.125526 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.125035    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/f93a119c-fb17-44bd-973c-699a3b87523b-konnectivity-ca\") pod \"konnectivity-agent-k2sk2\" (UID: \"f93a119c-fb17-44bd-973c-699a3b87523b\") " pod="kube-system/konnectivity-agent-k2sk2"
Oct 31 00:28:07.125526 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.125064    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/f93a119c-fb17-44bd-973c-699a3b87523b-agent-certs\") pod \"konnectivity-agent-k2sk2\" (UID: \"f93a119c-fb17-44bd-973c-699a3b87523b\") " pod="kube-system/konnectivity-agent-k2sk2"
Oct 31 00:28:07.163737 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.163686    4284 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Oct 31 00:28:07.163737 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.163690    4284 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-rlgrr\""
Oct 31 00:28:07.177685 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.177659    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kplpm\" (UniqueName: \"kubernetes.io/projected/85e4b969-79f8-4954-8539-0905bd5db99d-kube-api-access-kplpm\") pod \"network-metrics-daemon-plnkm\" (UID: \"85e4b969-79f8-4954-8539-0905bd5db99d\") " pod="openshift-multus/network-metrics-daemon-plnkm"
Oct 31 00:28:07.178069 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.178045    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hxhdf\" (UniqueName: \"kubernetes.io/projected/c4ce4bad-af31-4f2e-a684-b2123e9da265-kube-api-access-hxhdf\") pod \"network-check-target-jktkx\" (UID: \"c4ce4bad-af31-4f2e-a684-b2123e9da265\") " pod="openshift-network-diagnostics/network-check-target-jktkx"
Oct 31 00:28:07.179687 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.179668    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xhp7k\" (UniqueName: \"kubernetes.io/projected/1a72f876-1157-4447-845c-923cdfd5aa5e-kube-api-access-xhp7k\") pod \"aws-ebs-csi-driver-node-k6fgp\" (UID: \"1a72f876-1157-4447-845c-923cdfd5aa5e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k6fgp"
Oct 31 00:28:07.179968 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.179950    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x6bgk\" (UniqueName: \"kubernetes.io/projected/2c9341ef-0860-4fb6-8a86-272ac063756a-kube-api-access-x6bgk\") pod \"node-resolver-zjb6f\" (UID: \"2c9341ef-0860-4fb6-8a86-272ac063756a\") " pod="openshift-dns/node-resolver-zjb6f"
Oct 31 00:28:07.180077 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.180059    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m2phw\" (UniqueName: \"kubernetes.io/projected/6926e9b8-7c44-413e-bc7e-63cfff593066-kube-api-access-m2phw\") pod \"multus-d7msb\" (UID: \"6926e9b8-7c44-413e-bc7e-63cfff593066\") " pod="openshift-multus/multus-d7msb"
Oct 31 00:28:07.180827 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.180806    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vp9kn\" (UniqueName: \"kubernetes.io/projected/63bedd38-6ab7-493d-b94c-b3c288af3f48-kube-api-access-vp9kn\") pod \"ovnkube-node-rrk77\" (UID: \"63bedd38-6ab7-493d-b94c-b3c288af3f48\") " pod="openshift-ovn-kubernetes/ovnkube-node-rrk77"
Oct 31 00:28:07.180890 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.180823    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4xxhd\" (UniqueName: \"kubernetes.io/projected/9daae1e9-e437-437c-ba8b-3d9c206b64a3-kube-api-access-4xxhd\") pod \"iptables-alerter-p4txz\" (UID: \"9daae1e9-e437-437c-ba8b-3d9c206b64a3\") " pod="openshift-network-operator/iptables-alerter-p4txz"
Oct 31 00:28:07.181394 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.181376    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sslf9\" (UniqueName: \"kubernetes.io/projected/61c60450-eb50-4181-8ab7-e69e864ffb51-kube-api-access-sslf9\") pod \"node-ca-qchks\" (UID: \"61c60450-eb50-4181-8ab7-e69e864ffb51\") " pod="openshift-image-registry/node-ca-qchks"
Oct 31 00:28:07.181563 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.181548    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vrldg\" (UniqueName: \"kubernetes.io/projected/41c4cc4b-b387-4c70-8a27-58255ba632b9-kube-api-access-vrldg\") pod \"multus-additional-cni-plugins-2t5fx\" (UID: \"41c4cc4b-b387-4c70-8a27-58255ba632b9\") " pod="openshift-multus/multus-additional-cni-plugins-2t5fx"
Oct 31 00:28:07.181652 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.181638    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gm472\" (UniqueName: \"kubernetes.io/projected/21df46e4-2158-4385-ab9a-4b1790b64b55-kube-api-access-gm472\") pod \"tuned-hrsmn\" (UID: \"21df46e4-2158-4385-ab9a-4b1790b64b55\") " pod="openshift-cluster-node-tuning-operator/tuned-hrsmn"
Oct 31 00:28:07.225592 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.225561    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/41c4cc4b-b387-4c70-8a27-58255ba632b9-cnibin\") pod \"multus-additional-cni-plugins-2t5fx\" (UID: \"41c4cc4b-b387-4c70-8a27-58255ba632b9\") " pod="openshift-multus/multus-additional-cni-plugins-2t5fx"
Oct 31 00:28:07.225665 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.225604    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/63bedd38-6ab7-493d-b94c-b3c288af3f48-etc-openvswitch\") pod \"ovnkube-node-rrk77\" (UID: \"63bedd38-6ab7-493d-b94c-b3c288af3f48\") " pod="openshift-ovn-kubernetes/ovnkube-node-rrk77"
Oct 31 00:28:07.225665 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.225620    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/2c9341ef-0860-4fb6-8a86-272ac063756a-hosts-file\") pod \"node-resolver-zjb6f\" (UID: \"2c9341ef-0860-4fb6-8a86-272ac063756a\") " pod="openshift-dns/node-resolver-zjb6f"
Oct 31 00:28:07.225665 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.225642    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/6926e9b8-7c44-413e-bc7e-63cfff593066-multus-socket-dir-parent\") pod \"multus-d7msb\" (UID: \"6926e9b8-7c44-413e-bc7e-63cfff593066\") " pod="openshift-multus/multus-d7msb"
Oct 31 00:28:07.225799 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.225664    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/41c4cc4b-b387-4c70-8a27-58255ba632b9-cnibin\") pod \"multus-additional-cni-plugins-2t5fx\" (UID: \"41c4cc4b-b387-4c70-8a27-58255ba632b9\") " pod="openshift-multus/multus-additional-cni-plugins-2t5fx"
Oct 31 00:28:07.225799 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.225664    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6926e9b8-7c44-413e-bc7e-63cfff593066-multus-conf-dir\") pod \"multus-d7msb\" (UID: \"6926e9b8-7c44-413e-bc7e-63cfff593066\") " pod="openshift-multus/multus-d7msb"
Oct 31 00:28:07.225799 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.225683    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/63bedd38-6ab7-493d-b94c-b3c288af3f48-etc-openvswitch\") pod \"ovnkube-node-rrk77\" (UID: \"63bedd38-6ab7-493d-b94c-b3c288af3f48\") " pod="openshift-ovn-kubernetes/ovnkube-node-rrk77"
Oct 31 00:28:07.225799 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.225703    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/2c9341ef-0860-4fb6-8a86-272ac063756a-hosts-file\") pod \"node-resolver-zjb6f\" (UID: \"2c9341ef-0860-4fb6-8a86-272ac063756a\") " pod="openshift-dns/node-resolver-zjb6f"
Oct 31 00:28:07.225799 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.225714    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6926e9b8-7c44-413e-bc7e-63cfff593066-multus-conf-dir\") pod \"multus-d7msb\" (UID: \"6926e9b8-7c44-413e-bc7e-63cfff593066\") " pod="openshift-multus/multus-d7msb"
Oct 31 00:28:07.225799 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.225725    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/6926e9b8-7c44-413e-bc7e-63cfff593066-multus-socket-dir-parent\") pod \"multus-d7msb\" (UID: \"6926e9b8-7c44-413e-bc7e-63cfff593066\") " pod="openshift-multus/multus-d7msb"
Oct 31 00:28:07.225799 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.225723    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/63bedd38-6ab7-493d-b94c-b3c288af3f48-log-socket\") pod \"ovnkube-node-rrk77\" (UID: \"63bedd38-6ab7-493d-b94c-b3c288af3f48\") " pod="openshift-ovn-kubernetes/ovnkube-node-rrk77"
Oct 31 00:28:07.225799 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.225785    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/63bedd38-6ab7-493d-b94c-b3c288af3f48-log-socket\") pod \"ovnkube-node-rrk77\" (UID: \"63bedd38-6ab7-493d-b94c-b3c288af3f48\") " pod="openshift-ovn-kubernetes/ovnkube-node-rrk77"
Oct 31 00:28:07.226071 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.225805    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/21df46e4-2158-4385-ab9a-4b1790b64b55-sys\") pod \"tuned-hrsmn\" (UID: \"21df46e4-2158-4385-ab9a-4b1790b64b55\") " pod="openshift-cluster-node-tuning-operator/tuned-hrsmn"
Oct 31 00:28:07.226071 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.225824    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/21df46e4-2158-4385-ab9a-4b1790b64b55-var-lib-kubelet\") pod \"tuned-hrsmn\" (UID: \"21df46e4-2158-4385-ab9a-4b1790b64b55\") " pod="openshift-cluster-node-tuning-operator/tuned-hrsmn"
Oct 31 00:28:07.226071 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.225838    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6926e9b8-7c44-413e-bc7e-63cfff593066-system-cni-dir\") pod \"multus-d7msb\" (UID: \"6926e9b8-7c44-413e-bc7e-63cfff593066\") " pod="openshift-multus/multus-d7msb"
Oct 31 00:28:07.226071 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.225854    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/1a72f876-1157-4447-845c-923cdfd5aa5e-registration-dir\") pod \"aws-ebs-csi-driver-node-k6fgp\" (UID: \"1a72f876-1157-4447-845c-923cdfd5aa5e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k6fgp"
Oct 31 00:28:07.226071 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.225878    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/9daae1e9-e437-437c-ba8b-3d9c206b64a3-host-slash\") pod \"iptables-alerter-p4txz\" (UID: \"9daae1e9-e437-437c-ba8b-3d9c206b64a3\") " pod="openshift-network-operator/iptables-alerter-p4txz"
Oct 31 00:28:07.226071 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.225888    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/21df46e4-2158-4385-ab9a-4b1790b64b55-var-lib-kubelet\") pod \"tuned-hrsmn\" (UID: \"21df46e4-2158-4385-ab9a-4b1790b64b55\") " pod="openshift-cluster-node-tuning-operator/tuned-hrsmn"
Oct 31 00:28:07.226071 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.225901    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6926e9b8-7c44-413e-bc7e-63cfff593066-multus-cni-dir\") pod \"multus-d7msb\" (UID: \"6926e9b8-7c44-413e-bc7e-63cfff593066\") " pod="openshift-multus/multus-d7msb"
Oct 31 00:28:07.226071 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.226023    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/1a72f876-1157-4447-845c-923cdfd5aa5e-device-dir\") pod \"aws-ebs-csi-driver-node-k6fgp\" (UID: \"1a72f876-1157-4447-845c-923cdfd5aa5e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k6fgp"
Oct 31 00:28:07.226071 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.226049    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/1a72f876-1157-4447-845c-923cdfd5aa5e-etc-selinux\") pod \"aws-ebs-csi-driver-node-k6fgp\" (UID: \"1a72f876-1157-4447-845c-923cdfd5aa5e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k6fgp"
Oct 31 00:28:07.226071 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.225932    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/9daae1e9-e437-437c-ba8b-3d9c206b64a3-host-slash\") pod \"iptables-alerter-p4txz\" (UID: \"9daae1e9-e437-437c-ba8b-3d9c206b64a3\") " pod="openshift-network-operator/iptables-alerter-p4txz"
Oct 31 00:28:07.226071 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.226075    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/41c4cc4b-b387-4c70-8a27-58255ba632b9-system-cni-dir\") pod \"multus-additional-cni-plugins-2t5fx\" (UID: \"41c4cc4b-b387-4c70-8a27-58255ba632b9\") " pod="openshift-multus/multus-additional-cni-plugins-2t5fx"
Oct 31 00:28:07.226514 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.226093    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/6926e9b8-7c44-413e-bc7e-63cfff593066-host-var-lib-cni-multus\") pod \"multus-d7msb\" (UID: \"6926e9b8-7c44-413e-bc7e-63cfff593066\") " pod="openshift-multus/multus-d7msb"
Oct 31 00:28:07.226514 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.225972    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6926e9b8-7c44-413e-bc7e-63cfff593066-system-cni-dir\") pod \"multus-d7msb\" (UID: \"6926e9b8-7c44-413e-bc7e-63cfff593066\") " pod="openshift-multus/multus-d7msb"
Oct 31 00:28:07.226514 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.226114    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/1a72f876-1157-4447-845c-923cdfd5aa5e-sys-fs\") pod \"aws-ebs-csi-driver-node-k6fgp\" (UID: \"1a72f876-1157-4447-845c-923cdfd5aa5e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k6fgp"
Oct 31 00:28:07.226514 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.226121    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/1a72f876-1157-4447-845c-923cdfd5aa5e-etc-selinux\") pod \"aws-ebs-csi-driver-node-k6fgp\" (UID: \"1a72f876-1157-4447-845c-923cdfd5aa5e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k6fgp"
Oct 31 00:28:07.226514 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.226130    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/41c4cc4b-b387-4c70-8a27-58255ba632b9-system-cni-dir\") pod \"multus-additional-cni-plugins-2t5fx\" (UID: \"41c4cc4b-b387-4c70-8a27-58255ba632b9\") " pod="openshift-multus/multus-additional-cni-plugins-2t5fx"
Oct 31 00:28:07.226514 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.225934    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/1a72f876-1157-4447-845c-923cdfd5aa5e-registration-dir\") pod \"aws-ebs-csi-driver-node-k6fgp\" (UID: \"1a72f876-1157-4447-845c-923cdfd5aa5e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k6fgp"
Oct 31 00:28:07.226514 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.226140    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/1a72f876-1157-4447-845c-923cdfd5aa5e-device-dir\") pod \"aws-ebs-csi-driver-node-k6fgp\" (UID: \"1a72f876-1157-4447-845c-923cdfd5aa5e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k6fgp"
Oct 31 00:28:07.226514 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.225966    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6926e9b8-7c44-413e-bc7e-63cfff593066-multus-cni-dir\") pod \"multus-d7msb\" (UID: \"6926e9b8-7c44-413e-bc7e-63cfff593066\") " pod="openshift-multus/multus-d7msb"
Oct 31 00:28:07.226514 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.225933    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/21df46e4-2158-4385-ab9a-4b1790b64b55-sys\") pod \"tuned-hrsmn\" (UID: \"21df46e4-2158-4385-ab9a-4b1790b64b55\") " pod="openshift-cluster-node-tuning-operator/tuned-hrsmn"
Oct 31 00:28:07.226514 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.226431    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/63bedd38-6ab7-493d-b94c-b3c288af3f48-run-systemd\") pod \"ovnkube-node-rrk77\" (UID: \"63bedd38-6ab7-493d-b94c-b3c288af3f48\") " pod="openshift-ovn-kubernetes/ovnkube-node-rrk77"
Oct 31 00:28:07.226514 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.226485    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/1a72f876-1157-4447-845c-923cdfd5aa5e-sys-fs\") pod \"aws-ebs-csi-driver-node-k6fgp\" (UID: \"1a72f876-1157-4447-845c-923cdfd5aa5e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k6fgp"
Oct 31 00:28:07.227031 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.226167    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/6926e9b8-7c44-413e-bc7e-63cfff593066-host-var-lib-cni-multus\") pod \"multus-d7msb\" (UID: \"6926e9b8-7c44-413e-bc7e-63cfff593066\") " pod="openshift-multus/multus-d7msb"
Oct 31 00:28:07.227031 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.226652    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/63bedd38-6ab7-493d-b94c-b3c288af3f48-run-systemd\") pod \"ovnkube-node-rrk77\" (UID: \"63bedd38-6ab7-493d-b94c-b3c288af3f48\") " pod="openshift-ovn-kubernetes/ovnkube-node-rrk77"
Oct 31 00:28:07.227031 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.226698    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/63bedd38-6ab7-493d-b94c-b3c288af3f48-var-lib-openvswitch\") pod \"ovnkube-node-rrk77\" (UID: \"63bedd38-6ab7-493d-b94c-b3c288af3f48\") " pod="openshift-ovn-kubernetes/ovnkube-node-rrk77"
Oct 31 00:28:07.227031 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.226732    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/63bedd38-6ab7-493d-b94c-b3c288af3f48-var-lib-openvswitch\") pod \"ovnkube-node-rrk77\" (UID: \"63bedd38-6ab7-493d-b94c-b3c288af3f48\") " pod="openshift-ovn-kubernetes/ovnkube-node-rrk77"
Oct 31 00:28:07.227031 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.226756    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/63bedd38-6ab7-493d-b94c-b3c288af3f48-host-cni-netd\") pod \"ovnkube-node-rrk77\" (UID: \"63bedd38-6ab7-493d-b94c-b3c288af3f48\") " pod="openshift-ovn-kubernetes/ovnkube-node-rrk77"
Oct 31 00:28:07.227031 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.226786    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/21df46e4-2158-4385-ab9a-4b1790b64b55-etc-sysconfig\") pod \"tuned-hrsmn\" (UID: \"21df46e4-2158-4385-ab9a-4b1790b64b55\") " pod="openshift-cluster-node-tuning-operator/tuned-hrsmn"
Oct 31 00:28:07.227031 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.226918    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/63bedd38-6ab7-493d-b94c-b3c288af3f48-host-cni-bin\") pod \"ovnkube-node-rrk77\" (UID: \"63bedd38-6ab7-493d-b94c-b3c288af3f48\") " pod="openshift-ovn-kubernetes/ovnkube-node-rrk77"
Oct 31 00:28:07.227031 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.226987    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/63bedd38-6ab7-493d-b94c-b3c288af3f48-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-rrk77\" (UID: \"63bedd38-6ab7-493d-b94c-b3c288af3f48\") " pod="openshift-ovn-kubernetes/ovnkube-node-rrk77"
Oct 31 00:28:07.227798 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.227037    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/21df46e4-2158-4385-ab9a-4b1790b64b55-etc-sysctl-d\") pod \"tuned-hrsmn\" (UID: \"21df46e4-2158-4385-ab9a-4b1790b64b55\") " pod="openshift-cluster-node-tuning-operator/tuned-hrsmn"
Oct 31 00:28:07.227798 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.227042    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/63bedd38-6ab7-493d-b94c-b3c288af3f48-host-cni-netd\") pod \"ovnkube-node-rrk77\" (UID: \"63bedd38-6ab7-493d-b94c-b3c288af3f48\") " pod="openshift-ovn-kubernetes/ovnkube-node-rrk77"
Oct 31 00:28:07.227798 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.227061    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6926e9b8-7c44-413e-bc7e-63cfff593066-host-var-lib-kubelet\") pod \"multus-d7msb\" (UID: \"6926e9b8-7c44-413e-bc7e-63cfff593066\") " pod="openshift-multus/multus-d7msb"
Oct 31 00:28:07.227798 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.227093    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/63bedd38-6ab7-493d-b94c-b3c288af3f48-host-cni-bin\") pod \"ovnkube-node-rrk77\" (UID: \"63bedd38-6ab7-493d-b94c-b3c288af3f48\") " pod="openshift-ovn-kubernetes/ovnkube-node-rrk77"
Oct 31 00:28:07.227798 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.227094    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/63bedd38-6ab7-493d-b94c-b3c288af3f48-host-run-ovn-kubernetes\") pod \"ovnkube-node-rrk77\" (UID: \"63bedd38-6ab7-493d-b94c-b3c288af3f48\") " pod="openshift-ovn-kubernetes/ovnkube-node-rrk77"
Oct 31 00:28:07.227798 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.227135    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/63bedd38-6ab7-493d-b94c-b3c288af3f48-host-run-ovn-kubernetes\") pod \"ovnkube-node-rrk77\" (UID: \"63bedd38-6ab7-493d-b94c-b3c288af3f48\") " pod="openshift-ovn-kubernetes/ovnkube-node-rrk77"
Oct 31 00:28:07.227798 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.227095    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/63bedd38-6ab7-493d-b94c-b3c288af3f48-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-rrk77\" (UID: \"63bedd38-6ab7-493d-b94c-b3c288af3f48\") " pod="openshift-ovn-kubernetes/ovnkube-node-rrk77"
Oct 31 00:28:07.227798 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.227001    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/21df46e4-2158-4385-ab9a-4b1790b64b55-etc-sysconfig\") pod \"tuned-hrsmn\" (UID: \"21df46e4-2158-4385-ab9a-4b1790b64b55\") " pod="openshift-cluster-node-tuning-operator/tuned-hrsmn"
Oct 31 00:28:07.227798 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.227197    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6926e9b8-7c44-413e-bc7e-63cfff593066-host-var-lib-kubelet\") pod \"multus-d7msb\" (UID: \"6926e9b8-7c44-413e-bc7e-63cfff593066\") " pod="openshift-multus/multus-d7msb"
Oct 31 00:28:07.227798 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.227271    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6926e9b8-7c44-413e-bc7e-63cfff593066-host-run-netns\") pod \"multus-d7msb\" (UID: \"6926e9b8-7c44-413e-bc7e-63cfff593066\") " pod="openshift-multus/multus-d7msb"
Oct 31 00:28:07.227798 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.227278    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/21df46e4-2158-4385-ab9a-4b1790b64b55-etc-sysctl-d\") pod \"tuned-hrsmn\" (UID: \"21df46e4-2158-4385-ab9a-4b1790b64b55\") " pod="openshift-cluster-node-tuning-operator/tuned-hrsmn"
Oct 31 00:28:07.227798 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.227310    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/97410648-b7ae-4cf4-accf-3164c398b525-kubelet-config\") pod \"global-pull-secret-syncer-24sxq\" (UID: \"97410648-b7ae-4cf4-accf-3164c398b525\") " pod="kube-system/global-pull-secret-syncer-24sxq"
Oct 31 00:28:07.227798 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.227396    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/97410648-b7ae-4cf4-accf-3164c398b525-kubelet-config\") pod \"global-pull-secret-syncer-24sxq\" (UID: \"97410648-b7ae-4cf4-accf-3164c398b525\") " pod="kube-system/global-pull-secret-syncer-24sxq"
Oct 31 00:28:07.227798 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.227440    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6926e9b8-7c44-413e-bc7e-63cfff593066-host-run-netns\") pod \"multus-d7msb\" (UID: \"6926e9b8-7c44-413e-bc7e-63cfff593066\") " pod="openshift-multus/multus-d7msb"
Oct 31 00:28:07.227798 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.227492    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/41c4cc4b-b387-4c70-8a27-58255ba632b9-tuning-conf-dir\") pod \"multus-additional-cni-plugins-2t5fx\" (UID: \"41c4cc4b-b387-4c70-8a27-58255ba632b9\") " pod="openshift-multus/multus-additional-cni-plugins-2t5fx"
Oct 31 00:28:07.227798 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.227719    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/63bedd38-6ab7-493d-b94c-b3c288af3f48-host-run-netns\") pod \"ovnkube-node-rrk77\" (UID: \"63bedd38-6ab7-493d-b94c-b3c288af3f48\") " pod="openshift-ovn-kubernetes/ovnkube-node-rrk77"
Oct 31 00:28:07.227798 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.227756    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/21df46e4-2158-4385-ab9a-4b1790b64b55-etc-modprobe-d\") pod \"tuned-hrsmn\" (UID: \"21df46e4-2158-4385-ab9a-4b1790b64b55\") " pod="openshift-cluster-node-tuning-operator/tuned-hrsmn"
Oct 31 00:28:07.227798 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.227802    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6926e9b8-7c44-413e-bc7e-63cfff593066-cnibin\") pod \"multus-d7msb\" (UID: \"6926e9b8-7c44-413e-bc7e-63cfff593066\") " pod="openshift-multus/multus-d7msb"
Oct 31 00:28:07.228588 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.227830    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/63bedd38-6ab7-493d-b94c-b3c288af3f48-host-run-netns\") pod \"ovnkube-node-rrk77\" (UID: \"63bedd38-6ab7-493d-b94c-b3c288af3f48\") " pod="openshift-ovn-kubernetes/ovnkube-node-rrk77"
Oct 31 00:28:07.228588 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.227835    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/97410648-b7ae-4cf4-accf-3164c398b525-dbus\") pod \"global-pull-secret-syncer-24sxq\" (UID: \"97410648-b7ae-4cf4-accf-3164c398b525\") " pod="kube-system/global-pull-secret-syncer-24sxq"
Oct 31 00:28:07.228588 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.227911    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/97410648-b7ae-4cf4-accf-3164c398b525-dbus\") pod \"global-pull-secret-syncer-24sxq\" (UID: \"97410648-b7ae-4cf4-accf-3164c398b525\") " pod="kube-system/global-pull-secret-syncer-24sxq"
Oct 31 00:28:07.228588 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.227916    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/21df46e4-2158-4385-ab9a-4b1790b64b55-etc-systemd\") pod \"tuned-hrsmn\" (UID: \"21df46e4-2158-4385-ab9a-4b1790b64b55\") " pod="openshift-cluster-node-tuning-operator/tuned-hrsmn"
Oct 31 00:28:07.228588 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.227943    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/21df46e4-2158-4385-ab9a-4b1790b64b55-etc-modprobe-d\") pod \"tuned-hrsmn\" (UID: \"21df46e4-2158-4385-ab9a-4b1790b64b55\") " pod="openshift-cluster-node-tuning-operator/tuned-hrsmn"
Oct 31 00:28:07.228588 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.227945    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/6926e9b8-7c44-413e-bc7e-63cfff593066-hostroot\") pod \"multus-d7msb\" (UID: \"6926e9b8-7c44-413e-bc7e-63cfff593066\") " pod="openshift-multus/multus-d7msb"
Oct 31 00:28:07.228588 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.227986    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/6926e9b8-7c44-413e-bc7e-63cfff593066-hostroot\") pod \"multus-d7msb\" (UID: \"6926e9b8-7c44-413e-bc7e-63cfff593066\") " pod="openshift-multus/multus-d7msb"
Oct 31 00:28:07.228588 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.227997    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/6926e9b8-7c44-413e-bc7e-63cfff593066-host-run-multus-certs\") pod \"multus-d7msb\" (UID: \"6926e9b8-7c44-413e-bc7e-63cfff593066\") " pod="openshift-multus/multus-d7msb"
Oct 31 00:28:07.228588 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.228066    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/21df46e4-2158-4385-ab9a-4b1790b64b55-host\") pod \"tuned-hrsmn\" (UID: \"21df46e4-2158-4385-ab9a-4b1790b64b55\") " pod="openshift-cluster-node-tuning-operator/tuned-hrsmn"
Oct 31 00:28:07.228588 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.228068    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6926e9b8-7c44-413e-bc7e-63cfff593066-cnibin\") pod \"multus-d7msb\" (UID: \"6926e9b8-7c44-413e-bc7e-63cfff593066\") " pod="openshift-multus/multus-d7msb"
Oct 31 00:28:07.228588 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.228102    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/85e4b969-79f8-4954-8539-0905bd5db99d-metrics-certs\") pod \"network-metrics-daemon-plnkm\" (UID: \"85e4b969-79f8-4954-8539-0905bd5db99d\") " pod="openshift-multus/network-metrics-daemon-plnkm"
Oct 31 00:28:07.228588 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.228162    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/63bedd38-6ab7-493d-b94c-b3c288af3f48-host-kubelet\") pod \"ovnkube-node-rrk77\" (UID: \"63bedd38-6ab7-493d-b94c-b3c288af3f48\") " pod="openshift-ovn-kubernetes/ovnkube-node-rrk77"
Oct 31 00:28:07.228588 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.228168    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/6926e9b8-7c44-413e-bc7e-63cfff593066-host-run-multus-certs\") pod \"multus-d7msb\" (UID: \"6926e9b8-7c44-413e-bc7e-63cfff593066\") " pod="openshift-multus/multus-d7msb"
Oct 31 00:28:07.228588 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.228248    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/21df46e4-2158-4385-ab9a-4b1790b64b55-host\") pod \"tuned-hrsmn\" (UID: \"21df46e4-2158-4385-ab9a-4b1790b64b55\") " pod="openshift-cluster-node-tuning-operator/tuned-hrsmn"
Oct 31 00:28:07.228588 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.227833    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/41c4cc4b-b387-4c70-8a27-58255ba632b9-tuning-conf-dir\") pod \"multus-additional-cni-plugins-2t5fx\" (UID: \"41c4cc4b-b387-4c70-8a27-58255ba632b9\") " pod="openshift-multus/multus-additional-cni-plugins-2t5fx"
Oct 31 00:28:07.228588 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.228270    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/21df46e4-2158-4385-ab9a-4b1790b64b55-etc-systemd\") pod \"tuned-hrsmn\" (UID: \"21df46e4-2158-4385-ab9a-4b1790b64b55\") " pod="openshift-cluster-node-tuning-operator/tuned-hrsmn"
Oct 31 00:28:07.228588 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.228401    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/63bedd38-6ab7-493d-b94c-b3c288af3f48-systemd-units\") pod \"ovnkube-node-rrk77\" (UID: \"63bedd38-6ab7-493d-b94c-b3c288af3f48\") " pod="openshift-ovn-kubernetes/ovnkube-node-rrk77"
Oct 31 00:28:07.229502 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.228612    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/21df46e4-2158-4385-ab9a-4b1790b64b55-etc-sysctl-conf\") pod \"tuned-hrsmn\" (UID: \"21df46e4-2158-4385-ab9a-4b1790b64b55\") " pod="openshift-cluster-node-tuning-operator/tuned-hrsmn"
Oct 31 00:28:07.229502 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.228686    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:28:07.229502 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.228768    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/21df46e4-2158-4385-ab9a-4b1790b64b55-lib-modules\") pod \"tuned-hrsmn\" (UID: \"21df46e4-2158-4385-ab9a-4b1790b64b55\") " pod="openshift-cluster-node-tuning-operator/tuned-hrsmn"
Oct 31 00:28:07.229502 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.228809    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/1a72f876-1157-4447-845c-923cdfd5aa5e-kubelet-dir\") pod \"aws-ebs-csi-driver-node-k6fgp\" (UID: \"1a72f876-1157-4447-845c-923cdfd5aa5e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k6fgp"
Oct 31 00:28:07.229502 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.228812    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/63bedd38-6ab7-493d-b94c-b3c288af3f48-host-kubelet\") pod \"ovnkube-node-rrk77\" (UID: \"63bedd38-6ab7-493d-b94c-b3c288af3f48\") " pod="openshift-ovn-kubernetes/ovnkube-node-rrk77"
Oct 31 00:28:07.229502 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.228730    4284 swap_util.go:78] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Oct 31 00:28:07.229502 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.228871    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/61c60450-eb50-4181-8ab7-e69e864ffb51-host\") pod \"node-ca-qchks\" (UID: \"61c60450-eb50-4181-8ab7-e69e864ffb51\") " pod="openshift-image-registry/node-ca-qchks"
Oct 31 00:28:07.229502 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.228899    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/1a72f876-1157-4447-845c-923cdfd5aa5e-kubelet-dir\") pod \"aws-ebs-csi-driver-node-k6fgp\" (UID: \"1a72f876-1157-4447-845c-923cdfd5aa5e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k6fgp"
Oct 31 00:28:07.229502 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.228907    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/63bedd38-6ab7-493d-b94c-b3c288af3f48-run-openvswitch\") pod \"ovnkube-node-rrk77\" (UID: \"63bedd38-6ab7-493d-b94c-b3c288af3f48\") " pod="openshift-ovn-kubernetes/ovnkube-node-rrk77"
Oct 31 00:28:07.229502 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.228931    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/21df46e4-2158-4385-ab9a-4b1790b64b55-etc-sysctl-conf\") pod \"tuned-hrsmn\" (UID: \"21df46e4-2158-4385-ab9a-4b1790b64b55\") " pod="openshift-cluster-node-tuning-operator/tuned-hrsmn"
Oct 31 00:28:07.229502 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.228935    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/21df46e4-2158-4385-ab9a-4b1790b64b55-run\") pod \"tuned-hrsmn\" (UID: \"21df46e4-2158-4385-ab9a-4b1790b64b55\") " pod="openshift-cluster-node-tuning-operator/tuned-hrsmn"
Oct 31 00:28:07.229502 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.228964    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/63bedd38-6ab7-493d-b94c-b3c288af3f48-systemd-units\") pod \"ovnkube-node-rrk77\" (UID: \"63bedd38-6ab7-493d-b94c-b3c288af3f48\") " pod="openshift-ovn-kubernetes/ovnkube-node-rrk77"
Oct 31 00:28:07.229502 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.228981    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/6926e9b8-7c44-413e-bc7e-63cfff593066-host-run-k8s-cni-cncf-io\") pod \"multus-d7msb\" (UID: \"6926e9b8-7c44-413e-bc7e-63cfff593066\") " pod="openshift-multus/multus-d7msb"
Oct 31 00:28:07.229502 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.228985    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/21df46e4-2158-4385-ab9a-4b1790b64b55-run\") pod \"tuned-hrsmn\" (UID: \"21df46e4-2158-4385-ab9a-4b1790b64b55\") " pod="openshift-cluster-node-tuning-operator/tuned-hrsmn"
Oct 31 00:28:07.229502 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.228992    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/21df46e4-2158-4385-ab9a-4b1790b64b55-lib-modules\") pod \"tuned-hrsmn\" (UID: \"21df46e4-2158-4385-ab9a-4b1790b64b55\") " pod="openshift-cluster-node-tuning-operator/tuned-hrsmn"
Oct 31 00:28:07.229502 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.229008    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/1a72f876-1157-4447-845c-923cdfd5aa5e-socket-dir\") pod \"aws-ebs-csi-driver-node-k6fgp\" (UID: \"1a72f876-1157-4447-845c-923cdfd5aa5e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k6fgp"
Oct 31 00:28:07.229502 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.229014    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/61c60450-eb50-4181-8ab7-e69e864ffb51-host\") pod \"node-ca-qchks\" (UID: \"61c60450-eb50-4181-8ab7-e69e864ffb51\") " pod="openshift-image-registry/node-ca-qchks"
Oct 31 00:28:07.229502 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.229037    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/63bedd38-6ab7-493d-b94c-b3c288af3f48-run-openvswitch\") pod \"ovnkube-node-rrk77\" (UID: \"63bedd38-6ab7-493d-b94c-b3c288af3f48\") " pod="openshift-ovn-kubernetes/ovnkube-node-rrk77"
Oct 31 00:28:07.229502 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.229047    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/6926e9b8-7c44-413e-bc7e-63cfff593066-host-run-k8s-cni-cncf-io\") pod \"multus-d7msb\" (UID: \"6926e9b8-7c44-413e-bc7e-63cfff593066\") " pod="openshift-multus/multus-d7msb"
Oct 31 00:28:07.229502 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.229040    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/63bedd38-6ab7-493d-b94c-b3c288af3f48-run-ovn\") pod \"ovnkube-node-rrk77\" (UID: \"63bedd38-6ab7-493d-b94c-b3c288af3f48\") " pod="openshift-ovn-kubernetes/ovnkube-node-rrk77"
Oct 31 00:28:07.229502 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.229076    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/1a72f876-1157-4447-845c-923cdfd5aa5e-socket-dir\") pod \"aws-ebs-csi-driver-node-k6fgp\" (UID: \"1a72f876-1157-4447-845c-923cdfd5aa5e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k6fgp"
Oct 31 00:28:07.229502 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.229082    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/63bedd38-6ab7-493d-b94c-b3c288af3f48-run-ovn\") pod \"ovnkube-node-rrk77\" (UID: \"63bedd38-6ab7-493d-b94c-b3c288af3f48\") " pod="openshift-ovn-kubernetes/ovnkube-node-rrk77"
Oct 31 00:28:07.229502 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.229112    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/21df46e4-2158-4385-ab9a-4b1790b64b55-etc-kubernetes\") pod \"tuned-hrsmn\" (UID: \"21df46e4-2158-4385-ab9a-4b1790b64b55\") " pod="openshift-cluster-node-tuning-operator/tuned-hrsmn"
Oct 31 00:28:07.229502 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.229183    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/21df46e4-2158-4385-ab9a-4b1790b64b55-etc-kubernetes\") pod \"tuned-hrsmn\" (UID: \"21df46e4-2158-4385-ab9a-4b1790b64b55\") " pod="openshift-cluster-node-tuning-operator/tuned-hrsmn"
Oct 31 00:28:07.229502 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.229257    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6926e9b8-7c44-413e-bc7e-63cfff593066-etc-kubernetes\") pod \"multus-d7msb\" (UID: \"6926e9b8-7c44-413e-bc7e-63cfff593066\") " pod="openshift-multus/multus-d7msb"
Oct 31 00:28:07.229502 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.229316    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/63bedd38-6ab7-493d-b94c-b3c288af3f48-host-slash\") pod \"ovnkube-node-rrk77\" (UID: \"63bedd38-6ab7-493d-b94c-b3c288af3f48\") " pod="openshift-ovn-kubernetes/ovnkube-node-rrk77"
Oct 31 00:28:07.229502 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.229363    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/63bedd38-6ab7-493d-b94c-b3c288af3f48-node-log\") pod \"ovnkube-node-rrk77\" (UID: \"63bedd38-6ab7-493d-b94c-b3c288af3f48\") " pod="openshift-ovn-kubernetes/ovnkube-node-rrk77"
Oct 31 00:28:07.229502 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.229359    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6926e9b8-7c44-413e-bc7e-63cfff593066-etc-kubernetes\") pod \"multus-d7msb\" (UID: \"6926e9b8-7c44-413e-bc7e-63cfff593066\") " pod="openshift-multus/multus-d7msb"
Oct 31 00:28:07.229502 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.229493    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/63bedd38-6ab7-493d-b94c-b3c288af3f48-host-slash\") pod \"ovnkube-node-rrk77\" (UID: \"63bedd38-6ab7-493d-b94c-b3c288af3f48\") " pod="openshift-ovn-kubernetes/ovnkube-node-rrk77"
Oct 31 00:28:07.229502 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.229501    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6926e9b8-7c44-413e-bc7e-63cfff593066-host-var-lib-cni-bin\") pod \"multus-d7msb\" (UID: \"6926e9b8-7c44-413e-bc7e-63cfff593066\") " pod="openshift-multus/multus-d7msb"
Oct 31 00:28:07.230897 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.229646    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6926e9b8-7c44-413e-bc7e-63cfff593066-host-var-lib-cni-bin\") pod \"multus-d7msb\" (UID: \"6926e9b8-7c44-413e-bc7e-63cfff593066\") " pod="openshift-multus/multus-d7msb"
Oct 31 00:28:07.230897 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.229898    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/63bedd38-6ab7-493d-b94c-b3c288af3f48-node-log\") pod \"ovnkube-node-rrk77\" (UID: \"63bedd38-6ab7-493d-b94c-b3c288af3f48\") " pod="openshift-ovn-kubernetes/ovnkube-node-rrk77"
Oct 31 00:28:07.231472 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.231454    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/85e4b969-79f8-4954-8539-0905bd5db99d-metrics-certs\") pod \"network-metrics-daemon-plnkm\" (UID: \"85e4b969-79f8-4954-8539-0905bd5db99d\") " pod="openshift-multus/network-metrics-daemon-plnkm"
Oct 31 00:28:07.315384 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.315353    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-plnkm"
Oct 31 00:28:07.417440 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.417350    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:28:07.672968 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.672877    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-plnkm"]
Oct 31 00:28:07.719312 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:07.719271    4284 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod85e4b969_79f8_4954_8539_0905bd5db99d.slice/crio-0f1f6b7ec36bb1c4125d92c7e28a49920f2e99f3dce643421f8a6219b410f006 WatchSource:0}: Error finding container 0f1f6b7ec36bb1c4125d92c7e28a49920f2e99f3dce643421f8a6219b410f006: Status 404 returned error can't find the container with id 0f1f6b7ec36bb1c4125d92c7e28a49920f2e99f3dce643421f8a6219b410f006
Oct 31 00:28:07.721888 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.721867    4284 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 31 00:28:07.816123 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.816079    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:28:07.967276 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.967183    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:28:07.990457 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:07.990430    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-plnkm" event={"ID":"85e4b969-79f8-4954-8539-0905bd5db99d","Type":"ContainerStarted","Data":"0f1f6b7ec36bb1c4125d92c7e28a49920f2e99f3dce643421f8a6219b410f006"}
Oct 31 00:28:08.112298 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:08.112261    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:28:08.244520 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:08.244447    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-rrk77"
Oct 31 00:28:08.261631 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:08.261602    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:28:08.407703 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:08.407667    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:28:08.550347 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:08.550260    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:28:08.699714 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:08.699683    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:28:08.839830 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:08.839733    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:28:09.529458 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:09.529429    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:28:09.820851 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:09.820806    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:28:09.998475 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:09.998443    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-plnkm" event={"ID":"85e4b969-79f8-4954-8539-0905bd5db99d","Type":"ContainerStarted","Data":"6c1fb46e8424febc9c295b0e8c5fcf92315c8ee29c0dd663ee125e09efd5e111"}
Oct 31 00:28:09.998659 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:09.998490    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-plnkm" event={"ID":"85e4b969-79f8-4954-8539-0905bd5db99d","Type":"ContainerStarted","Data":"29fe5eacb822b15ad5e00bae041af4f88c452bfa086a73c207466251c72f972a"}
Oct 31 00:28:12.917139 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:12.917092    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-jktkx"
Oct 31 00:28:15.000133 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:15.000104    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-rrk77"
Oct 31 00:28:15.000588 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:15.000359    4284 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Oct 31 00:28:15.043457 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:15.043418    4284 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-66df5d496b-wz5p4"]
Oct 31 00:28:15.045385 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:15.045367    4284 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-bn67m"]
Oct 31 00:28:15.045503 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:15.045485    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-wz5p4"
Oct 31 00:28:15.047164 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:15.047147    4284 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-6dd789c8b7-fd796"]
Oct 31 00:28:15.047277 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:15.047261    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-bn67m"
Oct 31 00:28:15.049331 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:15.049315    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-6dd789c8b7-fd796"
Oct 31 00:28:15.072479 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:15.072455    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mvqf8\" (UniqueName: \"kubernetes.io/projected/308c723a-b97c-4acd-a65e-71dd5b389723-kube-api-access-mvqf8\") pod \"downloads-6dd789c8b7-fd796\" (UID: \"308c723a-b97c-4acd-a65e-71dd5b389723\") " pod="openshift-console/downloads-6dd789c8b7-fd796"
Oct 31 00:28:15.072680 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:15.072663    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/308c723a-b97c-4acd-a65e-71dd5b389723-tmp\") pod \"downloads-6dd789c8b7-fd796\" (UID: \"308c723a-b97c-4acd-a65e-71dd5b389723\") " pod="openshift-console/downloads-6dd789c8b7-fd796"
Oct 31 00:28:15.072837 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:15.072820    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/db5d4efa-ed50-44e9-84f2-851579e5c49e-cert\") pod \"ingress-canary-bn67m\" (UID: \"db5d4efa-ed50-44e9-84f2-851579e5c49e\") " pod="openshift-ingress-canary/ingress-canary-bn67m"
Oct 31 00:28:15.072908 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:15.072848    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rthdh\" (UniqueName: \"kubernetes.io/projected/1eccd43e-037f-4de4-a2de-c0961b97865e-kube-api-access-rthdh\") pod \"migrator-66df5d496b-wz5p4\" (UID: \"1eccd43e-037f-4de4-a2de-c0961b97865e\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-wz5p4"
Oct 31 00:28:15.072908 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:15.072870    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2vfz6\" (UniqueName: \"kubernetes.io/projected/db5d4efa-ed50-44e9-84f2-851579e5c49e-kube-api-access-2vfz6\") pod \"ingress-canary-bn67m\" (UID: \"db5d4efa-ed50-44e9-84f2-851579e5c49e\") " pod="openshift-ingress-canary/ingress-canary-bn67m"
Oct 31 00:28:15.091826 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:15.091805    4284 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-gnstv"]
Oct 31 00:28:15.094036 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:15.094022    4284 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-fd8sp"]
Oct 31 00:28:15.094147 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:15.094133    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-gnstv"
Oct 31 00:28:15.096026 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:15.096010    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-fd8sp"
Oct 31 00:28:15.102707 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:15.102688    4284 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Oct 31 00:28:15.138731 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:15.138699    4284 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Oct 31 00:28:15.138731 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:15.138699    4284 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-85rmc\""
Oct 31 00:28:15.138946 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:15.138781    4284 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Oct 31 00:28:15.138946 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:15.138810    4284 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\""
Oct 31 00:28:15.138946 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:15.138825    4284 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\""
Oct 31 00:28:15.138946 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:15.138845    4284 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Oct 31 00:28:15.138946 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:15.138849    4284 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Oct 31 00:28:15.138946 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:15.138901    4284 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"ingress-canary-dockercfg-hg2s2\""
Oct 31 00:28:15.139526 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:15.139500    4284 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-g6vkf"]
Oct 31 00:28:15.141657 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:15.141639    4284 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-85f776cbc8-lr9wd"]
Oct 31 00:28:15.141783 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:15.141768    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-g6vkf"
Oct 31 00:28:15.145382 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:15.145361    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-66df5d496b-wz5p4"]
Oct 31 00:28:15.145495 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:15.145403    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-bn67m"]
Oct 31 00:28:15.145495 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:15.145455    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-85f776cbc8-lr9wd"
Oct 31 00:28:15.149153 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:15.149132    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-6dd789c8b7-fd796"]
Oct 31 00:28:15.151113 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:15.151022    4284 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Oct 31 00:28:15.151631 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:15.151599    4284 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Oct 31 00:28:15.151810 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:15.151772    4284 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Oct 31 00:28:15.151997 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:15.151979    4284 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Oct 31 00:28:15.173738 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:15.173716    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mvqf8\" (UniqueName: \"kubernetes.io/projected/308c723a-b97c-4acd-a65e-71dd5b389723-kube-api-access-mvqf8\") pod \"downloads-6dd789c8b7-fd796\" (UID: \"308c723a-b97c-4acd-a65e-71dd5b389723\") " pod="openshift-console/downloads-6dd789c8b7-fd796"
Oct 31 00:28:15.173860 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:15.173747    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g9427\" (UniqueName: \"kubernetes.io/projected/18948a7b-85a1-4ab0-b61b-ee42cf31b815-kube-api-access-g9427\") pod \"image-registry-85f776cbc8-lr9wd\" (UID: \"18948a7b-85a1-4ab0-b61b-ee42cf31b815\") " pod="openshift-image-registry/image-registry-85f776cbc8-lr9wd"
Oct 31 00:28:15.173860 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:15.173774    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/18948a7b-85a1-4ab0-b61b-ee42cf31b815-trusted-ca\") pod \"image-registry-85f776cbc8-lr9wd\" (UID: \"18948a7b-85a1-4ab0-b61b-ee42cf31b815\") " pod="openshift-image-registry/image-registry-85f776cbc8-lr9wd"
Oct 31 00:28:15.173860 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:15.173795    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/18948a7b-85a1-4ab0-b61b-ee42cf31b815-installation-pull-secrets\") pod \"image-registry-85f776cbc8-lr9wd\" (UID: \"18948a7b-85a1-4ab0-b61b-ee42cf31b815\") " pod="openshift-image-registry/image-registry-85f776cbc8-lr9wd"
Oct 31 00:28:15.174027 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:15.173914    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d59dc7c0-7b09-48d7-9459-0af53d2159ad-tmp-dir\") pod \"dns-default-gnstv\" (UID: \"d59dc7c0-7b09-48d7-9459-0af53d2159ad\") " pod="openshift-dns/dns-default-gnstv"
Oct 31 00:28:15.174027 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:15.173943    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zww5h\" (UniqueName: \"kubernetes.io/projected/d59dc7c0-7b09-48d7-9459-0af53d2159ad-kube-api-access-zww5h\") pod \"dns-default-gnstv\" (UID: \"d59dc7c0-7b09-48d7-9459-0af53d2159ad\") " pod="openshift-dns/dns-default-gnstv"
Oct 31 00:28:15.174027 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:15.173973    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/18948a7b-85a1-4ab0-b61b-ee42cf31b815-image-registry-private-configuration\") pod \"image-registry-85f776cbc8-lr9wd\" (UID: \"18948a7b-85a1-4ab0-b61b-ee42cf31b815\") " pod="openshift-image-registry/image-registry-85f776cbc8-lr9wd"
Oct 31 00:28:15.174027 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:15.174000    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/690b2e89-76c8-42f1-a5cd-07567475bbaa-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-fd8sp\" (UID: \"690b2e89-76c8-42f1-a5cd-07567475bbaa\") " pod="openshift-insights/insights-runtime-extractor-fd8sp"
Oct 31 00:28:15.174329 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:15.174022    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/4a4ec287-fc28-413d-83fa-d9a05ddedba3-tls-certificates\") pod \"prometheus-operator-admission-webhook-6489d8bf89-g6vkf\" (UID: \"4a4ec287-fc28-413d-83fa-d9a05ddedba3\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-g6vkf"
Oct 31 00:28:15.174329 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:15.174061    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/308c723a-b97c-4acd-a65e-71dd5b389723-tmp\") pod \"downloads-6dd789c8b7-fd796\" (UID: \"308c723a-b97c-4acd-a65e-71dd5b389723\") " pod="openshift-console/downloads-6dd789c8b7-fd796"
Oct 31 00:28:15.174329 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:15.174176    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/690b2e89-76c8-42f1-a5cd-07567475bbaa-crio-socket\") pod \"insights-runtime-extractor-fd8sp\" (UID: \"690b2e89-76c8-42f1-a5cd-07567475bbaa\") " pod="openshift-insights/insights-runtime-extractor-fd8sp"
Oct 31 00:28:15.174329 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:15.174276    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mwlnh\" (UniqueName: \"kubernetes.io/projected/690b2e89-76c8-42f1-a5cd-07567475bbaa-kube-api-access-mwlnh\") pod \"insights-runtime-extractor-fd8sp\" (UID: \"690b2e89-76c8-42f1-a5cd-07567475bbaa\") " pod="openshift-insights/insights-runtime-extractor-fd8sp"
Oct 31 00:28:15.174329 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:15.174310    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/18948a7b-85a1-4ab0-b61b-ee42cf31b815-ca-trust-extracted\") pod \"image-registry-85f776cbc8-lr9wd\" (UID: \"18948a7b-85a1-4ab0-b61b-ee42cf31b815\") " pod="openshift-image-registry/image-registry-85f776cbc8-lr9wd"
Oct 31 00:28:15.174574 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:15.174386    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/db5d4efa-ed50-44e9-84f2-851579e5c49e-cert\") pod \"ingress-canary-bn67m\" (UID: \"db5d4efa-ed50-44e9-84f2-851579e5c49e\") " pod="openshift-ingress-canary/ingress-canary-bn67m"
Oct 31 00:28:15.174574 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:15.174418    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rthdh\" (UniqueName: \"kubernetes.io/projected/1eccd43e-037f-4de4-a2de-c0961b97865e-kube-api-access-rthdh\") pod \"migrator-66df5d496b-wz5p4\" (UID: \"1eccd43e-037f-4de4-a2de-c0961b97865e\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-wz5p4"
Oct 31 00:28:15.174574 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:15.174438    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2vfz6\" (UniqueName: \"kubernetes.io/projected/db5d4efa-ed50-44e9-84f2-851579e5c49e-kube-api-access-2vfz6\") pod \"ingress-canary-bn67m\" (UID: \"db5d4efa-ed50-44e9-84f2-851579e5c49e\") " pod="openshift-ingress-canary/ingress-canary-bn67m"
Oct 31 00:28:15.174574 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:15.174454    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/d59dc7c0-7b09-48d7-9459-0af53d2159ad-config-volume\") pod \"dns-default-gnstv\" (UID: \"d59dc7c0-7b09-48d7-9459-0af53d2159ad\") " pod="openshift-dns/dns-default-gnstv"
Oct 31 00:28:15.174574 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:15.174471    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/18948a7b-85a1-4ab0-b61b-ee42cf31b815-bound-sa-token\") pod \"image-registry-85f776cbc8-lr9wd\" (UID: \"18948a7b-85a1-4ab0-b61b-ee42cf31b815\") " pod="openshift-image-registry/image-registry-85f776cbc8-lr9wd"
Oct 31 00:28:15.174574 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:15.174497    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/18948a7b-85a1-4ab0-b61b-ee42cf31b815-registry-tls\") pod \"image-registry-85f776cbc8-lr9wd\" (UID: \"18948a7b-85a1-4ab0-b61b-ee42cf31b815\") " pod="openshift-image-registry/image-registry-85f776cbc8-lr9wd"
Oct 31 00:28:15.174574 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:15.174543    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/18948a7b-85a1-4ab0-b61b-ee42cf31b815-registry-certificates\") pod \"image-registry-85f776cbc8-lr9wd\" (UID: \"18948a7b-85a1-4ab0-b61b-ee42cf31b815\") " pod="openshift-image-registry/image-registry-85f776cbc8-lr9wd"
Oct 31 00:28:15.174826 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:15.174605    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/690b2e89-76c8-42f1-a5cd-07567475bbaa-data-volume\") pod \"insights-runtime-extractor-fd8sp\" (UID: \"690b2e89-76c8-42f1-a5cd-07567475bbaa\") " pod="openshift-insights/insights-runtime-extractor-fd8sp"
Oct 31 00:28:15.174826 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:15.174645    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/690b2e89-76c8-42f1-a5cd-07567475bbaa-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-fd8sp\" (UID: \"690b2e89-76c8-42f1-a5cd-07567475bbaa\") " pod="openshift-insights/insights-runtime-extractor-fd8sp"
Oct 31 00:28:15.174826 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:15.174681    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d59dc7c0-7b09-48d7-9459-0af53d2159ad-metrics-tls\") pod \"dns-default-gnstv\" (UID: \"d59dc7c0-7b09-48d7-9459-0af53d2159ad\") " pod="openshift-dns/dns-default-gnstv"
Oct 31 00:28:15.174826 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:15.174684    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/308c723a-b97c-4acd-a65e-71dd5b389723-tmp\") pod \"downloads-6dd789c8b7-fd796\" (UID: \"308c723a-b97c-4acd-a65e-71dd5b389723\") " pod="openshift-console/downloads-6dd789c8b7-fd796"
Oct 31 00:28:15.177008 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:15.176987    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/db5d4efa-ed50-44e9-84f2-851579e5c49e-cert\") pod \"ingress-canary-bn67m\" (UID: \"db5d4efa-ed50-44e9-84f2-851579e5c49e\") " pod="openshift-ingress-canary/ingress-canary-bn67m"
Oct 31 00:28:15.181126 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:15.181103    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:28:15.189487 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:15.189465    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-fd8sp"]
Oct 31 00:28:15.190006 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:15.189989    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-gnstv"]
Oct 31 00:28:15.197529 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:15.197508    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-85f776cbc8-lr9wd"]
Oct 31 00:28:15.234563 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:15.234531    4284 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-9qgw5\""
Oct 31 00:28:15.235361 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:15.235341    4284 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Oct 31 00:28:15.236512 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:15.236494    4284 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Oct 31 00:28:15.236890 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:15.236873    4284 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"downloads-dockercfg-tw2cs\""
Oct 31 00:28:15.236968 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:15.236899    4284 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-52f7q\""
Oct 31 00:28:15.236968 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:15.236913    4284 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Oct 31 00:28:15.236968 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:15.236903    4284 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-lpmz8\""
Oct 31 00:28:15.236968 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:15.236919    4284 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Oct 31 00:28:15.236968 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:15.236919    4284 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Oct 31 00:28:15.236968 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:15.236956    4284 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\""
Oct 31 00:28:15.237148 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:15.236982    4284 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-trjc8\""
Oct 31 00:28:15.245234 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:15.245195    4284 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Oct 31 00:28:15.245322 ip-10-0-132-173 kubenswrapper[4284]: E1031 00:28:15.245271    4284 projected.go:291] Couldn't get configMap openshift-kube-storage-version-migrator/openshift-service-ca.crt: configmap "openshift-service-ca.crt" not found
Oct 31 00:28:15.245322 ip-10-0-132-173 kubenswrapper[4284]: E1031 00:28:15.245289    4284 projected.go:196] Error preparing data for projected volume kube-api-access-rthdh for pod openshift-kube-storage-version-migrator/migrator-66df5d496b-wz5p4: configmap "openshift-service-ca.crt" not found
Oct 31 00:28:15.245440 ip-10-0-132-173 kubenswrapper[4284]: E1031 00:28:15.245367    4284 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1eccd43e-037f-4de4-a2de-c0961b97865e-kube-api-access-rthdh podName:1eccd43e-037f-4de4-a2de-c0961b97865e nodeName:}" failed. No retries permitted until 2026-10-31 00:28:15.745344363 +0000 UTC m=+9.890351279 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-rthdh" (UniqueName: "kubernetes.io/projected/1eccd43e-037f-4de4-a2de-c0961b97865e-kube-api-access-rthdh") pod "migrator-66df5d496b-wz5p4" (UID: "1eccd43e-037f-4de4-a2de-c0961b97865e") : configmap "openshift-service-ca.crt" not found
Oct 31 00:28:15.247698 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:15.247654    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2vfz6\" (UniqueName: \"kubernetes.io/projected/db5d4efa-ed50-44e9-84f2-851579e5c49e-kube-api-access-2vfz6\") pod \"ingress-canary-bn67m\" (UID: \"db5d4efa-ed50-44e9-84f2-851579e5c49e\") " pod="openshift-ingress-canary/ingress-canary-bn67m"
Oct 31 00:28:15.247916 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:15.247892    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mvqf8\" (UniqueName: \"kubernetes.io/projected/308c723a-b97c-4acd-a65e-71dd5b389723-kube-api-access-mvqf8\") pod \"downloads-6dd789c8b7-fd796\" (UID: \"308c723a-b97c-4acd-a65e-71dd5b389723\") " pod="openshift-console/downloads-6dd789c8b7-fd796"
Oct 31 00:28:15.248944 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:15.248888    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-g6vkf"]
Oct 31 00:28:15.275157 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:15.275095    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/18948a7b-85a1-4ab0-b61b-ee42cf31b815-trusted-ca\") pod \"image-registry-85f776cbc8-lr9wd\" (UID: \"18948a7b-85a1-4ab0-b61b-ee42cf31b815\") " pod="openshift-image-registry/image-registry-85f776cbc8-lr9wd"
Oct 31 00:28:15.275157 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:15.275132    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/18948a7b-85a1-4ab0-b61b-ee42cf31b815-installation-pull-secrets\") pod \"image-registry-85f776cbc8-lr9wd\" (UID: \"18948a7b-85a1-4ab0-b61b-ee42cf31b815\") " pod="openshift-image-registry/image-registry-85f776cbc8-lr9wd"
Oct 31 00:28:15.275157 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:15.275151    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d59dc7c0-7b09-48d7-9459-0af53d2159ad-tmp-dir\") pod \"dns-default-gnstv\" (UID: \"d59dc7c0-7b09-48d7-9459-0af53d2159ad\") " pod="openshift-dns/dns-default-gnstv"
Oct 31 00:28:15.275392 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:15.275200    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zww5h\" (UniqueName: \"kubernetes.io/projected/d59dc7c0-7b09-48d7-9459-0af53d2159ad-kube-api-access-zww5h\") pod \"dns-default-gnstv\" (UID: \"d59dc7c0-7b09-48d7-9459-0af53d2159ad\") " pod="openshift-dns/dns-default-gnstv"
Oct 31 00:28:15.275392 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:15.275253    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/18948a7b-85a1-4ab0-b61b-ee42cf31b815-image-registry-private-configuration\") pod \"image-registry-85f776cbc8-lr9wd\" (UID: \"18948a7b-85a1-4ab0-b61b-ee42cf31b815\") " pod="openshift-image-registry/image-registry-85f776cbc8-lr9wd"
Oct 31 00:28:15.275392 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:15.275276    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/690b2e89-76c8-42f1-a5cd-07567475bbaa-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-fd8sp\" (UID: \"690b2e89-76c8-42f1-a5cd-07567475bbaa\") " pod="openshift-insights/insights-runtime-extractor-fd8sp"
Oct 31 00:28:15.275392 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:15.275298    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/4a4ec287-fc28-413d-83fa-d9a05ddedba3-tls-certificates\") pod \"prometheus-operator-admission-webhook-6489d8bf89-g6vkf\" (UID: \"4a4ec287-fc28-413d-83fa-d9a05ddedba3\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-g6vkf"
Oct 31 00:28:15.275392 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:15.275321    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/690b2e89-76c8-42f1-a5cd-07567475bbaa-crio-socket\") pod \"insights-runtime-extractor-fd8sp\" (UID: \"690b2e89-76c8-42f1-a5cd-07567475bbaa\") " pod="openshift-insights/insights-runtime-extractor-fd8sp"
Oct 31 00:28:15.275392 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:15.275337    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mwlnh\" (UniqueName: \"kubernetes.io/projected/690b2e89-76c8-42f1-a5cd-07567475bbaa-kube-api-access-mwlnh\") pod \"insights-runtime-extractor-fd8sp\" (UID: \"690b2e89-76c8-42f1-a5cd-07567475bbaa\") " pod="openshift-insights/insights-runtime-extractor-fd8sp"
Oct 31 00:28:15.275392 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:15.275354    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/18948a7b-85a1-4ab0-b61b-ee42cf31b815-ca-trust-extracted\") pod \"image-registry-85f776cbc8-lr9wd\" (UID: \"18948a7b-85a1-4ab0-b61b-ee42cf31b815\") " pod="openshift-image-registry/image-registry-85f776cbc8-lr9wd"
Oct 31 00:28:15.275728 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:15.275432    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/690b2e89-76c8-42f1-a5cd-07567475bbaa-crio-socket\") pod \"insights-runtime-extractor-fd8sp\" (UID: \"690b2e89-76c8-42f1-a5cd-07567475bbaa\") " pod="openshift-insights/insights-runtime-extractor-fd8sp"
Oct 31 00:28:15.275728 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:15.275561    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/d59dc7c0-7b09-48d7-9459-0af53d2159ad-config-volume\") pod \"dns-default-gnstv\" (UID: \"d59dc7c0-7b09-48d7-9459-0af53d2159ad\") " pod="openshift-dns/dns-default-gnstv"
Oct 31 00:28:15.275728 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:15.275592    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/18948a7b-85a1-4ab0-b61b-ee42cf31b815-bound-sa-token\") pod \"image-registry-85f776cbc8-lr9wd\" (UID: \"18948a7b-85a1-4ab0-b61b-ee42cf31b815\") " pod="openshift-image-registry/image-registry-85f776cbc8-lr9wd"
Oct 31 00:28:15.275728 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:15.275625    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/18948a7b-85a1-4ab0-b61b-ee42cf31b815-registry-tls\") pod \"image-registry-85f776cbc8-lr9wd\" (UID: \"18948a7b-85a1-4ab0-b61b-ee42cf31b815\") " pod="openshift-image-registry/image-registry-85f776cbc8-lr9wd"
Oct 31 00:28:15.275728 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:15.275650    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/18948a7b-85a1-4ab0-b61b-ee42cf31b815-registry-certificates\") pod \"image-registry-85f776cbc8-lr9wd\" (UID: \"18948a7b-85a1-4ab0-b61b-ee42cf31b815\") " pod="openshift-image-registry/image-registry-85f776cbc8-lr9wd"
Oct 31 00:28:15.275728 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:15.275668    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d59dc7c0-7b09-48d7-9459-0af53d2159ad-tmp-dir\") pod \"dns-default-gnstv\" (UID: \"d59dc7c0-7b09-48d7-9459-0af53d2159ad\") " pod="openshift-dns/dns-default-gnstv"
Oct 31 00:28:15.275728 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:15.275682    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/690b2e89-76c8-42f1-a5cd-07567475bbaa-data-volume\") pod \"insights-runtime-extractor-fd8sp\" (UID: \"690b2e89-76c8-42f1-a5cd-07567475bbaa\") " pod="openshift-insights/insights-runtime-extractor-fd8sp"
Oct 31 00:28:15.276052 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:15.275730    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/690b2e89-76c8-42f1-a5cd-07567475bbaa-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-fd8sp\" (UID: \"690b2e89-76c8-42f1-a5cd-07567475bbaa\") " pod="openshift-insights/insights-runtime-extractor-fd8sp"
Oct 31 00:28:15.276052 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:15.275756    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d59dc7c0-7b09-48d7-9459-0af53d2159ad-metrics-tls\") pod \"dns-default-gnstv\" (UID: \"d59dc7c0-7b09-48d7-9459-0af53d2159ad\") " pod="openshift-dns/dns-default-gnstv"
Oct 31 00:28:15.276052 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:15.275786    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g9427\" (UniqueName: \"kubernetes.io/projected/18948a7b-85a1-4ab0-b61b-ee42cf31b815-kube-api-access-g9427\") pod \"image-registry-85f776cbc8-lr9wd\" (UID: \"18948a7b-85a1-4ab0-b61b-ee42cf31b815\") " pod="openshift-image-registry/image-registry-85f776cbc8-lr9wd"
Oct 31 00:28:15.276052 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:15.276009    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/18948a7b-85a1-4ab0-b61b-ee42cf31b815-ca-trust-extracted\") pod \"image-registry-85f776cbc8-lr9wd\" (UID: \"18948a7b-85a1-4ab0-b61b-ee42cf31b815\") " pod="openshift-image-registry/image-registry-85f776cbc8-lr9wd"
Oct 31 00:28:15.276285 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:15.276263    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/d59dc7c0-7b09-48d7-9459-0af53d2159ad-config-volume\") pod \"dns-default-gnstv\" (UID: \"d59dc7c0-7b09-48d7-9459-0af53d2159ad\") " pod="openshift-dns/dns-default-gnstv"
Oct 31 00:28:15.276351 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:15.276294    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/690b2e89-76c8-42f1-a5cd-07567475bbaa-data-volume\") pod \"insights-runtime-extractor-fd8sp\" (UID: \"690b2e89-76c8-42f1-a5cd-07567475bbaa\") " pod="openshift-insights/insights-runtime-extractor-fd8sp"
Oct 31 00:28:15.276403 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:15.276379    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/690b2e89-76c8-42f1-a5cd-07567475bbaa-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-fd8sp\" (UID: \"690b2e89-76c8-42f1-a5cd-07567475bbaa\") " pod="openshift-insights/insights-runtime-extractor-fd8sp"
Oct 31 00:28:15.276844 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:15.276816    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/18948a7b-85a1-4ab0-b61b-ee42cf31b815-trusted-ca\") pod \"image-registry-85f776cbc8-lr9wd\" (UID: \"18948a7b-85a1-4ab0-b61b-ee42cf31b815\") " pod="openshift-image-registry/image-registry-85f776cbc8-lr9wd"
Oct 31 00:28:15.277105 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:15.277079    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/18948a7b-85a1-4ab0-b61b-ee42cf31b815-registry-certificates\") pod \"image-registry-85f776cbc8-lr9wd\" (UID: \"18948a7b-85a1-4ab0-b61b-ee42cf31b815\") " pod="openshift-image-registry/image-registry-85f776cbc8-lr9wd"
Oct 31 00:28:15.278144 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:15.278121    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/18948a7b-85a1-4ab0-b61b-ee42cf31b815-image-registry-private-configuration\") pod \"image-registry-85f776cbc8-lr9wd\" (UID: \"18948a7b-85a1-4ab0-b61b-ee42cf31b815\") " pod="openshift-image-registry/image-registry-85f776cbc8-lr9wd"
Oct 31 00:28:15.278538 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:15.278514    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/4a4ec287-fc28-413d-83fa-d9a05ddedba3-tls-certificates\") pod \"prometheus-operator-admission-webhook-6489d8bf89-g6vkf\" (UID: \"4a4ec287-fc28-413d-83fa-d9a05ddedba3\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-g6vkf"
Oct 31 00:28:15.278667 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:15.278652    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d59dc7c0-7b09-48d7-9459-0af53d2159ad-metrics-tls\") pod \"dns-default-gnstv\" (UID: \"d59dc7c0-7b09-48d7-9459-0af53d2159ad\") " pod="openshift-dns/dns-default-gnstv"
Oct 31 00:28:15.279008 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:15.278983    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/690b2e89-76c8-42f1-a5cd-07567475bbaa-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-fd8sp\" (UID: \"690b2e89-76c8-42f1-a5cd-07567475bbaa\") " pod="openshift-insights/insights-runtime-extractor-fd8sp"
Oct 31 00:28:15.279123 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:15.279104    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/18948a7b-85a1-4ab0-b61b-ee42cf31b815-installation-pull-secrets\") pod \"image-registry-85f776cbc8-lr9wd\" (UID: \"18948a7b-85a1-4ab0-b61b-ee42cf31b815\") " pod="openshift-image-registry/image-registry-85f776cbc8-lr9wd"
Oct 31 00:28:15.279305 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:15.279287    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/18948a7b-85a1-4ab0-b61b-ee42cf31b815-registry-tls\") pod \"image-registry-85f776cbc8-lr9wd\" (UID: \"18948a7b-85a1-4ab0-b61b-ee42cf31b815\") " pod="openshift-image-registry/image-registry-85f776cbc8-lr9wd"
Oct 31 00:28:15.334946 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:15.334909    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zww5h\" (UniqueName: \"kubernetes.io/projected/d59dc7c0-7b09-48d7-9459-0af53d2159ad-kube-api-access-zww5h\") pod \"dns-default-gnstv\" (UID: \"d59dc7c0-7b09-48d7-9459-0af53d2159ad\") " pod="openshift-dns/dns-default-gnstv"
Oct 31 00:28:15.335082 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:15.334949    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mwlnh\" (UniqueName: \"kubernetes.io/projected/690b2e89-76c8-42f1-a5cd-07567475bbaa-kube-api-access-mwlnh\") pod \"insights-runtime-extractor-fd8sp\" (UID: \"690b2e89-76c8-42f1-a5cd-07567475bbaa\") " pod="openshift-insights/insights-runtime-extractor-fd8sp"
Oct 31 00:28:15.335248 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:15.335232    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g9427\" (UniqueName: \"kubernetes.io/projected/18948a7b-85a1-4ab0-b61b-ee42cf31b815-kube-api-access-g9427\") pod \"image-registry-85f776cbc8-lr9wd\" (UID: \"18948a7b-85a1-4ab0-b61b-ee42cf31b815\") " pod="openshift-image-registry/image-registry-85f776cbc8-lr9wd"
Oct 31 00:28:15.338718 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:15.338694    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/18948a7b-85a1-4ab0-b61b-ee42cf31b815-bound-sa-token\") pod \"image-registry-85f776cbc8-lr9wd\" (UID: \"18948a7b-85a1-4ab0-b61b-ee42cf31b815\") " pod="openshift-image-registry/image-registry-85f776cbc8-lr9wd"
Oct 31 00:28:15.361865 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:15.361847    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-bn67m"
Oct 31 00:28:15.365769 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:15.365750    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-6dd789c8b7-fd796"
Oct 31 00:28:15.406160 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:15.406121    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-gnstv"
Oct 31 00:28:15.414464 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:15.414445    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-fd8sp"
Oct 31 00:28:15.429977 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:15.428618    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:28:15.460170 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:15.459361    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-g6vkf"
Oct 31 00:28:15.462249 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:15.462023    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-85f776cbc8-lr9wd"
Oct 31 00:28:15.576388 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:15.576349    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:28:15.719158 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:15.719127    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:28:15.726815 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:15.726790    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-6dd789c8b7-fd796"]
Oct 31 00:28:15.729672 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:15.729649    4284 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod308c723a_b97c_4acd_a65e_71dd5b389723.slice/crio-c11d693f696edcaea8c9dc59e2fd6318d21fed9ed9ae65708df749632bfc5ec7 WatchSource:0}: Error finding container c11d693f696edcaea8c9dc59e2fd6318d21fed9ed9ae65708df749632bfc5ec7: Status 404 returned error can't find the container with id c11d693f696edcaea8c9dc59e2fd6318d21fed9ed9ae65708df749632bfc5ec7
Oct 31 00:28:15.746987 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:15.746969    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-bn67m"]
Oct 31 00:28:15.749363 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:15.749334    4284 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddb5d4efa_ed50_44e9_84f2_851579e5c49e.slice/crio-6a1a07793986c87d92eb17ad2c413093aa594e55b4d5e564fd66ff5f958c6597 WatchSource:0}: Error finding container 6a1a07793986c87d92eb17ad2c413093aa594e55b4d5e564fd66ff5f958c6597: Status 404 returned error can't find the container with id 6a1a07793986c87d92eb17ad2c413093aa594e55b4d5e564fd66ff5f958c6597
Oct 31 00:28:15.780305 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:15.780268    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rthdh\" (UniqueName: \"kubernetes.io/projected/1eccd43e-037f-4de4-a2de-c0961b97865e-kube-api-access-rthdh\") pod \"migrator-66df5d496b-wz5p4\" (UID: \"1eccd43e-037f-4de4-a2de-c0961b97865e\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-wz5p4"
Oct 31 00:28:15.782789 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:15.782734    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rthdh\" (UniqueName: \"kubernetes.io/projected/1eccd43e-037f-4de4-a2de-c0961b97865e-kube-api-access-rthdh\") pod \"migrator-66df5d496b-wz5p4\" (UID: \"1eccd43e-037f-4de4-a2de-c0961b97865e\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-wz5p4"
Oct 31 00:28:15.832839 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:15.832797    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-gnstv"]
Oct 31 00:28:15.835162 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:15.835130    4284 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd59dc7c0_7b09_48d7_9459_0af53d2159ad.slice/crio-de58b97c7dd598c7f110d5566af21d82a9abd57c5453f8a83c45d9d41e67c414 WatchSource:0}: Error finding container de58b97c7dd598c7f110d5566af21d82a9abd57c5453f8a83c45d9d41e67c414: Status 404 returned error can't find the container with id de58b97c7dd598c7f110d5566af21d82a9abd57c5453f8a83c45d9d41e67c414
Oct 31 00:28:15.842624 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:15.841035    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-fd8sp"]
Oct 31 00:28:15.842624 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:15.842010    4284 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod690b2e89_76c8_42f1_a5cd_07567475bbaa.slice/crio-dd8c14f7de731c20cc09ef7fc7bcf3a20e9b49983db4388906595a159bec8d97 WatchSource:0}: Error finding container dd8c14f7de731c20cc09ef7fc7bcf3a20e9b49983db4388906595a159bec8d97: Status 404 returned error can't find the container with id dd8c14f7de731c20cc09ef7fc7bcf3a20e9b49983db4388906595a159bec8d97
Oct 31 00:28:15.849741 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:15.849718    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-g6vkf"]
Oct 31 00:28:15.852663 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:15.852645    4284 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4a4ec287_fc28_413d_83fa_d9a05ddedba3.slice/crio-77af367c4044ca0f3ed2140ea34e59d7f73b5747ba67a6f017aa3ec800d562a2 WatchSource:0}: Error finding container 77af367c4044ca0f3ed2140ea34e59d7f73b5747ba67a6f017aa3ec800d562a2: Status 404 returned error can't find the container with id 77af367c4044ca0f3ed2140ea34e59d7f73b5747ba67a6f017aa3ec800d562a2
Oct 31 00:28:15.867778 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:15.867755    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-85f776cbc8-lr9wd"]
Oct 31 00:28:15.873037 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:15.873016    4284 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod18948a7b_85a1_4ab0_b61b_ee42cf31b815.slice/crio-027e56b25b2e516d6098705352c17b610915337fdb489ba257dd2d0dde195b67 WatchSource:0}: Error finding container 027e56b25b2e516d6098705352c17b610915337fdb489ba257dd2d0dde195b67: Status 404 returned error can't find the container with id 027e56b25b2e516d6098705352c17b610915337fdb489ba257dd2d0dde195b67
Oct 31 00:28:15.955677 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:15.955645    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-wz5p4"
Oct 31 00:28:16.014569 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:16.014542    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-g6vkf" event={"ID":"4a4ec287-fc28-413d-83fa-d9a05ddedba3","Type":"ContainerStarted","Data":"77af367c4044ca0f3ed2140ea34e59d7f73b5747ba67a6f017aa3ec800d562a2"}
Oct 31 00:28:16.016294 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:16.016102    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-bn67m" event={"ID":"db5d4efa-ed50-44e9-84f2-851579e5c49e","Type":"ContainerStarted","Data":"6a1a07793986c87d92eb17ad2c413093aa594e55b4d5e564fd66ff5f958c6597"}
Oct 31 00:28:16.017953 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:16.017888    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-85f776cbc8-lr9wd" event={"ID":"18948a7b-85a1-4ab0-b61b-ee42cf31b815","Type":"ContainerStarted","Data":"927f47dc7296c08d245a017c8ea5951f4f5283f766a5c630d3ff00f0e591809f"}
Oct 31 00:28:16.017953 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:16.017926    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-85f776cbc8-lr9wd" event={"ID":"18948a7b-85a1-4ab0-b61b-ee42cf31b815","Type":"ContainerStarted","Data":"027e56b25b2e516d6098705352c17b610915337fdb489ba257dd2d0dde195b67"}
Oct 31 00:28:16.025277 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:16.025238    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-fd8sp" event={"ID":"690b2e89-76c8-42f1-a5cd-07567475bbaa","Type":"ContainerStarted","Data":"6186eae38e27a299cc6f070abff5178bf08a20a19fb19d125c1c94722adc18e7"}
Oct 31 00:28:16.025379 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:16.025299    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-fd8sp" event={"ID":"690b2e89-76c8-42f1-a5cd-07567475bbaa","Type":"ContainerStarted","Data":"dd8c14f7de731c20cc09ef7fc7bcf3a20e9b49983db4388906595a159bec8d97"}
Oct 31 00:28:16.026460 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:16.026435    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-gnstv" event={"ID":"d59dc7c0-7b09-48d7-9459-0af53d2159ad","Type":"ContainerStarted","Data":"de58b97c7dd598c7f110d5566af21d82a9abd57c5453f8a83c45d9d41e67c414"}
Oct 31 00:28:16.027585 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:16.027549    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-6dd789c8b7-fd796" event={"ID":"308c723a-b97c-4acd-a65e-71dd5b389723","Type":"ContainerStarted","Data":"c11d693f696edcaea8c9dc59e2fd6318d21fed9ed9ae65708df749632bfc5ec7"}
Oct 31 00:28:16.133924 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:16.133822    4284 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-image-registry/image-registry-85f776cbc8-lr9wd" podStartSLOduration=1.133808117 podStartE2EDuration="1.133808117s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 00:28:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:28:16.133518207 +0000 UTC m=+10.278525123" watchObservedRunningTime="2026-10-31 00:28:16.133808117 +0000 UTC m=+10.278815032"
Oct 31 00:28:16.315499 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:16.315477    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-66df5d496b-wz5p4"]
Oct 31 00:28:16.317623 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:16.317597    4284 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1eccd43e_037f_4de4_a2de_c0961b97865e.slice/crio-6b74d5e3567dac2652253647265d50a9a2dd12914ceddd364ffdf1da2feddc9a WatchSource:0}: Error finding container 6b74d5e3567dac2652253647265d50a9a2dd12914ceddd364ffdf1da2feddc9a: Status 404 returned error can't find the container with id 6b74d5e3567dac2652253647265d50a9a2dd12914ceddd364ffdf1da2feddc9a
Oct 31 00:28:16.750605 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:16.750575    4284 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-console/console-c787779cc-hg2q9"]
Oct 31 00:28:16.779527 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:16.779502    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-c787779cc-hg2q9"
Oct 31 00:28:16.810876 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:16.810817    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-c787779cc-hg2q9"]
Oct 31 00:28:16.830566 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:16.830541    4284 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Oct 31 00:28:16.830566 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:16.830541    4284 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Oct 31 00:28:16.830760 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:16.830542    4284 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Oct 31 00:28:16.830849 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:16.830835    4284 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Oct 31 00:28:16.831052 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:16.831032    4284 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Oct 31 00:28:16.831052 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:16.831039    4284 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-wqtpp\""
Oct 31 00:28:16.886885 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:16.886853    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4bbd777b-3d7c-4f4c-8187-69301e8ee512-console-oauth-config\") pod \"console-c787779cc-hg2q9\" (UID: \"4bbd777b-3d7c-4f4c-8187-69301e8ee512\") " pod="openshift-console/console-c787779cc-hg2q9"
Oct 31 00:28:16.886885 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:16.886885    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d26g6\" (UniqueName: \"kubernetes.io/projected/4bbd777b-3d7c-4f4c-8187-69301e8ee512-kube-api-access-d26g6\") pod \"console-c787779cc-hg2q9\" (UID: \"4bbd777b-3d7c-4f4c-8187-69301e8ee512\") " pod="openshift-console/console-c787779cc-hg2q9"
Oct 31 00:28:16.887070 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:16.886916    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4bbd777b-3d7c-4f4c-8187-69301e8ee512-console-serving-cert\") pod \"console-c787779cc-hg2q9\" (UID: \"4bbd777b-3d7c-4f4c-8187-69301e8ee512\") " pod="openshift-console/console-c787779cc-hg2q9"
Oct 31 00:28:16.887070 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:16.886968    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4bbd777b-3d7c-4f4c-8187-69301e8ee512-console-config\") pod \"console-c787779cc-hg2q9\" (UID: \"4bbd777b-3d7c-4f4c-8187-69301e8ee512\") " pod="openshift-console/console-c787779cc-hg2q9"
Oct 31 00:28:16.887070 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:16.887019    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4bbd777b-3d7c-4f4c-8187-69301e8ee512-tmp\") pod \"console-c787779cc-hg2q9\" (UID: \"4bbd777b-3d7c-4f4c-8187-69301e8ee512\") " pod="openshift-console/console-c787779cc-hg2q9"
Oct 31 00:28:16.887181 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:16.887102    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4bbd777b-3d7c-4f4c-8187-69301e8ee512-oauth-serving-cert\") pod \"console-c787779cc-hg2q9\" (UID: \"4bbd777b-3d7c-4f4c-8187-69301e8ee512\") " pod="openshift-console/console-c787779cc-hg2q9"
Oct 31 00:28:16.887181 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:16.887121    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4bbd777b-3d7c-4f4c-8187-69301e8ee512-service-ca\") pod \"console-c787779cc-hg2q9\" (UID: \"4bbd777b-3d7c-4f4c-8187-69301e8ee512\") " pod="openshift-console/console-c787779cc-hg2q9"
Oct 31 00:28:16.988447 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:16.988415    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d26g6\" (UniqueName: \"kubernetes.io/projected/4bbd777b-3d7c-4f4c-8187-69301e8ee512-kube-api-access-d26g6\") pod \"console-c787779cc-hg2q9\" (UID: \"4bbd777b-3d7c-4f4c-8187-69301e8ee512\") " pod="openshift-console/console-c787779cc-hg2q9"
Oct 31 00:28:16.988598 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:16.988461    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4bbd777b-3d7c-4f4c-8187-69301e8ee512-console-serving-cert\") pod \"console-c787779cc-hg2q9\" (UID: \"4bbd777b-3d7c-4f4c-8187-69301e8ee512\") " pod="openshift-console/console-c787779cc-hg2q9"
Oct 31 00:28:16.988598 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:16.988491    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4bbd777b-3d7c-4f4c-8187-69301e8ee512-console-config\") pod \"console-c787779cc-hg2q9\" (UID: \"4bbd777b-3d7c-4f4c-8187-69301e8ee512\") " pod="openshift-console/console-c787779cc-hg2q9"
Oct 31 00:28:16.988598 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:16.988515    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4bbd777b-3d7c-4f4c-8187-69301e8ee512-tmp\") pod \"console-c787779cc-hg2q9\" (UID: \"4bbd777b-3d7c-4f4c-8187-69301e8ee512\") " pod="openshift-console/console-c787779cc-hg2q9"
Oct 31 00:28:16.988598 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:16.988539    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4bbd777b-3d7c-4f4c-8187-69301e8ee512-oauth-serving-cert\") pod \"console-c787779cc-hg2q9\" (UID: \"4bbd777b-3d7c-4f4c-8187-69301e8ee512\") " pod="openshift-console/console-c787779cc-hg2q9"
Oct 31 00:28:16.988796 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:16.988779    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4bbd777b-3d7c-4f4c-8187-69301e8ee512-service-ca\") pod \"console-c787779cc-hg2q9\" (UID: \"4bbd777b-3d7c-4f4c-8187-69301e8ee512\") " pod="openshift-console/console-c787779cc-hg2q9"
Oct 31 00:28:16.988850 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:16.988824    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4bbd777b-3d7c-4f4c-8187-69301e8ee512-console-oauth-config\") pod \"console-c787779cc-hg2q9\" (UID: \"4bbd777b-3d7c-4f4c-8187-69301e8ee512\") " pod="openshift-console/console-c787779cc-hg2q9"
Oct 31 00:28:16.988908 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:16.988847    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4bbd777b-3d7c-4f4c-8187-69301e8ee512-tmp\") pod \"console-c787779cc-hg2q9\" (UID: \"4bbd777b-3d7c-4f4c-8187-69301e8ee512\") " pod="openshift-console/console-c787779cc-hg2q9"
Oct 31 00:28:16.989264 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:16.989243    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4bbd777b-3d7c-4f4c-8187-69301e8ee512-oauth-serving-cert\") pod \"console-c787779cc-hg2q9\" (UID: \"4bbd777b-3d7c-4f4c-8187-69301e8ee512\") " pod="openshift-console/console-c787779cc-hg2q9"
Oct 31 00:28:16.989368 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:16.989351    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4bbd777b-3d7c-4f4c-8187-69301e8ee512-console-config\") pod \"console-c787779cc-hg2q9\" (UID: \"4bbd777b-3d7c-4f4c-8187-69301e8ee512\") " pod="openshift-console/console-c787779cc-hg2q9"
Oct 31 00:28:16.989509 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:16.989492    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4bbd777b-3d7c-4f4c-8187-69301e8ee512-service-ca\") pod \"console-c787779cc-hg2q9\" (UID: \"4bbd777b-3d7c-4f4c-8187-69301e8ee512\") " pod="openshift-console/console-c787779cc-hg2q9"
Oct 31 00:28:16.991745 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:16.991720    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4bbd777b-3d7c-4f4c-8187-69301e8ee512-console-oauth-config\") pod \"console-c787779cc-hg2q9\" (UID: \"4bbd777b-3d7c-4f4c-8187-69301e8ee512\") " pod="openshift-console/console-c787779cc-hg2q9"
Oct 31 00:28:16.995702 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:16.995679    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4bbd777b-3d7c-4f4c-8187-69301e8ee512-console-serving-cert\") pod \"console-c787779cc-hg2q9\" (UID: \"4bbd777b-3d7c-4f4c-8187-69301e8ee512\") " pod="openshift-console/console-c787779cc-hg2q9"
Oct 31 00:28:17.032181 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:17.032146    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-wz5p4" event={"ID":"1eccd43e-037f-4de4-a2de-c0961b97865e","Type":"ContainerStarted","Data":"6b74d5e3567dac2652253647265d50a9a2dd12914ceddd364ffdf1da2feddc9a"}
Oct 31 00:28:17.032643 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:17.032192    4284 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Oct 31 00:28:17.048444 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:17.048413    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d26g6\" (UniqueName: \"kubernetes.io/projected/4bbd777b-3d7c-4f4c-8187-69301e8ee512-kube-api-access-d26g6\") pod \"console-c787779cc-hg2q9\" (UID: \"4bbd777b-3d7c-4f4c-8187-69301e8ee512\") " pod="openshift-console/console-c787779cc-hg2q9"
Oct 31 00:28:17.110198 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:17.110124    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-c787779cc-hg2q9"
Oct 31 00:28:17.541308 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:17.541198    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-c787779cc-hg2q9"]
Oct 31 00:28:17.556193 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:17.556154    4284 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4bbd777b_3d7c_4f4c_8187_69301e8ee512.slice/crio-1b3189fab4d98112899a582b111937f81724fb9afd67af1aed4503670f73fe21 WatchSource:0}: Error finding container 1b3189fab4d98112899a582b111937f81724fb9afd67af1aed4503670f73fe21: Status 404 returned error can't find the container with id 1b3189fab4d98112899a582b111937f81724fb9afd67af1aed4503670f73fe21
Oct 31 00:28:17.868400 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:17.868237    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:28:18.015963 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:18.015925    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:28:18.038009 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:18.037975    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-c787779cc-hg2q9" event={"ID":"4bbd777b-3d7c-4f4c-8187-69301e8ee512","Type":"ContainerStarted","Data":"1b3189fab4d98112899a582b111937f81724fb9afd67af1aed4503670f73fe21"}
Oct 31 00:28:18.476362 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:18.476317    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-85f776cbc8-lr9wd"
Oct 31 00:28:18.927571 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:18.927504    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:28:19.045887 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:19.045713    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-fd8sp" event={"ID":"690b2e89-76c8-42f1-a5cd-07567475bbaa","Type":"ContainerStarted","Data":"039a02b1c72383141ec0d1286511d1ed2a530385ae3d628e262deb0260f3b375"}
Oct 31 00:28:19.048579 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:19.048551    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-g6vkf" event={"ID":"4a4ec287-fc28-413d-83fa-d9a05ddedba3","Type":"ContainerStarted","Data":"9687c6dacc86ee4e110e0229d50ac60a9c1ce71c9b3ac8891c4f728cada8daf1"}
Oct 31 00:28:19.049435 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:19.049413    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-g6vkf"
Oct 31 00:28:19.051684 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:19.051658    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-wz5p4" event={"ID":"1eccd43e-037f-4de4-a2de-c0961b97865e","Type":"ContainerStarted","Data":"d4f0f1084cdbeace14ec2c8706c77cd6ffef067c098203c50588175013f5c9b6"}
Oct 31 00:28:19.055671 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:19.055651    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-g6vkf"
Oct 31 00:28:19.124901 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:19.124839    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:28:19.168989 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:19.168932    4284 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-g6vkf" podStartSLOduration=1.681640758 podStartE2EDuration="4.168914982s" totalImagesPullingTime="2.487274224s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 00:28:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:28:19.168727569 +0000 UTC m=+13.313734490" watchObservedRunningTime="2026-10-31 00:28:19.168914982 +0000 UTC m=+13.313921902"
Oct 31 00:28:19.362670 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:19.362173    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-k2sk2"
Oct 31 00:28:19.429888 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:19.429846    4284 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-wz5p4" podStartSLOduration=2.969459649 podStartE2EDuration="5.429832132s" totalImagesPullingTime="2.460372483s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 00:28:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:28:19.290148309 +0000 UTC m=+13.435155225" watchObservedRunningTime="2026-10-31 00:28:19.429832132 +0000 UTC m=+13.574839026"
Oct 31 00:28:19.574116 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:19.574032    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:28:19.777244 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:19.777120    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:28:19.924953 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:19.924825    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:28:20.060152 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:20.060110    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-gnstv" event={"ID":"d59dc7c0-7b09-48d7-9459-0af53d2159ad","Type":"ContainerStarted","Data":"b8060eb3588b7a2141a5672483e7fbb34d2ebe669a55021392cf8b263c36de9a"}
Oct 31 00:28:20.060682 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:20.060180    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-gnstv" event={"ID":"d59dc7c0-7b09-48d7-9459-0af53d2159ad","Type":"ContainerStarted","Data":"127c03ddaa30d9cebfff582a6773acd2db3f78652d3b8a0762257a619cf18db9"}
Oct 31 00:28:20.067783 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:20.067713    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:28:20.174522 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:20.174444    4284 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-dns/dns-default-gnstv" podStartSLOduration=1.825992995 podStartE2EDuration="5.174425343s" totalImagesPullingTime="3.348432348s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 00:28:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:28:20.173568676 +0000 UTC m=+14.318575596" watchObservedRunningTime="2026-10-31 00:28:20.174425343 +0000 UTC m=+14.319432262"
Oct 31 00:28:20.221978 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:20.221909    4284 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-9cd89b74c-4zn4l"]
Oct 31 00:28:20.224419 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:20.224397    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-9cd89b74c-4zn4l"
Oct 31 00:28:20.254639 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:20.254563    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:28:20.282829 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:20.282801    4284 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Oct 31 00:28:20.284225 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:20.284191    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-9cd89b74c-4zn4l"]
Oct 31 00:28:20.284570 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:20.284555    4284 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Oct 31 00:28:20.284651 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:20.284620    4284 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Oct 31 00:28:20.284738 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:20.284726    4284 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\""
Oct 31 00:28:20.285052 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:20.285037    4284 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-db8k2\""
Oct 31 00:28:20.286541 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:20.286515    4284 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\""
Oct 31 00:28:20.317012 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:20.316779    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/98471199-826e-416a-8052-d88790dc7893-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-9cd89b74c-4zn4l\" (UID: \"98471199-826e-416a-8052-d88790dc7893\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-4zn4l"
Oct 31 00:28:20.317012 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:20.316822    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/98471199-826e-416a-8052-d88790dc7893-metrics-client-ca\") pod \"prometheus-operator-9cd89b74c-4zn4l\" (UID: \"98471199-826e-416a-8052-d88790dc7893\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-4zn4l"
Oct 31 00:28:20.317012 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:20.316844    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/98471199-826e-416a-8052-d88790dc7893-prometheus-operator-tls\") pod \"prometheus-operator-9cd89b74c-4zn4l\" (UID: \"98471199-826e-416a-8052-d88790dc7893\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-4zn4l"
Oct 31 00:28:20.317012 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:20.316880    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-856w6\" (UniqueName: \"kubernetes.io/projected/98471199-826e-416a-8052-d88790dc7893-kube-api-access-856w6\") pod \"prometheus-operator-9cd89b74c-4zn4l\" (UID: \"98471199-826e-416a-8052-d88790dc7893\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-4zn4l"
Oct 31 00:28:20.417716 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:20.417673    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/98471199-826e-416a-8052-d88790dc7893-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-9cd89b74c-4zn4l\" (UID: \"98471199-826e-416a-8052-d88790dc7893\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-4zn4l"
Oct 31 00:28:20.417895 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:20.417736    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/98471199-826e-416a-8052-d88790dc7893-metrics-client-ca\") pod \"prometheus-operator-9cd89b74c-4zn4l\" (UID: \"98471199-826e-416a-8052-d88790dc7893\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-4zn4l"
Oct 31 00:28:20.417895 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:20.417764    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/98471199-826e-416a-8052-d88790dc7893-prometheus-operator-tls\") pod \"prometheus-operator-9cd89b74c-4zn4l\" (UID: \"98471199-826e-416a-8052-d88790dc7893\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-4zn4l"
Oct 31 00:28:20.417895 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:20.417808    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-856w6\" (UniqueName: \"kubernetes.io/projected/98471199-826e-416a-8052-d88790dc7893-kube-api-access-856w6\") pod \"prometheus-operator-9cd89b74c-4zn4l\" (UID: \"98471199-826e-416a-8052-d88790dc7893\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-4zn4l"
Oct 31 00:28:20.419007 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:20.418960    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/98471199-826e-416a-8052-d88790dc7893-metrics-client-ca\") pod \"prometheus-operator-9cd89b74c-4zn4l\" (UID: \"98471199-826e-416a-8052-d88790dc7893\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-4zn4l"
Oct 31 00:28:20.421199 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:20.421179    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/98471199-826e-416a-8052-d88790dc7893-prometheus-operator-tls\") pod \"prometheus-operator-9cd89b74c-4zn4l\" (UID: \"98471199-826e-416a-8052-d88790dc7893\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-4zn4l"
Oct 31 00:28:20.422897 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:20.422876    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/98471199-826e-416a-8052-d88790dc7893-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-9cd89b74c-4zn4l\" (UID: \"98471199-826e-416a-8052-d88790dc7893\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-4zn4l"
Oct 31 00:28:20.482015 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:20.481937    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-856w6\" (UniqueName: \"kubernetes.io/projected/98471199-826e-416a-8052-d88790dc7893-kube-api-access-856w6\") pod \"prometheus-operator-9cd89b74c-4zn4l\" (UID: \"98471199-826e-416a-8052-d88790dc7893\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-4zn4l"
Oct 31 00:28:20.539598 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:20.539553    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-9cd89b74c-4zn4l"
Oct 31 00:28:20.978982 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:20.978854    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-9cd89b74c-4zn4l"]
Oct 31 00:28:20.985745 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:20.985703    4284 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod98471199_826e_416a_8052_d88790dc7893.slice/crio-b464dc310fefa6c91c8457488e7468f2b7ce9938a4dfa08cd1c2b0d972fdca7c WatchSource:0}: Error finding container b464dc310fefa6c91c8457488e7468f2b7ce9938a4dfa08cd1c2b0d972fdca7c: Status 404 returned error can't find the container with id b464dc310fefa6c91c8457488e7468f2b7ce9938a4dfa08cd1c2b0d972fdca7c
Oct 31 00:28:21.068502 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:21.068419    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-fd8sp" event={"ID":"690b2e89-76c8-42f1-a5cd-07567475bbaa","Type":"ContainerStarted","Data":"7b03818f0dd48828552245b82d19d6d651f3d7b26186f7ee4ed19d35ef680a6b"}
Oct 31 00:28:21.070240 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:21.070188    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-9cd89b74c-4zn4l" event={"ID":"98471199-826e-416a-8052-d88790dc7893","Type":"ContainerStarted","Data":"b464dc310fefa6c91c8457488e7468f2b7ce9938a4dfa08cd1c2b0d972fdca7c"}
Oct 31 00:28:21.070721 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:21.070699    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-gnstv"
Oct 31 00:28:22.078762 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:22.078011    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-bn67m" event={"ID":"db5d4efa-ed50-44e9-84f2-851579e5c49e","Type":"ContainerStarted","Data":"a4cbbcc13240f05afe2f299e8f0818fbf4c8279e11b90d6ae5cde3a991bce5fd"}
Oct 31 00:28:22.087168 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:22.086639    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-c787779cc-hg2q9" event={"ID":"4bbd777b-3d7c-4f4c-8187-69301e8ee512","Type":"ContainerStarted","Data":"b2c94cc3eb599995496d52c2f8a9c7288fa0539dd3550af99beb50147f821ce8"}
Oct 31 00:28:22.190644 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:22.190577    4284 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-fd8sp" podStartSLOduration=2.360521062 podStartE2EDuration="7.190557869s" totalImagesPullingTime="4.830036807s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-31 00:28:15 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:28:21.184363458 +0000 UTC m=+15.329370375" watchObservedRunningTime="2026-10-31 00:28:22.190557869 +0000 UTC m=+16.335564785"
Oct 31 00:28:22.191152 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:22.191122    4284 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-bn67m" podStartSLOduration=1.945171438 podStartE2EDuration="8.191110205s" totalImagesPullingTime="6.245938767s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 00:28:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:28:22.189871451 +0000 UTC m=+16.334878368" watchObservedRunningTime="2026-10-31 00:28:22.191110205 +0000 UTC m=+16.336117118"
Oct 31 00:28:22.304028 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:22.303922    4284 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-console/console-c787779cc-hg2q9" podStartSLOduration=1.8718048569999999 podStartE2EDuration="6.303901408s" totalImagesPullingTime="4.432096551s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 00:28:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:28:22.303367113 +0000 UTC m=+16.448374030" watchObservedRunningTime="2026-10-31 00:28:22.303901408 +0000 UTC m=+16.448908327"
Oct 31 00:28:22.702592 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:22.702556    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:28:22.981708 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:22.980490    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-rrk77"
Oct 31 00:28:23.091294 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:23.091260    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-9cd89b74c-4zn4l" event={"ID":"98471199-826e-416a-8052-d88790dc7893","Type":"ContainerStarted","Data":"7fbbdd3fbebd31f77f3b8910a591191d7f940a6a69660942fdf69755a19c74e1"}
Oct 31 00:28:23.091703 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:23.091333    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-9cd89b74c-4zn4l" event={"ID":"98471199-826e-416a-8052-d88790dc7893","Type":"ContainerStarted","Data":"5ea6de4cdad776526ed518fe61436119302650a8f158f5a1739fa46157f6d995"}
Oct 31 00:28:23.312902 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:23.312797    4284 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-9cd89b74c-4zn4l" podStartSLOduration=1.385297735 podStartE2EDuration="3.312776014s" totalImagesPullingTime="1.927478279s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 00:28:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:28:23.31245128 +0000 UTC m=+17.457458203" watchObservedRunningTime="2026-10-31 00:28:23.312776014 +0000 UTC m=+17.457782932"
Oct 31 00:28:23.347997 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:23.347960    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:28:23.561951 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:23.561912    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:28:25.490819 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:25.490777    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:28:26.054904 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:26.054864    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:28:26.206322 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:26.206284    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:28:26.542333 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:26.542300    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:28:26.686280 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:26.686244    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:28:26.844967 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:26.844897    4284 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-np54c"]
Oct 31 00:28:26.847956 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:26.847929    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-np54c"
Oct 31 00:28:26.852432 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:26.852411    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:28:26.870548 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:26.870528    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/27f80c70-260d-492b-b0c5-8dd358f8a8f1-metrics-client-ca\") pod \"node-exporter-np54c\" (UID: \"27f80c70-260d-492b-b0c5-8dd358f8a8f1\") " pod="openshift-monitoring/node-exporter-np54c"
Oct 31 00:28:26.870665 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:26.870558    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/27f80c70-260d-492b-b0c5-8dd358f8a8f1-node-exporter-tls\") pod \"node-exporter-np54c\" (UID: \"27f80c70-260d-492b-b0c5-8dd358f8a8f1\") " pod="openshift-monitoring/node-exporter-np54c"
Oct 31 00:28:26.870665 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:26.870583    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/27f80c70-260d-492b-b0c5-8dd358f8a8f1-node-exporter-accelerators-collector-config\") pod \"node-exporter-np54c\" (UID: \"27f80c70-260d-492b-b0c5-8dd358f8a8f1\") " pod="openshift-monitoring/node-exporter-np54c"
Oct 31 00:28:26.870768 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:26.870672    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/27f80c70-260d-492b-b0c5-8dd358f8a8f1-node-exporter-textfile\") pod \"node-exporter-np54c\" (UID: \"27f80c70-260d-492b-b0c5-8dd358f8a8f1\") " pod="openshift-monitoring/node-exporter-np54c"
Oct 31 00:28:26.870768 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:26.870711    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/27f80c70-260d-492b-b0c5-8dd358f8a8f1-node-exporter-wtmp\") pod \"node-exporter-np54c\" (UID: \"27f80c70-260d-492b-b0c5-8dd358f8a8f1\") " pod="openshift-monitoring/node-exporter-np54c"
Oct 31 00:28:26.870855 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:26.870833    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/27f80c70-260d-492b-b0c5-8dd358f8a8f1-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-np54c\" (UID: \"27f80c70-260d-492b-b0c5-8dd358f8a8f1\") " pod="openshift-monitoring/node-exporter-np54c"
Oct 31 00:28:26.870893 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:26.870868    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/27f80c70-260d-492b-b0c5-8dd358f8a8f1-sys\") pod \"node-exporter-np54c\" (UID: \"27f80c70-260d-492b-b0c5-8dd358f8a8f1\") " pod="openshift-monitoring/node-exporter-np54c"
Oct 31 00:28:26.870893 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:26.870885    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bwclp\" (UniqueName: \"kubernetes.io/projected/27f80c70-260d-492b-b0c5-8dd358f8a8f1-kube-api-access-bwclp\") pod \"node-exporter-np54c\" (UID: \"27f80c70-260d-492b-b0c5-8dd358f8a8f1\") " pod="openshift-monitoring/node-exporter-np54c"
Oct 31 00:28:26.870990 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:26.870908    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/27f80c70-260d-492b-b0c5-8dd358f8a8f1-root\") pod \"node-exporter-np54c\" (UID: \"27f80c70-260d-492b-b0c5-8dd358f8a8f1\") " pod="openshift-monitoring/node-exporter-np54c"
Oct 31 00:28:26.971725 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:26.971694    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/27f80c70-260d-492b-b0c5-8dd358f8a8f1-node-exporter-tls\") pod \"node-exporter-np54c\" (UID: \"27f80c70-260d-492b-b0c5-8dd358f8a8f1\") " pod="openshift-monitoring/node-exporter-np54c"
Oct 31 00:28:26.971902 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:26.971735    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/27f80c70-260d-492b-b0c5-8dd358f8a8f1-node-exporter-accelerators-collector-config\") pod \"node-exporter-np54c\" (UID: \"27f80c70-260d-492b-b0c5-8dd358f8a8f1\") " pod="openshift-monitoring/node-exporter-np54c"
Oct 31 00:28:26.971902 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:26.971771    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/27f80c70-260d-492b-b0c5-8dd358f8a8f1-node-exporter-textfile\") pod \"node-exporter-np54c\" (UID: \"27f80c70-260d-492b-b0c5-8dd358f8a8f1\") " pod="openshift-monitoring/node-exporter-np54c"
Oct 31 00:28:26.971902 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:26.971866    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/27f80c70-260d-492b-b0c5-8dd358f8a8f1-node-exporter-wtmp\") pod \"node-exporter-np54c\" (UID: \"27f80c70-260d-492b-b0c5-8dd358f8a8f1\") " pod="openshift-monitoring/node-exporter-np54c"
Oct 31 00:28:26.972076 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:26.971931    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/27f80c70-260d-492b-b0c5-8dd358f8a8f1-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-np54c\" (UID: \"27f80c70-260d-492b-b0c5-8dd358f8a8f1\") " pod="openshift-monitoring/node-exporter-np54c"
Oct 31 00:28:26.972076 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:26.971967    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/27f80c70-260d-492b-b0c5-8dd358f8a8f1-sys\") pod \"node-exporter-np54c\" (UID: \"27f80c70-260d-492b-b0c5-8dd358f8a8f1\") " pod="openshift-monitoring/node-exporter-np54c"
Oct 31 00:28:26.972076 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:26.971990    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bwclp\" (UniqueName: \"kubernetes.io/projected/27f80c70-260d-492b-b0c5-8dd358f8a8f1-kube-api-access-bwclp\") pod \"node-exporter-np54c\" (UID: \"27f80c70-260d-492b-b0c5-8dd358f8a8f1\") " pod="openshift-monitoring/node-exporter-np54c"
Oct 31 00:28:26.972076 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:26.972031    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/27f80c70-260d-492b-b0c5-8dd358f8a8f1-root\") pod \"node-exporter-np54c\" (UID: \"27f80c70-260d-492b-b0c5-8dd358f8a8f1\") " pod="openshift-monitoring/node-exporter-np54c"
Oct 31 00:28:26.972076 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:26.972048    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/27f80c70-260d-492b-b0c5-8dd358f8a8f1-node-exporter-wtmp\") pod \"node-exporter-np54c\" (UID: \"27f80c70-260d-492b-b0c5-8dd358f8a8f1\") " pod="openshift-monitoring/node-exporter-np54c"
Oct 31 00:28:26.972076 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:26.972063    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/27f80c70-260d-492b-b0c5-8dd358f8a8f1-metrics-client-ca\") pod \"node-exporter-np54c\" (UID: \"27f80c70-260d-492b-b0c5-8dd358f8a8f1\") " pod="openshift-monitoring/node-exporter-np54c"
Oct 31 00:28:26.972344 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:26.972131    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/27f80c70-260d-492b-b0c5-8dd358f8a8f1-node-exporter-textfile\") pod \"node-exporter-np54c\" (UID: \"27f80c70-260d-492b-b0c5-8dd358f8a8f1\") " pod="openshift-monitoring/node-exporter-np54c"
Oct 31 00:28:26.972344 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:26.972189    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/27f80c70-260d-492b-b0c5-8dd358f8a8f1-sys\") pod \"node-exporter-np54c\" (UID: \"27f80c70-260d-492b-b0c5-8dd358f8a8f1\") " pod="openshift-monitoring/node-exporter-np54c"
Oct 31 00:28:26.972344 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:26.972197    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/27f80c70-260d-492b-b0c5-8dd358f8a8f1-root\") pod \"node-exporter-np54c\" (UID: \"27f80c70-260d-492b-b0c5-8dd358f8a8f1\") " pod="openshift-monitoring/node-exporter-np54c"
Oct 31 00:28:26.972716 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:26.972691    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/27f80c70-260d-492b-b0c5-8dd358f8a8f1-metrics-client-ca\") pod \"node-exporter-np54c\" (UID: \"27f80c70-260d-492b-b0c5-8dd358f8a8f1\") " pod="openshift-monitoring/node-exporter-np54c"
Oct 31 00:28:26.999916 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:26.999885    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:28:27.010720 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:27.010692    4284 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Oct 31 00:28:27.010940 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:27.010922    4284 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Oct 31 00:28:27.010980 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:27.010928    4284 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-7hhvp\""
Oct 31 00:28:27.011016 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:27.010982    4284 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Oct 31 00:28:27.012777 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:27.012727    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/27f80c70-260d-492b-b0c5-8dd358f8a8f1-node-exporter-accelerators-collector-config\") pod \"node-exporter-np54c\" (UID: \"27f80c70-260d-492b-b0c5-8dd358f8a8f1\") " pod="openshift-monitoring/node-exporter-np54c"
Oct 31 00:28:27.014341 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:27.014317    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/27f80c70-260d-492b-b0c5-8dd358f8a8f1-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-np54c\" (UID: \"27f80c70-260d-492b-b0c5-8dd358f8a8f1\") " pod="openshift-monitoring/node-exporter-np54c"
Oct 31 00:28:27.014472 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:27.014454    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/27f80c70-260d-492b-b0c5-8dd358f8a8f1-node-exporter-tls\") pod \"node-exporter-np54c\" (UID: \"27f80c70-260d-492b-b0c5-8dd358f8a8f1\") " pod="openshift-monitoring/node-exporter-np54c"
Oct 31 00:28:27.031866 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:27.031839    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bwclp\" (UniqueName: \"kubernetes.io/projected/27f80c70-260d-492b-b0c5-8dd358f8a8f1-kube-api-access-bwclp\") pod \"node-exporter-np54c\" (UID: \"27f80c70-260d-492b-b0c5-8dd358f8a8f1\") " pod="openshift-monitoring/node-exporter-np54c"
Oct 31 00:28:27.110325 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:27.110254    4284 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-c787779cc-hg2q9"
Oct 31 00:28:27.110496 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:27.110353    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-c787779cc-hg2q9"
Oct 31 00:28:27.111768 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:27.111742    4284 patch_prober.go:29] interesting pod/console-c787779cc-hg2q9 container/console namespace/openshift-console: Startup probe status=failure output="Get \"https://10.134.0.13:8443/health\": dial tcp 10.134.0.13:8443: connect: connection refused" start-of-body=
Oct 31 00:28:27.111878 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:27.111835    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-console/console-c787779cc-hg2q9" podUID="4bbd777b-3d7c-4f4c-8187-69301e8ee512" containerName="console" probeResult="failure" output="Get \"https://10.134.0.13:8443/health\": dial tcp 10.134.0.13:8443: connect: connection refused"
Oct 31 00:28:27.162559 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:27.162516    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:28:27.164549 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:27.164525    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-np54c"
Oct 31 00:28:27.176681 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:27.176657    4284 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod27f80c70_260d_492b_b0c5_8dd358f8a8f1.slice/crio-b787511639945255d423ce6b6bb4c1c66f6980b329d85cc009f7b1c063864989 WatchSource:0}: Error finding container b787511639945255d423ce6b6bb4c1c66f6980b329d85cc009f7b1c063864989: Status 404 returned error can't find the container with id b787511639945255d423ce6b6bb4c1c66f6980b329d85cc009f7b1c063864989
Oct 31 00:28:27.398331 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:27.398253    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:28:27.907239 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:27.907168    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:28:28.111731 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:28.111692    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-np54c" event={"ID":"27f80c70-260d-492b-b0c5-8dd358f8a8f1","Type":"ContainerStarted","Data":"b787511639945255d423ce6b6bb4c1c66f6980b329d85cc009f7b1c063864989"}
Oct 31 00:28:28.126992 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:28.126961    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:28:28.323111 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:28.323031    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:28:29.055806 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:29.055777    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:28:29.117515 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:29.117433    4284 generic.go:356] "Generic (PLEG): container finished" podID="27f80c70-260d-492b-b0c5-8dd358f8a8f1" containerID="a5148408a069f4a4aeae4ae52e24e9f3224f2ecb64ec3f857567002b46a4bcb8" exitCode=0
Oct 31 00:28:29.117515 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:29.117486    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-np54c" event={"ID":"27f80c70-260d-492b-b0c5-8dd358f8a8f1","Type":"ContainerDied","Data":"a5148408a069f4a4aeae4ae52e24e9f3224f2ecb64ec3f857567002b46a4bcb8"}
Oct 31 00:28:29.333665 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:29.333624    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:28:29.527735 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:29.527696    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:28:30.124912 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:30.124872    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-np54c" event={"ID":"27f80c70-260d-492b-b0c5-8dd358f8a8f1","Type":"ContainerStarted","Data":"36f814ab0026fb966a34563f3957455a15991c89f4add7886ea36e3d48cffdcb"}
Oct 31 00:28:30.125496 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:30.124936    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-np54c" event={"ID":"27f80c70-260d-492b-b0c5-8dd358f8a8f1","Type":"ContainerStarted","Data":"37190eec464fd1d204d852fcaee9f1f8c093f094c677ecb653f8a609cb34a9e0"}
Oct 31 00:28:30.265813 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:30.265750    4284 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-np54c" podStartSLOduration=2.5211148679999997 podStartE2EDuration="4.26572947s" totalImagesPullingTime="1.661303925s" totalInitContainerRuntime="83.310677ms" isStatefulPod=true podCreationTimestamp="2026-10-31 00:28:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:28:30.264273877 +0000 UTC m=+24.409280794" watchObservedRunningTime="2026-10-31 00:28:30.26572947 +0000 UTC m=+24.410736387"
Oct 31 00:28:30.378129 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:30.378049    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:28:30.581353 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:30.581316    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:28:31.090322 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:31.090287    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-gnstv"
Oct 31 00:28:31.276958 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:31.276921    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:28:31.452733 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:31.452644    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:28:31.632429 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:31.632384    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:28:31.797004 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:31.796888    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:28:32.877233 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:32.877182    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:28:33.031771 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:33.031731    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:28:34.200225 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:34.200180    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:28:34.422756 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:34.422704    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:28:34.518070 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:34.517998    4284 patch_prober.go:29] interesting pod/image-registry-85f776cbc8-lr9wd container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Oct 31 00:28:34.518070 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:34.518047    4284 prober.go:122] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-85f776cbc8-lr9wd" podUID="18948a7b-85a1-4ab0-b61b-ee42cf31b815" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 31 00:28:35.147431 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:35.147395    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-6dd789c8b7-fd796" event={"ID":"308c723a-b97c-4acd-a65e-71dd5b389723","Type":"ContainerStarted","Data":"f57b81a7eeb31b48bc8f946f578ad59c9774149834c39384807bf7609387fb0a"}
Oct 31 00:28:35.147959 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:35.147906    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-6dd789c8b7-fd796"
Oct 31 00:28:35.155344 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:35.155307    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-6dd789c8b7-fd796"
Oct 31 00:28:35.355703 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:35.355640    4284 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-console/downloads-6dd789c8b7-fd796" podStartSLOduration=2.801604694 podStartE2EDuration="21.35562352s" totalImagesPullingTime="18.554018826s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 00:28:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:28:35.307639891 +0000 UTC m=+29.452646808" watchObservedRunningTime="2026-10-31 00:28:35.35562352 +0000 UTC m=+29.500630436"
Oct 31 00:28:36.261294 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:36.261249    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:28:36.453182 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:36.453141    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:28:37.116136 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:37.116103    4284 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-c787779cc-hg2q9"
Oct 31 00:28:37.120541 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:37.120516    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-c787779cc-hg2q9"
Oct 31 00:28:38.156023 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:38.155984    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:28:38.343430 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:38.343391    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:28:38.481932 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:38.481862    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-85f776cbc8-lr9wd"
Oct 31 00:28:38.511182 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:38.511144    4284 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-console/console-687df98b6b-7szfw"]
Oct 31 00:28:38.541624 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:38.541594    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-687df98b6b-7szfw"
Oct 31 00:28:38.582779 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:38.582749    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-687df98b6b-7szfw"]
Oct 31 00:28:38.673809 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:38.673769    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/56309073-dd28-4b64-a190-21a11522bc6a-oauth-serving-cert\") pod \"console-687df98b6b-7szfw\" (UID: \"56309073-dd28-4b64-a190-21a11522bc6a\") " pod="openshift-console/console-687df98b6b-7szfw"
Oct 31 00:28:38.673979 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:38.673840    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/56309073-dd28-4b64-a190-21a11522bc6a-console-serving-cert\") pod \"console-687df98b6b-7szfw\" (UID: \"56309073-dd28-4b64-a190-21a11522bc6a\") " pod="openshift-console/console-687df98b6b-7szfw"
Oct 31 00:28:38.674042 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:38.673977    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/56309073-dd28-4b64-a190-21a11522bc6a-console-config\") pod \"console-687df98b6b-7szfw\" (UID: \"56309073-dd28-4b64-a190-21a11522bc6a\") " pod="openshift-console/console-687df98b6b-7szfw"
Oct 31 00:28:38.674042 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:38.674006    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/56309073-dd28-4b64-a190-21a11522bc6a-trusted-ca-bundle\") pod \"console-687df98b6b-7szfw\" (UID: \"56309073-dd28-4b64-a190-21a11522bc6a\") " pod="openshift-console/console-687df98b6b-7szfw"
Oct 31 00:28:38.674042 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:38.674032    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/56309073-dd28-4b64-a190-21a11522bc6a-console-oauth-config\") pod \"console-687df98b6b-7szfw\" (UID: \"56309073-dd28-4b64-a190-21a11522bc6a\") " pod="openshift-console/console-687df98b6b-7szfw"
Oct 31 00:28:38.674170 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:38.674137    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9r2wp\" (UniqueName: \"kubernetes.io/projected/56309073-dd28-4b64-a190-21a11522bc6a-kube-api-access-9r2wp\") pod \"console-687df98b6b-7szfw\" (UID: \"56309073-dd28-4b64-a190-21a11522bc6a\") " pod="openshift-console/console-687df98b6b-7szfw"
Oct 31 00:28:38.674347 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:38.674203    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/56309073-dd28-4b64-a190-21a11522bc6a-tmp\") pod \"console-687df98b6b-7szfw\" (UID: \"56309073-dd28-4b64-a190-21a11522bc6a\") " pod="openshift-console/console-687df98b6b-7szfw"
Oct 31 00:28:38.674431 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:38.674369    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/56309073-dd28-4b64-a190-21a11522bc6a-service-ca\") pod \"console-687df98b6b-7szfw\" (UID: \"56309073-dd28-4b64-a190-21a11522bc6a\") " pod="openshift-console/console-687df98b6b-7szfw"
Oct 31 00:28:38.703384 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:38.703310    4284 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Oct 31 00:28:38.774334 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:38.774233    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:28:38.775145 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:38.775119    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/56309073-dd28-4b64-a190-21a11522bc6a-console-serving-cert\") pod \"console-687df98b6b-7szfw\" (UID: \"56309073-dd28-4b64-a190-21a11522bc6a\") " pod="openshift-console/console-687df98b6b-7szfw"
Oct 31 00:28:38.775267 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:38.775165    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/56309073-dd28-4b64-a190-21a11522bc6a-console-config\") pod \"console-687df98b6b-7szfw\" (UID: \"56309073-dd28-4b64-a190-21a11522bc6a\") " pod="openshift-console/console-687df98b6b-7szfw"
Oct 31 00:28:38.775267 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:38.775186    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/56309073-dd28-4b64-a190-21a11522bc6a-trusted-ca-bundle\") pod \"console-687df98b6b-7szfw\" (UID: \"56309073-dd28-4b64-a190-21a11522bc6a\") " pod="openshift-console/console-687df98b6b-7szfw"
Oct 31 00:28:38.775267 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:38.775228    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/56309073-dd28-4b64-a190-21a11522bc6a-console-oauth-config\") pod \"console-687df98b6b-7szfw\" (UID: \"56309073-dd28-4b64-a190-21a11522bc6a\") " pod="openshift-console/console-687df98b6b-7szfw"
Oct 31 00:28:38.775267 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:38.775250    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9r2wp\" (UniqueName: \"kubernetes.io/projected/56309073-dd28-4b64-a190-21a11522bc6a-kube-api-access-9r2wp\") pod \"console-687df98b6b-7szfw\" (UID: \"56309073-dd28-4b64-a190-21a11522bc6a\") " pod="openshift-console/console-687df98b6b-7szfw"
Oct 31 00:28:38.775476 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:38.775291    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/56309073-dd28-4b64-a190-21a11522bc6a-tmp\") pod \"console-687df98b6b-7szfw\" (UID: \"56309073-dd28-4b64-a190-21a11522bc6a\") " pod="openshift-console/console-687df98b6b-7szfw"
Oct 31 00:28:38.775476 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:38.775315    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/56309073-dd28-4b64-a190-21a11522bc6a-service-ca\") pod \"console-687df98b6b-7szfw\" (UID: \"56309073-dd28-4b64-a190-21a11522bc6a\") " pod="openshift-console/console-687df98b6b-7szfw"
Oct 31 00:28:38.775476 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:38.775345    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/56309073-dd28-4b64-a190-21a11522bc6a-oauth-serving-cert\") pod \"console-687df98b6b-7szfw\" (UID: \"56309073-dd28-4b64-a190-21a11522bc6a\") " pod="openshift-console/console-687df98b6b-7szfw"
Oct 31 00:28:38.775949 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:38.775924    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/56309073-dd28-4b64-a190-21a11522bc6a-console-config\") pod \"console-687df98b6b-7szfw\" (UID: \"56309073-dd28-4b64-a190-21a11522bc6a\") " pod="openshift-console/console-687df98b6b-7szfw"
Oct 31 00:28:38.776239 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:38.776200    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/56309073-dd28-4b64-a190-21a11522bc6a-tmp\") pod \"console-687df98b6b-7szfw\" (UID: \"56309073-dd28-4b64-a190-21a11522bc6a\") " pod="openshift-console/console-687df98b6b-7szfw"
Oct 31 00:28:38.776545 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:38.776492    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/56309073-dd28-4b64-a190-21a11522bc6a-trusted-ca-bundle\") pod \"console-687df98b6b-7szfw\" (UID: \"56309073-dd28-4b64-a190-21a11522bc6a\") " pod="openshift-console/console-687df98b6b-7szfw"
Oct 31 00:28:38.776754 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:38.776725    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/56309073-dd28-4b64-a190-21a11522bc6a-service-ca\") pod \"console-687df98b6b-7szfw\" (UID: \"56309073-dd28-4b64-a190-21a11522bc6a\") " pod="openshift-console/console-687df98b6b-7szfw"
Oct 31 00:28:38.777119 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:38.777093    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/56309073-dd28-4b64-a190-21a11522bc6a-oauth-serving-cert\") pod \"console-687df98b6b-7szfw\" (UID: \"56309073-dd28-4b64-a190-21a11522bc6a\") " pod="openshift-console/console-687df98b6b-7szfw"
Oct 31 00:28:38.778246 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:38.778206    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/56309073-dd28-4b64-a190-21a11522bc6a-console-serving-cert\") pod \"console-687df98b6b-7szfw\" (UID: \"56309073-dd28-4b64-a190-21a11522bc6a\") " pod="openshift-console/console-687df98b6b-7szfw"
Oct 31 00:28:38.778735 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:38.778710    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/56309073-dd28-4b64-a190-21a11522bc6a-console-oauth-config\") pod \"console-687df98b6b-7szfw\" (UID: \"56309073-dd28-4b64-a190-21a11522bc6a\") " pod="openshift-console/console-687df98b6b-7szfw"
Oct 31 00:28:38.843793 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:38.843761    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9r2wp\" (UniqueName: \"kubernetes.io/projected/56309073-dd28-4b64-a190-21a11522bc6a-kube-api-access-9r2wp\") pod \"console-687df98b6b-7szfw\" (UID: \"56309073-dd28-4b64-a190-21a11522bc6a\") " pod="openshift-console/console-687df98b6b-7szfw"
Oct 31 00:28:38.869493 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:38.869457    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-687df98b6b-7szfw"
Oct 31 00:28:38.943453 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:38.943404    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:28:39.106123 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:39.106033    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:28:39.250593 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:39.250507    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:28:39.283824 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:39.283796    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-687df98b6b-7szfw"]
Oct 31 00:28:39.285817 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:28:39.285784    4284 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod56309073_dd28_4b64_a190_21a11522bc6a.slice/crio-01e19869887b31d7e0c7c851cc6db8374b8514c09a7ab66b31087e2ab048a2ee WatchSource:0}: Error finding container 01e19869887b31d7e0c7c851cc6db8374b8514c09a7ab66b31087e2ab048a2ee: Status 404 returned error can't find the container with id 01e19869887b31d7e0c7c851cc6db8374b8514c09a7ab66b31087e2ab048a2ee
Oct 31 00:28:40.175986 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:40.175948    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-687df98b6b-7szfw" event={"ID":"56309073-dd28-4b64-a190-21a11522bc6a","Type":"ContainerStarted","Data":"792b6540036a21938f728d09bf0cd19df5f9aae41f8bff56845e708d78fdd344"}
Oct 31 00:28:40.176171 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:40.176004    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-687df98b6b-7szfw" event={"ID":"56309073-dd28-4b64-a190-21a11522bc6a","Type":"ContainerStarted","Data":"01e19869887b31d7e0c7c851cc6db8374b8514c09a7ab66b31087e2ab048a2ee"}
Oct 31 00:28:42.511713 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:42.511672    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:28:42.706411 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:42.706375    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:28:42.920301 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:42.920265    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-jktkx"
Oct 31 00:28:43.032792 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:43.032729    4284 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-console/console-687df98b6b-7szfw" podStartSLOduration=5.032706959 podStartE2EDuration="5.032706959s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 00:28:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:28:40.292870124 +0000 UTC m=+34.437877040" watchObservedRunningTime="2026-10-31 00:28:43.032706959 +0000 UTC m=+37.177713876"
Oct 31 00:28:44.102776 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:44.102747    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:28:44.247419 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:44.247383    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:28:44.698272 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:44.698238    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:28:44.896115 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:44.896071    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:28:45.301753 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:45.301715    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:28:45.451579 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:45.451533    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:28:47.013066 ip-10-0-132-173 kubenswrapper[4284]: E1031 00:28:47.012987    4284 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fd8sp" containerName="extractor"
Oct 31 00:28:47.657368 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:47.657333    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:28:47.844205 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:47.844174    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:28:48.869759 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:48.869714    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-687df98b6b-7szfw"
Oct 31 00:28:48.870179 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:48.869787    4284 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-687df98b6b-7szfw"
Oct 31 00:28:48.874444 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:48.874423    4284 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-687df98b6b-7szfw"
Oct 31 00:28:48.945278 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:48.945249    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:28:49.114563 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:49.114519    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:28:49.213934 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:49.213905    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-687df98b6b-7szfw"
Oct 31 00:28:49.398247 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:49.398206    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-console/console-c787779cc-hg2q9"]
Oct 31 00:28:51.631898 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:51.631870    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:28:51.780937 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:51.780898    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:28:51.930736 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:28:51.930705    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:29:02.869984 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:29:02.869945    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:29:03.027293 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:29:03.027255    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:29:04.669892 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:29:04.669855    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:29:04.827597 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:29:04.827543    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:29:04.981492 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:29:04.981409    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:29:09.481704 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:29:09.481668    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:29:09.635092 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:29:09.635058    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:29:09.777354 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:29:09.777274    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:29:10.595733 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:29:10.595690    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:29:10.782062 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:29:10.782028    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:29:14.418737 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:29:14.418690    4284 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-console/console-c787779cc-hg2q9" podUID="4bbd777b-3d7c-4f4c-8187-69301e8ee512" containerName="console" containerID="cri-o://b2c94cc3eb599995496d52c2f8a9c7288fa0539dd3550af99beb50147f821ce8" gracePeriod=15
Oct 31 00:29:14.682649 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:29:14.682590    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-c787779cc-hg2q9"
Oct 31 00:29:14.750673 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:29:14.750643    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/4bbd777b-3d7c-4f4c-8187-69301e8ee512-console-config\" (UniqueName: \"kubernetes.io/configmap/4bbd777b-3d7c-4f4c-8187-69301e8ee512-console-config\") pod \"4bbd777b-3d7c-4f4c-8187-69301e8ee512\" (UID: \"4bbd777b-3d7c-4f4c-8187-69301e8ee512\") "
Oct 31 00:29:14.750837 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:29:14.750685    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/4bbd777b-3d7c-4f4c-8187-69301e8ee512-service-ca\" (UniqueName: \"kubernetes.io/configmap/4bbd777b-3d7c-4f4c-8187-69301e8ee512-service-ca\") pod \"4bbd777b-3d7c-4f4c-8187-69301e8ee512\" (UID: \"4bbd777b-3d7c-4f4c-8187-69301e8ee512\") "
Oct 31 00:29:14.750837 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:29:14.750710    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/4bbd777b-3d7c-4f4c-8187-69301e8ee512-console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4bbd777b-3d7c-4f4c-8187-69301e8ee512-console-serving-cert\") pod \"4bbd777b-3d7c-4f4c-8187-69301e8ee512\" (UID: \"4bbd777b-3d7c-4f4c-8187-69301e8ee512\") "
Oct 31 00:29:14.750837 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:29:14.750733    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/4bbd777b-3d7c-4f4c-8187-69301e8ee512-oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4bbd777b-3d7c-4f4c-8187-69301e8ee512-oauth-serving-cert\") pod \"4bbd777b-3d7c-4f4c-8187-69301e8ee512\" (UID: \"4bbd777b-3d7c-4f4c-8187-69301e8ee512\") "
Oct 31 00:29:14.750837 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:29:14.750754    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4bbd777b-3d7c-4f4c-8187-69301e8ee512-tmp\" (UniqueName: \"kubernetes.io/empty-dir/4bbd777b-3d7c-4f4c-8187-69301e8ee512-tmp\") pod \"4bbd777b-3d7c-4f4c-8187-69301e8ee512\" (UID: \"4bbd777b-3d7c-4f4c-8187-69301e8ee512\") "
Oct 31 00:29:14.750837 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:29:14.750784    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4bbd777b-3d7c-4f4c-8187-69301e8ee512-kube-api-access-d26g6\" (UniqueName: \"kubernetes.io/projected/4bbd777b-3d7c-4f4c-8187-69301e8ee512-kube-api-access-d26g6\") pod \"4bbd777b-3d7c-4f4c-8187-69301e8ee512\" (UID: \"4bbd777b-3d7c-4f4c-8187-69301e8ee512\") "
Oct 31 00:29:14.750837 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:29:14.750804    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/4bbd777b-3d7c-4f4c-8187-69301e8ee512-console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4bbd777b-3d7c-4f4c-8187-69301e8ee512-console-oauth-config\") pod \"4bbd777b-3d7c-4f4c-8187-69301e8ee512\" (UID: \"4bbd777b-3d7c-4f4c-8187-69301e8ee512\") "
Oct 31 00:29:14.751230 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:29:14.751176    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4bbd777b-3d7c-4f4c-8187-69301e8ee512-tmp" pod "4bbd777b-3d7c-4f4c-8187-69301e8ee512" (UID: "4bbd777b-3d7c-4f4c-8187-69301e8ee512"). InnerVolumeSpecName "tmp". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:29:14.751309 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:29:14.751233    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4bbd777b-3d7c-4f4c-8187-69301e8ee512-service-ca" pod "4bbd777b-3d7c-4f4c-8187-69301e8ee512" (UID: "4bbd777b-3d7c-4f4c-8187-69301e8ee512"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 00:29:14.751309 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:29:14.751185    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4bbd777b-3d7c-4f4c-8187-69301e8ee512-console-config" pod "4bbd777b-3d7c-4f4c-8187-69301e8ee512" (UID: "4bbd777b-3d7c-4f4c-8187-69301e8ee512"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 00:29:14.751309 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:29:14.751194    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4bbd777b-3d7c-4f4c-8187-69301e8ee512-oauth-serving-cert" pod "4bbd777b-3d7c-4f4c-8187-69301e8ee512" (UID: "4bbd777b-3d7c-4f4c-8187-69301e8ee512"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 00:29:14.753173 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:29:14.753148    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4bbd777b-3d7c-4f4c-8187-69301e8ee512-console-serving-cert" pod "4bbd777b-3d7c-4f4c-8187-69301e8ee512" (UID: "4bbd777b-3d7c-4f4c-8187-69301e8ee512"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 00:29:14.753173 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:29:14.753151    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4bbd777b-3d7c-4f4c-8187-69301e8ee512-kube-api-access-d26g6" pod "4bbd777b-3d7c-4f4c-8187-69301e8ee512" (UID: "4bbd777b-3d7c-4f4c-8187-69301e8ee512"). InnerVolumeSpecName "kube-api-access-d26g6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 00:29:14.753173 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:29:14.753164    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4bbd777b-3d7c-4f4c-8187-69301e8ee512-console-oauth-config" pod "4bbd777b-3d7c-4f4c-8187-69301e8ee512" (UID: "4bbd777b-3d7c-4f4c-8187-69301e8ee512"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 00:29:14.851872 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:29:14.851836    4284 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4bbd777b-3d7c-4f4c-8187-69301e8ee512-console-serving-cert\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:29:14.851872 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:29:14.851869    4284 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4bbd777b-3d7c-4f4c-8187-69301e8ee512-oauth-serving-cert\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:29:14.851872 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:29:14.851880    4284 reconciler_common.go:299] "Volume detached for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4bbd777b-3d7c-4f4c-8187-69301e8ee512-tmp\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:29:14.852097 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:29:14.851889    4284 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d26g6\" (UniqueName: \"kubernetes.io/projected/4bbd777b-3d7c-4f4c-8187-69301e8ee512-kube-api-access-d26g6\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:29:14.852097 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:29:14.851899    4284 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4bbd777b-3d7c-4f4c-8187-69301e8ee512-console-oauth-config\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:29:14.852097 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:29:14.851908    4284 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4bbd777b-3d7c-4f4c-8187-69301e8ee512-console-config\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:29:14.852097 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:29:14.851918    4284 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4bbd777b-3d7c-4f4c-8187-69301e8ee512-service-ca\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:29:15.297061 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:29:15.297030    4284 generic.go:356] "Generic (PLEG): container finished" podID="4bbd777b-3d7c-4f4c-8187-69301e8ee512" containerID="b2c94cc3eb599995496d52c2f8a9c7288fa0539dd3550af99beb50147f821ce8" exitCode=0
Oct 31 00:29:15.297281 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:29:15.297080    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-c787779cc-hg2q9"
Oct 31 00:29:15.297281 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:29:15.297089    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-c787779cc-hg2q9" event={"ID":"4bbd777b-3d7c-4f4c-8187-69301e8ee512","Type":"ContainerDied","Data":"b2c94cc3eb599995496d52c2f8a9c7288fa0539dd3550af99beb50147f821ce8"}
Oct 31 00:29:15.297281 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:29:15.297131    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-c787779cc-hg2q9" event={"ID":"4bbd777b-3d7c-4f4c-8187-69301e8ee512","Type":"ContainerDied","Data":"1b3189fab4d98112899a582b111937f81724fb9afd67af1aed4503670f73fe21"}
Oct 31 00:29:15.297281 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:29:15.297150    4284 scope.go:122] "RemoveContainer" containerID="b2c94cc3eb599995496d52c2f8a9c7288fa0539dd3550af99beb50147f821ce8"
Oct 31 00:29:15.305647 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:29:15.305632    4284 scope.go:122] "RemoveContainer" containerID="b2c94cc3eb599995496d52c2f8a9c7288fa0539dd3550af99beb50147f821ce8"
Oct 31 00:29:15.305935 ip-10-0-132-173 kubenswrapper[4284]: E1031 00:29:15.305915    4284 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b2c94cc3eb599995496d52c2f8a9c7288fa0539dd3550af99beb50147f821ce8\": container with ID starting with b2c94cc3eb599995496d52c2f8a9c7288fa0539dd3550af99beb50147f821ce8 not found: ID does not exist" containerID="b2c94cc3eb599995496d52c2f8a9c7288fa0539dd3550af99beb50147f821ce8"
Oct 31 00:29:15.305985 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:29:15.305946    4284 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b2c94cc3eb599995496d52c2f8a9c7288fa0539dd3550af99beb50147f821ce8"} err="failed to get container status \"b2c94cc3eb599995496d52c2f8a9c7288fa0539dd3550af99beb50147f821ce8\": rpc error: code = NotFound desc = could not find container \"b2c94cc3eb599995496d52c2f8a9c7288fa0539dd3550af99beb50147f821ce8\": container with ID starting with b2c94cc3eb599995496d52c2f8a9c7288fa0539dd3550af99beb50147f821ce8 not found: ID does not exist"
Oct 31 00:29:15.454887 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:29:15.454843    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-console/console-c787779cc-hg2q9"]
Oct 31 00:29:15.502432 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:29:15.502399    4284 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-c787779cc-hg2q9"]
Oct 31 00:29:15.963990 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:29:15.963955    4284 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="4bbd777b-3d7c-4f4c-8187-69301e8ee512" path="/var/lib/kubelet/pods/4bbd777b-3d7c-4f4c-8187-69301e8ee512/volumes"
Oct 31 00:29:17.013798 ip-10-0-132-173 kubenswrapper[4284]: E1031 00:29:17.013766    4284 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fd8sp" containerName="extractor"
Oct 31 00:29:17.865138 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:29:17.865102    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:29:18.066149 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:29:18.066114    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:29:21.489374 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:29:21.489337    4284 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-console/console-7cc6cf47b7-fqmmm"]
Oct 31 00:29:21.489859 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:29:21.489628    4284 cpu_manager.go:414] "containerMap: removing container" podUID="4bbd777b-3d7c-4f4c-8187-69301e8ee512" containerName="console"
Oct 31 00:29:21.489859 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:29:21.489641    4284 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4bbd777b-3d7c-4f4c-8187-69301e8ee512" containerName="console"
Oct 31 00:29:21.489859 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:29:21.489700    4284 memory_manager.go:365] "RemoveStaleState removing state" podUID="4bbd777b-3d7c-4f4c-8187-69301e8ee512" containerName="console"
Oct 31 00:29:21.492487 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:29:21.492466    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7cc6cf47b7-fqmmm"
Oct 31 00:29:21.496262 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:29:21.496240    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a8ec2bd2-73fb-4d63-b6c3-0ab12370d2ee-trusted-ca-bundle\") pod \"console-7cc6cf47b7-fqmmm\" (UID: \"a8ec2bd2-73fb-4d63-b6c3-0ab12370d2ee\") " pod="openshift-console/console-7cc6cf47b7-fqmmm"
Oct 31 00:29:21.496391 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:29:21.496274    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a8ec2bd2-73fb-4d63-b6c3-0ab12370d2ee-console-config\") pod \"console-7cc6cf47b7-fqmmm\" (UID: \"a8ec2bd2-73fb-4d63-b6c3-0ab12370d2ee\") " pod="openshift-console/console-7cc6cf47b7-fqmmm"
Oct 31 00:29:21.496391 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:29:21.496292    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z9cll\" (UniqueName: \"kubernetes.io/projected/a8ec2bd2-73fb-4d63-b6c3-0ab12370d2ee-kube-api-access-z9cll\") pod \"console-7cc6cf47b7-fqmmm\" (UID: \"a8ec2bd2-73fb-4d63-b6c3-0ab12370d2ee\") " pod="openshift-console/console-7cc6cf47b7-fqmmm"
Oct 31 00:29:21.496391 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:29:21.496314    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a8ec2bd2-73fb-4d63-b6c3-0ab12370d2ee-service-ca\") pod \"console-7cc6cf47b7-fqmmm\" (UID: \"a8ec2bd2-73fb-4d63-b6c3-0ab12370d2ee\") " pod="openshift-console/console-7cc6cf47b7-fqmmm"
Oct 31 00:29:21.496551 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:29:21.496428    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a8ec2bd2-73fb-4d63-b6c3-0ab12370d2ee-console-serving-cert\") pod \"console-7cc6cf47b7-fqmmm\" (UID: \"a8ec2bd2-73fb-4d63-b6c3-0ab12370d2ee\") " pod="openshift-console/console-7cc6cf47b7-fqmmm"
Oct 31 00:29:21.496551 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:29:21.496457    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a8ec2bd2-73fb-4d63-b6c3-0ab12370d2ee-oauth-serving-cert\") pod \"console-7cc6cf47b7-fqmmm\" (UID: \"a8ec2bd2-73fb-4d63-b6c3-0ab12370d2ee\") " pod="openshift-console/console-7cc6cf47b7-fqmmm"
Oct 31 00:29:21.496551 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:29:21.496496    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a8ec2bd2-73fb-4d63-b6c3-0ab12370d2ee-console-oauth-config\") pod \"console-7cc6cf47b7-fqmmm\" (UID: \"a8ec2bd2-73fb-4d63-b6c3-0ab12370d2ee\") " pod="openshift-console/console-7cc6cf47b7-fqmmm"
Oct 31 00:29:21.496551 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:29:21.496534    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a8ec2bd2-73fb-4d63-b6c3-0ab12370d2ee-tmp\") pod \"console-7cc6cf47b7-fqmmm\" (UID: \"a8ec2bd2-73fb-4d63-b6c3-0ab12370d2ee\") " pod="openshift-console/console-7cc6cf47b7-fqmmm"
Oct 31 00:29:21.549304 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:29:21.549268    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7cc6cf47b7-fqmmm"]
Oct 31 00:29:21.596835 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:29:21.596805    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a8ec2bd2-73fb-4d63-b6c3-0ab12370d2ee-console-config\") pod \"console-7cc6cf47b7-fqmmm\" (UID: \"a8ec2bd2-73fb-4d63-b6c3-0ab12370d2ee\") " pod="openshift-console/console-7cc6cf47b7-fqmmm"
Oct 31 00:29:21.596835 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:29:21.596836    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z9cll\" (UniqueName: \"kubernetes.io/projected/a8ec2bd2-73fb-4d63-b6c3-0ab12370d2ee-kube-api-access-z9cll\") pod \"console-7cc6cf47b7-fqmmm\" (UID: \"a8ec2bd2-73fb-4d63-b6c3-0ab12370d2ee\") " pod="openshift-console/console-7cc6cf47b7-fqmmm"
Oct 31 00:29:21.597094 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:29:21.596862    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a8ec2bd2-73fb-4d63-b6c3-0ab12370d2ee-service-ca\") pod \"console-7cc6cf47b7-fqmmm\" (UID: \"a8ec2bd2-73fb-4d63-b6c3-0ab12370d2ee\") " pod="openshift-console/console-7cc6cf47b7-fqmmm"
Oct 31 00:29:21.597094 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:29:21.596881    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a8ec2bd2-73fb-4d63-b6c3-0ab12370d2ee-console-serving-cert\") pod \"console-7cc6cf47b7-fqmmm\" (UID: \"a8ec2bd2-73fb-4d63-b6c3-0ab12370d2ee\") " pod="openshift-console/console-7cc6cf47b7-fqmmm"
Oct 31 00:29:21.597094 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:29:21.596909    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a8ec2bd2-73fb-4d63-b6c3-0ab12370d2ee-oauth-serving-cert\") pod \"console-7cc6cf47b7-fqmmm\" (UID: \"a8ec2bd2-73fb-4d63-b6c3-0ab12370d2ee\") " pod="openshift-console/console-7cc6cf47b7-fqmmm"
Oct 31 00:29:21.597094 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:29:21.596960    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a8ec2bd2-73fb-4d63-b6c3-0ab12370d2ee-console-oauth-config\") pod \"console-7cc6cf47b7-fqmmm\" (UID: \"a8ec2bd2-73fb-4d63-b6c3-0ab12370d2ee\") " pod="openshift-console/console-7cc6cf47b7-fqmmm"
Oct 31 00:29:21.597094 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:29:21.596991    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a8ec2bd2-73fb-4d63-b6c3-0ab12370d2ee-tmp\") pod \"console-7cc6cf47b7-fqmmm\" (UID: \"a8ec2bd2-73fb-4d63-b6c3-0ab12370d2ee\") " pod="openshift-console/console-7cc6cf47b7-fqmmm"
Oct 31 00:29:21.597094 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:29:21.597022    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a8ec2bd2-73fb-4d63-b6c3-0ab12370d2ee-trusted-ca-bundle\") pod \"console-7cc6cf47b7-fqmmm\" (UID: \"a8ec2bd2-73fb-4d63-b6c3-0ab12370d2ee\") " pod="openshift-console/console-7cc6cf47b7-fqmmm"
Oct 31 00:29:21.597721 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:29:21.597696    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a8ec2bd2-73fb-4d63-b6c3-0ab12370d2ee-console-config\") pod \"console-7cc6cf47b7-fqmmm\" (UID: \"a8ec2bd2-73fb-4d63-b6c3-0ab12370d2ee\") " pod="openshift-console/console-7cc6cf47b7-fqmmm"
Oct 31 00:29:21.597896 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:29:21.597696    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a8ec2bd2-73fb-4d63-b6c3-0ab12370d2ee-oauth-serving-cert\") pod \"console-7cc6cf47b7-fqmmm\" (UID: \"a8ec2bd2-73fb-4d63-b6c3-0ab12370d2ee\") " pod="openshift-console/console-7cc6cf47b7-fqmmm"
Oct 31 00:29:21.598033 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:29:21.597734    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a8ec2bd2-73fb-4d63-b6c3-0ab12370d2ee-tmp\") pod \"console-7cc6cf47b7-fqmmm\" (UID: \"a8ec2bd2-73fb-4d63-b6c3-0ab12370d2ee\") " pod="openshift-console/console-7cc6cf47b7-fqmmm"
Oct 31 00:29:21.598033 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:29:21.597814    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a8ec2bd2-73fb-4d63-b6c3-0ab12370d2ee-service-ca\") pod \"console-7cc6cf47b7-fqmmm\" (UID: \"a8ec2bd2-73fb-4d63-b6c3-0ab12370d2ee\") " pod="openshift-console/console-7cc6cf47b7-fqmmm"
Oct 31 00:29:21.598181 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:29:21.598096    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a8ec2bd2-73fb-4d63-b6c3-0ab12370d2ee-trusted-ca-bundle\") pod \"console-7cc6cf47b7-fqmmm\" (UID: \"a8ec2bd2-73fb-4d63-b6c3-0ab12370d2ee\") " pod="openshift-console/console-7cc6cf47b7-fqmmm"
Oct 31 00:29:21.599656 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:29:21.599638    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a8ec2bd2-73fb-4d63-b6c3-0ab12370d2ee-console-oauth-config\") pod \"console-7cc6cf47b7-fqmmm\" (UID: \"a8ec2bd2-73fb-4d63-b6c3-0ab12370d2ee\") " pod="openshift-console/console-7cc6cf47b7-fqmmm"
Oct 31 00:29:21.600042 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:29:21.600020    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a8ec2bd2-73fb-4d63-b6c3-0ab12370d2ee-console-serving-cert\") pod \"console-7cc6cf47b7-fqmmm\" (UID: \"a8ec2bd2-73fb-4d63-b6c3-0ab12370d2ee\") " pod="openshift-console/console-7cc6cf47b7-fqmmm"
Oct 31 00:29:21.656685 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:29:21.656652    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z9cll\" (UniqueName: \"kubernetes.io/projected/a8ec2bd2-73fb-4d63-b6c3-0ab12370d2ee-kube-api-access-z9cll\") pod \"console-7cc6cf47b7-fqmmm\" (UID: \"a8ec2bd2-73fb-4d63-b6c3-0ab12370d2ee\") " pod="openshift-console/console-7cc6cf47b7-fqmmm"
Oct 31 00:29:21.751013 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:29:21.750929    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:29:21.802604 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:29:21.802569    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7cc6cf47b7-fqmmm"
Oct 31 00:29:21.893102 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:29:21.893071    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:29:22.121846 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:29:22.121764    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:29:22.170414 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:29:22.169957    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7cc6cf47b7-fqmmm"]
Oct 31 00:29:22.171959 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:29:22.171920    4284 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda8ec2bd2_73fb_4d63_b6c3_0ab12370d2ee.slice/crio-6a933e54f15ea6b00fbf032e87af29f0d02b20518096777fdc71756d1a13fffe WatchSource:0}: Error finding container 6a933e54f15ea6b00fbf032e87af29f0d02b20518096777fdc71756d1a13fffe: Status 404 returned error can't find the container with id 6a933e54f15ea6b00fbf032e87af29f0d02b20518096777fdc71756d1a13fffe
Oct 31 00:29:22.272248 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:29:22.272186    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:29:22.322993 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:29:22.322961    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7cc6cf47b7-fqmmm" event={"ID":"a8ec2bd2-73fb-4d63-b6c3-0ab12370d2ee","Type":"ContainerStarted","Data":"f8db77a37a8a97a4a603f9884252d9111f670e51f46194fbb40a7c7019f5ccd6"}
Oct 31 00:29:22.323170 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:29:22.323007    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7cc6cf47b7-fqmmm" event={"ID":"a8ec2bd2-73fb-4d63-b6c3-0ab12370d2ee","Type":"ContainerStarted","Data":"6a933e54f15ea6b00fbf032e87af29f0d02b20518096777fdc71756d1a13fffe"}
Oct 31 00:29:22.437168 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:29:22.437114    4284 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-console/console-7cc6cf47b7-fqmmm" podStartSLOduration=1.4370974429999999 podStartE2EDuration="1.437097443s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 00:29:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:29:22.436427926 +0000 UTC m=+76.581434842" watchObservedRunningTime="2026-10-31 00:29:22.437097443 +0000 UTC m=+76.582104359"
Oct 31 00:29:31.803433 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:29:31.803395    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7cc6cf47b7-fqmmm"
Oct 31 00:29:31.803841 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:29:31.803456    4284 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7cc6cf47b7-fqmmm"
Oct 31 00:29:31.808249 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:29:31.808199    4284 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7cc6cf47b7-fqmmm"
Oct 31 00:29:32.360991 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:29:32.360966    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7cc6cf47b7-fqmmm"
Oct 31 00:29:32.523452 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:29:32.523419    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-console/console-687df98b6b-7szfw"]
Oct 31 00:29:32.865363 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:29:32.865329    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:29:33.014832 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:29:33.014795    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:29:47.012540 ip-10-0-132-173 kubenswrapper[4284]: E1031 00:29:47.012503    4284 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fd8sp" containerName="extractor"
Oct 31 00:29:47.698072 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:29:47.698032    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:29:48.253573 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:29:48.253543    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:29:49.275035 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:29:49.275004    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:29:57.542516 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:29:57.542476    4284 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-console/console-687df98b6b-7szfw" podUID="56309073-dd28-4b64-a190-21a11522bc6a" containerName="console" containerID="cri-o://792b6540036a21938f728d09bf0cd19df5f9aae41f8bff56845e708d78fdd344" gracePeriod=15
Oct 31 00:29:57.789575 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:29:57.789553    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-687df98b6b-7szfw"
Oct 31 00:29:57.850964 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:29:57.850877    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/56309073-dd28-4b64-a190-21a11522bc6a-oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/56309073-dd28-4b64-a190-21a11522bc6a-oauth-serving-cert\") pod \"56309073-dd28-4b64-a190-21a11522bc6a\" (UID: \"56309073-dd28-4b64-a190-21a11522bc6a\") "
Oct 31 00:29:57.850964 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:29:57.850922    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/56309073-dd28-4b64-a190-21a11522bc6a-tmp\" (UniqueName: \"kubernetes.io/empty-dir/56309073-dd28-4b64-a190-21a11522bc6a-tmp\") pod \"56309073-dd28-4b64-a190-21a11522bc6a\" (UID: \"56309073-dd28-4b64-a190-21a11522bc6a\") "
Oct 31 00:29:57.850964 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:29:57.850967    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/56309073-dd28-4b64-a190-21a11522bc6a-console-serving-cert\" (UniqueName: \"kubernetes.io/secret/56309073-dd28-4b64-a190-21a11522bc6a-console-serving-cert\") pod \"56309073-dd28-4b64-a190-21a11522bc6a\" (UID: \"56309073-dd28-4b64-a190-21a11522bc6a\") "
Oct 31 00:29:57.851257 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:29:57.850986    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/56309073-dd28-4b64-a190-21a11522bc6a-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/56309073-dd28-4b64-a190-21a11522bc6a-trusted-ca-bundle\") pod \"56309073-dd28-4b64-a190-21a11522bc6a\" (UID: \"56309073-dd28-4b64-a190-21a11522bc6a\") "
Oct 31 00:29:57.851257 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:29:57.851009    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/56309073-dd28-4b64-a190-21a11522bc6a-console-config\" (UniqueName: \"kubernetes.io/configmap/56309073-dd28-4b64-a190-21a11522bc6a-console-config\") pod \"56309073-dd28-4b64-a190-21a11522bc6a\" (UID: \"56309073-dd28-4b64-a190-21a11522bc6a\") "
Oct 31 00:29:57.851257 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:29:57.851035    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/56309073-dd28-4b64-a190-21a11522bc6a-service-ca\" (UniqueName: \"kubernetes.io/configmap/56309073-dd28-4b64-a190-21a11522bc6a-service-ca\") pod \"56309073-dd28-4b64-a190-21a11522bc6a\" (UID: \"56309073-dd28-4b64-a190-21a11522bc6a\") "
Oct 31 00:29:57.851257 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:29:57.851067    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/56309073-dd28-4b64-a190-21a11522bc6a-console-oauth-config\" (UniqueName: \"kubernetes.io/secret/56309073-dd28-4b64-a190-21a11522bc6a-console-oauth-config\") pod \"56309073-dd28-4b64-a190-21a11522bc6a\" (UID: \"56309073-dd28-4b64-a190-21a11522bc6a\") "
Oct 31 00:29:57.851257 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:29:57.851101    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/56309073-dd28-4b64-a190-21a11522bc6a-kube-api-access-9r2wp\" (UniqueName: \"kubernetes.io/projected/56309073-dd28-4b64-a190-21a11522bc6a-kube-api-access-9r2wp\") pod \"56309073-dd28-4b64-a190-21a11522bc6a\" (UID: \"56309073-dd28-4b64-a190-21a11522bc6a\") "
Oct 31 00:29:57.851573 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:29:57.851404    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/56309073-dd28-4b64-a190-21a11522bc6a-tmp" pod "56309073-dd28-4b64-a190-21a11522bc6a" (UID: "56309073-dd28-4b64-a190-21a11522bc6a"). InnerVolumeSpecName "tmp". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:29:57.851573 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:29:57.851434    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/56309073-dd28-4b64-a190-21a11522bc6a-oauth-serving-cert" pod "56309073-dd28-4b64-a190-21a11522bc6a" (UID: "56309073-dd28-4b64-a190-21a11522bc6a"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 00:29:57.851573 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:29:57.851496    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/56309073-dd28-4b64-a190-21a11522bc6a-service-ca" pod "56309073-dd28-4b64-a190-21a11522bc6a" (UID: "56309073-dd28-4b64-a190-21a11522bc6a"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 00:29:57.851573 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:29:57.851514    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/56309073-dd28-4b64-a190-21a11522bc6a-trusted-ca-bundle" pod "56309073-dd28-4b64-a190-21a11522bc6a" (UID: "56309073-dd28-4b64-a190-21a11522bc6a"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 00:29:57.851843 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:29:57.851816    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/56309073-dd28-4b64-a190-21a11522bc6a-console-config" pod "56309073-dd28-4b64-a190-21a11522bc6a" (UID: "56309073-dd28-4b64-a190-21a11522bc6a"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 00:29:57.853381 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:29:57.853347    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/56309073-dd28-4b64-a190-21a11522bc6a-console-oauth-config" pod "56309073-dd28-4b64-a190-21a11522bc6a" (UID: "56309073-dd28-4b64-a190-21a11522bc6a"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 00:29:57.853480 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:29:57.853395    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/56309073-dd28-4b64-a190-21a11522bc6a-console-serving-cert" pod "56309073-dd28-4b64-a190-21a11522bc6a" (UID: "56309073-dd28-4b64-a190-21a11522bc6a"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 00:29:57.853480 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:29:57.853412    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/56309073-dd28-4b64-a190-21a11522bc6a-kube-api-access-9r2wp" pod "56309073-dd28-4b64-a190-21a11522bc6a" (UID: "56309073-dd28-4b64-a190-21a11522bc6a"). InnerVolumeSpecName "kube-api-access-9r2wp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 00:29:57.951908 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:29:57.951870    4284 reconciler_common.go:299] "Volume detached for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/56309073-dd28-4b64-a190-21a11522bc6a-tmp\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:29:57.951908 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:29:57.951901    4284 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/56309073-dd28-4b64-a190-21a11522bc6a-console-serving-cert\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:29:57.951908 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:29:57.951911    4284 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/56309073-dd28-4b64-a190-21a11522bc6a-trusted-ca-bundle\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:29:57.952133 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:29:57.951921    4284 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/56309073-dd28-4b64-a190-21a11522bc6a-console-config\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:29:57.952133 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:29:57.951931    4284 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/56309073-dd28-4b64-a190-21a11522bc6a-service-ca\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:29:57.952133 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:29:57.951938    4284 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/56309073-dd28-4b64-a190-21a11522bc6a-console-oauth-config\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:29:57.952133 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:29:57.951947    4284 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9r2wp\" (UniqueName: \"kubernetes.io/projected/56309073-dd28-4b64-a190-21a11522bc6a-kube-api-access-9r2wp\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:29:57.952133 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:29:57.951958    4284 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/56309073-dd28-4b64-a190-21a11522bc6a-oauth-serving-cert\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:29:58.264752 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:29:58.264722    4284 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-console/console-5c6cb57997-z2szp"]
Oct 31 00:29:58.265005 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:29:58.264992    4284 cpu_manager.go:414] "containerMap: removing container" podUID="56309073-dd28-4b64-a190-21a11522bc6a" containerName="console"
Oct 31 00:29:58.265048 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:29:58.265007    4284 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="56309073-dd28-4b64-a190-21a11522bc6a" containerName="console"
Oct 31 00:29:58.265048 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:29:58.265047    4284 memory_manager.go:365] "RemoveStaleState removing state" podUID="56309073-dd28-4b64-a190-21a11522bc6a" containerName="console"
Oct 31 00:29:58.267089 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:29:58.267072    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5c6cb57997-z2szp"
Oct 31 00:29:58.355753 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:29:58.355711    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/01ee1aab-bb64-40eb-bc77-b775bd6e4ea3-console-serving-cert\") pod \"console-5c6cb57997-z2szp\" (UID: \"01ee1aab-bb64-40eb-bc77-b775bd6e4ea3\") " pod="openshift-console/console-5c6cb57997-z2szp"
Oct 31 00:29:58.355753 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:29:58.355754    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/01ee1aab-bb64-40eb-bc77-b775bd6e4ea3-service-ca\") pod \"console-5c6cb57997-z2szp\" (UID: \"01ee1aab-bb64-40eb-bc77-b775bd6e4ea3\") " pod="openshift-console/console-5c6cb57997-z2szp"
Oct 31 00:29:58.355960 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:29:58.355773    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/01ee1aab-bb64-40eb-bc77-b775bd6e4ea3-tmp\") pod \"console-5c6cb57997-z2szp\" (UID: \"01ee1aab-bb64-40eb-bc77-b775bd6e4ea3\") " pod="openshift-console/console-5c6cb57997-z2szp"
Oct 31 00:29:58.355960 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:29:58.355906    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kp8zq\" (UniqueName: \"kubernetes.io/projected/01ee1aab-bb64-40eb-bc77-b775bd6e4ea3-kube-api-access-kp8zq\") pod \"console-5c6cb57997-z2szp\" (UID: \"01ee1aab-bb64-40eb-bc77-b775bd6e4ea3\") " pod="openshift-console/console-5c6cb57997-z2szp"
Oct 31 00:29:58.355960 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:29:58.355935    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/01ee1aab-bb64-40eb-bc77-b775bd6e4ea3-trusted-ca-bundle\") pod \"console-5c6cb57997-z2szp\" (UID: \"01ee1aab-bb64-40eb-bc77-b775bd6e4ea3\") " pod="openshift-console/console-5c6cb57997-z2szp"
Oct 31 00:29:58.355960 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:29:58.355952    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/01ee1aab-bb64-40eb-bc77-b775bd6e4ea3-oauth-serving-cert\") pod \"console-5c6cb57997-z2szp\" (UID: \"01ee1aab-bb64-40eb-bc77-b775bd6e4ea3\") " pod="openshift-console/console-5c6cb57997-z2szp"
Oct 31 00:29:58.356080 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:29:58.355972    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/01ee1aab-bb64-40eb-bc77-b775bd6e4ea3-console-oauth-config\") pod \"console-5c6cb57997-z2szp\" (UID: \"01ee1aab-bb64-40eb-bc77-b775bd6e4ea3\") " pod="openshift-console/console-5c6cb57997-z2szp"
Oct 31 00:29:58.356080 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:29:58.356059    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/01ee1aab-bb64-40eb-bc77-b775bd6e4ea3-console-config\") pod \"console-5c6cb57997-z2szp\" (UID: \"01ee1aab-bb64-40eb-bc77-b775bd6e4ea3\") " pod="openshift-console/console-5c6cb57997-z2szp"
Oct 31 00:29:58.364579 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:29:58.364550    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5c6cb57997-z2szp"]
Oct 31 00:29:58.442133 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:29:58.442101    4284 generic.go:356] "Generic (PLEG): container finished" podID="56309073-dd28-4b64-a190-21a11522bc6a" containerID="792b6540036a21938f728d09bf0cd19df5f9aae41f8bff56845e708d78fdd344" exitCode=0
Oct 31 00:29:58.442325 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:29:58.442166    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-687df98b6b-7szfw"
Oct 31 00:29:58.442325 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:29:58.442175    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-687df98b6b-7szfw" event={"ID":"56309073-dd28-4b64-a190-21a11522bc6a","Type":"ContainerDied","Data":"792b6540036a21938f728d09bf0cd19df5f9aae41f8bff56845e708d78fdd344"}
Oct 31 00:29:58.442325 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:29:58.442239    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-687df98b6b-7szfw" event={"ID":"56309073-dd28-4b64-a190-21a11522bc6a","Type":"ContainerDied","Data":"01e19869887b31d7e0c7c851cc6db8374b8514c09a7ab66b31087e2ab048a2ee"}
Oct 31 00:29:58.442325 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:29:58.442263    4284 scope.go:122] "RemoveContainer" containerID="792b6540036a21938f728d09bf0cd19df5f9aae41f8bff56845e708d78fdd344"
Oct 31 00:29:58.449862 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:29:58.449836    4284 scope.go:122] "RemoveContainer" containerID="792b6540036a21938f728d09bf0cd19df5f9aae41f8bff56845e708d78fdd344"
Oct 31 00:29:58.450183 ip-10-0-132-173 kubenswrapper[4284]: E1031 00:29:58.450161    4284 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"792b6540036a21938f728d09bf0cd19df5f9aae41f8bff56845e708d78fdd344\": container with ID starting with 792b6540036a21938f728d09bf0cd19df5f9aae41f8bff56845e708d78fdd344 not found: ID does not exist" containerID="792b6540036a21938f728d09bf0cd19df5f9aae41f8bff56845e708d78fdd344"
Oct 31 00:29:58.450270 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:29:58.450194    4284 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"792b6540036a21938f728d09bf0cd19df5f9aae41f8bff56845e708d78fdd344"} err="failed to get container status \"792b6540036a21938f728d09bf0cd19df5f9aae41f8bff56845e708d78fdd344\": rpc error: code = NotFound desc = could not find container \"792b6540036a21938f728d09bf0cd19df5f9aae41f8bff56845e708d78fdd344\": container with ID starting with 792b6540036a21938f728d09bf0cd19df5f9aae41f8bff56845e708d78fdd344 not found: ID does not exist"
Oct 31 00:29:58.456471 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:29:58.456450    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/01ee1aab-bb64-40eb-bc77-b775bd6e4ea3-console-serving-cert\") pod \"console-5c6cb57997-z2szp\" (UID: \"01ee1aab-bb64-40eb-bc77-b775bd6e4ea3\") " pod="openshift-console/console-5c6cb57997-z2szp"
Oct 31 00:29:58.456548 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:29:58.456485    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/01ee1aab-bb64-40eb-bc77-b775bd6e4ea3-service-ca\") pod \"console-5c6cb57997-z2szp\" (UID: \"01ee1aab-bb64-40eb-bc77-b775bd6e4ea3\") " pod="openshift-console/console-5c6cb57997-z2szp"
Oct 31 00:29:58.456548 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:29:58.456502    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/01ee1aab-bb64-40eb-bc77-b775bd6e4ea3-tmp\") pod \"console-5c6cb57997-z2szp\" (UID: \"01ee1aab-bb64-40eb-bc77-b775bd6e4ea3\") " pod="openshift-console/console-5c6cb57997-z2szp"
Oct 31 00:29:58.456623 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:29:58.456549    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kp8zq\" (UniqueName: \"kubernetes.io/projected/01ee1aab-bb64-40eb-bc77-b775bd6e4ea3-kube-api-access-kp8zq\") pod \"console-5c6cb57997-z2szp\" (UID: \"01ee1aab-bb64-40eb-bc77-b775bd6e4ea3\") " pod="openshift-console/console-5c6cb57997-z2szp"
Oct 31 00:29:58.456623 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:29:58.456571    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/01ee1aab-bb64-40eb-bc77-b775bd6e4ea3-trusted-ca-bundle\") pod \"console-5c6cb57997-z2szp\" (UID: \"01ee1aab-bb64-40eb-bc77-b775bd6e4ea3\") " pod="openshift-console/console-5c6cb57997-z2szp"
Oct 31 00:29:58.456623 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:29:58.456592    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/01ee1aab-bb64-40eb-bc77-b775bd6e4ea3-oauth-serving-cert\") pod \"console-5c6cb57997-z2szp\" (UID: \"01ee1aab-bb64-40eb-bc77-b775bd6e4ea3\") " pod="openshift-console/console-5c6cb57997-z2szp"
Oct 31 00:29:58.456623 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:29:58.456619    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/01ee1aab-bb64-40eb-bc77-b775bd6e4ea3-console-oauth-config\") pod \"console-5c6cb57997-z2szp\" (UID: \"01ee1aab-bb64-40eb-bc77-b775bd6e4ea3\") " pod="openshift-console/console-5c6cb57997-z2szp"
Oct 31 00:29:58.456806 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:29:58.456648    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/01ee1aab-bb64-40eb-bc77-b775bd6e4ea3-console-config\") pod \"console-5c6cb57997-z2szp\" (UID: \"01ee1aab-bb64-40eb-bc77-b775bd6e4ea3\") " pod="openshift-console/console-5c6cb57997-z2szp"
Oct 31 00:29:58.456966 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:29:58.456947    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/01ee1aab-bb64-40eb-bc77-b775bd6e4ea3-tmp\") pod \"console-5c6cb57997-z2szp\" (UID: \"01ee1aab-bb64-40eb-bc77-b775bd6e4ea3\") " pod="openshift-console/console-5c6cb57997-z2szp"
Oct 31 00:29:58.457313 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:29:58.457251    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/01ee1aab-bb64-40eb-bc77-b775bd6e4ea3-service-ca\") pod \"console-5c6cb57997-z2szp\" (UID: \"01ee1aab-bb64-40eb-bc77-b775bd6e4ea3\") " pod="openshift-console/console-5c6cb57997-z2szp"
Oct 31 00:29:58.457621 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:29:58.457601    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/01ee1aab-bb64-40eb-bc77-b775bd6e4ea3-oauth-serving-cert\") pod \"console-5c6cb57997-z2szp\" (UID: \"01ee1aab-bb64-40eb-bc77-b775bd6e4ea3\") " pod="openshift-console/console-5c6cb57997-z2szp"
Oct 31 00:29:58.457707 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:29:58.457693    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/01ee1aab-bb64-40eb-bc77-b775bd6e4ea3-trusted-ca-bundle\") pod \"console-5c6cb57997-z2szp\" (UID: \"01ee1aab-bb64-40eb-bc77-b775bd6e4ea3\") " pod="openshift-console/console-5c6cb57997-z2szp"
Oct 31 00:29:58.457854 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:29:58.457830    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/01ee1aab-bb64-40eb-bc77-b775bd6e4ea3-console-config\") pod \"console-5c6cb57997-z2szp\" (UID: \"01ee1aab-bb64-40eb-bc77-b775bd6e4ea3\") " pod="openshift-console/console-5c6cb57997-z2szp"
Oct 31 00:29:58.458946 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:29:58.458924    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/01ee1aab-bb64-40eb-bc77-b775bd6e4ea3-console-serving-cert\") pod \"console-5c6cb57997-z2szp\" (UID: \"01ee1aab-bb64-40eb-bc77-b775bd6e4ea3\") " pod="openshift-console/console-5c6cb57997-z2szp"
Oct 31 00:29:58.459125 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:29:58.459106    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/01ee1aab-bb64-40eb-bc77-b775bd6e4ea3-console-oauth-config\") pod \"console-5c6cb57997-z2szp\" (UID: \"01ee1aab-bb64-40eb-bc77-b775bd6e4ea3\") " pod="openshift-console/console-5c6cb57997-z2szp"
Oct 31 00:29:58.517773 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:29:58.517689    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kp8zq\" (UniqueName: \"kubernetes.io/projected/01ee1aab-bb64-40eb-bc77-b775bd6e4ea3-kube-api-access-kp8zq\") pod \"console-5c6cb57997-z2szp\" (UID: \"01ee1aab-bb64-40eb-bc77-b775bd6e4ea3\") " pod="openshift-console/console-5c6cb57997-z2szp"
Oct 31 00:29:58.527928 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:29:58.527899    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:29:58.577272 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:29:58.577247    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5c6cb57997-z2szp"
Oct 31 00:29:58.598110 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:29:58.598084    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-console/console-687df98b6b-7szfw"]
Oct 31 00:29:58.604364 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:29:58.604339    4284 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-687df98b6b-7szfw"]
Oct 31 00:29:58.678761 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:29:58.678717    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:29:58.911175 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:29:58.911137    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:29:58.942097 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:29:58.941983    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5c6cb57997-z2szp"]
Oct 31 00:29:58.944615 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:29:58.944590    4284 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod01ee1aab_bb64_40eb_bc77_b775bd6e4ea3.slice/crio-ee6451108fbdc17332808ea8cac2303d343eb75ca8546d49b3c8e1465cce64ef WatchSource:0}: Error finding container ee6451108fbdc17332808ea8cac2303d343eb75ca8546d49b3c8e1465cce64ef: Status 404 returned error can't find the container with id ee6451108fbdc17332808ea8cac2303d343eb75ca8546d49b3c8e1465cce64ef
Oct 31 00:29:59.062773 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:29:59.062739    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:29:59.448331 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:29:59.448294    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5c6cb57997-z2szp" event={"ID":"01ee1aab-bb64-40eb-bc77-b775bd6e4ea3","Type":"ContainerStarted","Data":"f76280c00b2c5b497328e51f51a77a7ad3e580eb1ccf3374c144f6f8405c4947"}
Oct 31 00:29:59.448475 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:29:59.448343    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5c6cb57997-z2szp" event={"ID":"01ee1aab-bb64-40eb-bc77-b775bd6e4ea3","Type":"ContainerStarted","Data":"ee6451108fbdc17332808ea8cac2303d343eb75ca8546d49b3c8e1465cce64ef"}
Oct 31 00:29:59.563152 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:29:59.563102    4284 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-console/console-5c6cb57997-z2szp" podStartSLOduration=1.563088936 podStartE2EDuration="1.563088936s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 00:29:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:29:59.561988306 +0000 UTC m=+113.706995222" watchObservedRunningTime="2026-10-31 00:29:59.563088936 +0000 UTC m=+113.708095855"
Oct 31 00:29:59.960173 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:29:59.960140    4284 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="56309073-dd28-4b64-a190-21a11522bc6a" path="/var/lib/kubelet/pods/56309073-dd28-4b64-a190-21a11522bc6a/volumes"
Oct 31 00:30:01.024340 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:30:01.024310    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:30:01.168869 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:30:01.168832    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:30:05.534408 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:30:05.534363    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:30:05.726897 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:30:05.726859    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:30:08.578282 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:30:08.578240    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5c6cb57997-z2szp"
Oct 31 00:30:08.578698 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:30:08.578389    4284 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5c6cb57997-z2szp"
Oct 31 00:30:08.583097 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:30:08.583075    4284 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5c6cb57997-z2szp"
Oct 31 00:30:09.484615 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:30:09.484585    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5c6cb57997-z2szp"
Oct 31 00:30:09.647140 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:30:09.647100    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7cc6cf47b7-fqmmm"]
Oct 31 00:30:17.011329 ip-10-0-132-173 kubenswrapper[4284]: E1031 00:30:17.011250    4284 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fd8sp" containerName="extractor"
Oct 31 00:30:25.840157 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:30:25.840123    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:30:26.016770 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:30:26.016734    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:30:34.667081 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:30:34.667043    4284 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-console/console-7cc6cf47b7-fqmmm" podUID="a8ec2bd2-73fb-4d63-b6c3-0ab12370d2ee" containerName="console" containerID="cri-o://f8db77a37a8a97a4a603f9884252d9111f670e51f46194fbb40a7c7019f5ccd6" gracePeriod=15
Oct 31 00:30:34.909620 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:30:34.909596    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7cc6cf47b7-fqmmm"
Oct 31 00:30:35.029787 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:30:35.029712    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a8ec2bd2-73fb-4d63-b6c3-0ab12370d2ee-tmp\" (UniqueName: \"kubernetes.io/empty-dir/a8ec2bd2-73fb-4d63-b6c3-0ab12370d2ee-tmp\") pod \"a8ec2bd2-73fb-4d63-b6c3-0ab12370d2ee\" (UID: \"a8ec2bd2-73fb-4d63-b6c3-0ab12370d2ee\") "
Oct 31 00:30:35.029787 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:30:35.029758    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a8ec2bd2-73fb-4d63-b6c3-0ab12370d2ee-kube-api-access-z9cll\" (UniqueName: \"kubernetes.io/projected/a8ec2bd2-73fb-4d63-b6c3-0ab12370d2ee-kube-api-access-z9cll\") pod \"a8ec2bd2-73fb-4d63-b6c3-0ab12370d2ee\" (UID: \"a8ec2bd2-73fb-4d63-b6c3-0ab12370d2ee\") "
Oct 31 00:30:35.030076 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:30:35.029790    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a8ec2bd2-73fb-4d63-b6c3-0ab12370d2ee-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a8ec2bd2-73fb-4d63-b6c3-0ab12370d2ee-trusted-ca-bundle\") pod \"a8ec2bd2-73fb-4d63-b6c3-0ab12370d2ee\" (UID: \"a8ec2bd2-73fb-4d63-b6c3-0ab12370d2ee\") "
Oct 31 00:30:35.030076 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:30:35.029847    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a8ec2bd2-73fb-4d63-b6c3-0ab12370d2ee-console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a8ec2bd2-73fb-4d63-b6c3-0ab12370d2ee-console-serving-cert\") pod \"a8ec2bd2-73fb-4d63-b6c3-0ab12370d2ee\" (UID: \"a8ec2bd2-73fb-4d63-b6c3-0ab12370d2ee\") "
Oct 31 00:30:35.030076 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:30:35.030007    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a8ec2bd2-73fb-4d63-b6c3-0ab12370d2ee-tmp" pod "a8ec2bd2-73fb-4d63-b6c3-0ab12370d2ee" (UID: "a8ec2bd2-73fb-4d63-b6c3-0ab12370d2ee"). InnerVolumeSpecName "tmp". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:30:35.030264 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:30:35.030236    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a8ec2bd2-73fb-4d63-b6c3-0ab12370d2ee-trusted-ca-bundle" pod "a8ec2bd2-73fb-4d63-b6c3-0ab12370d2ee" (UID: "a8ec2bd2-73fb-4d63-b6c3-0ab12370d2ee"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 00:30:35.031280 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:30:35.031243    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a8ec2bd2-73fb-4d63-b6c3-0ab12370d2ee-oauth-serving-cert" pod "a8ec2bd2-73fb-4d63-b6c3-0ab12370d2ee" (UID: "a8ec2bd2-73fb-4d63-b6c3-0ab12370d2ee"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 00:30:35.031280 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:30:35.031259    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a8ec2bd2-73fb-4d63-b6c3-0ab12370d2ee-oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a8ec2bd2-73fb-4d63-b6c3-0ab12370d2ee-oauth-serving-cert\") pod \"a8ec2bd2-73fb-4d63-b6c3-0ab12370d2ee\" (UID: \"a8ec2bd2-73fb-4d63-b6c3-0ab12370d2ee\") "
Oct 31 00:30:35.031473 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:30:35.031339    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a8ec2bd2-73fb-4d63-b6c3-0ab12370d2ee-console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a8ec2bd2-73fb-4d63-b6c3-0ab12370d2ee-console-oauth-config\") pod \"a8ec2bd2-73fb-4d63-b6c3-0ab12370d2ee\" (UID: \"a8ec2bd2-73fb-4d63-b6c3-0ab12370d2ee\") "
Oct 31 00:30:35.031473 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:30:35.031387    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a8ec2bd2-73fb-4d63-b6c3-0ab12370d2ee-service-ca\" (UniqueName: \"kubernetes.io/configmap/a8ec2bd2-73fb-4d63-b6c3-0ab12370d2ee-service-ca\") pod \"a8ec2bd2-73fb-4d63-b6c3-0ab12370d2ee\" (UID: \"a8ec2bd2-73fb-4d63-b6c3-0ab12370d2ee\") "
Oct 31 00:30:35.031929 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:30:35.031894    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a8ec2bd2-73fb-4d63-b6c3-0ab12370d2ee-service-ca" pod "a8ec2bd2-73fb-4d63-b6c3-0ab12370d2ee" (UID: "a8ec2bd2-73fb-4d63-b6c3-0ab12370d2ee"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 00:30:35.036309 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:30:35.032141    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a8ec2bd2-73fb-4d63-b6c3-0ab12370d2ee-console-config\" (UniqueName: \"kubernetes.io/configmap/a8ec2bd2-73fb-4d63-b6c3-0ab12370d2ee-console-config\") pod \"a8ec2bd2-73fb-4d63-b6c3-0ab12370d2ee\" (UID: \"a8ec2bd2-73fb-4d63-b6c3-0ab12370d2ee\") "
Oct 31 00:30:35.036309 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:30:35.033983    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a8ec2bd2-73fb-4d63-b6c3-0ab12370d2ee-console-config" pod "a8ec2bd2-73fb-4d63-b6c3-0ab12370d2ee" (UID: "a8ec2bd2-73fb-4d63-b6c3-0ab12370d2ee"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 00:30:35.036309 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:30:35.034631    4284 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a8ec2bd2-73fb-4d63-b6c3-0ab12370d2ee-service-ca\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:30:35.036309 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:30:35.034662    4284 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a8ec2bd2-73fb-4d63-b6c3-0ab12370d2ee-console-config\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:30:35.036309 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:30:35.034677    4284 reconciler_common.go:299] "Volume detached for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a8ec2bd2-73fb-4d63-b6c3-0ab12370d2ee-tmp\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:30:35.036309 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:30:35.034692    4284 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a8ec2bd2-73fb-4d63-b6c3-0ab12370d2ee-trusted-ca-bundle\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:30:35.036309 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:30:35.034712    4284 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a8ec2bd2-73fb-4d63-b6c3-0ab12370d2ee-oauth-serving-cert\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:30:35.038054 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:30:35.038019    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a8ec2bd2-73fb-4d63-b6c3-0ab12370d2ee-kube-api-access-z9cll" pod "a8ec2bd2-73fb-4d63-b6c3-0ab12370d2ee" (UID: "a8ec2bd2-73fb-4d63-b6c3-0ab12370d2ee"). InnerVolumeSpecName "kube-api-access-z9cll". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 00:30:35.038790 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:30:35.038307    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a8ec2bd2-73fb-4d63-b6c3-0ab12370d2ee-console-oauth-config" pod "a8ec2bd2-73fb-4d63-b6c3-0ab12370d2ee" (UID: "a8ec2bd2-73fb-4d63-b6c3-0ab12370d2ee"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 00:30:35.040107 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:30:35.040074    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a8ec2bd2-73fb-4d63-b6c3-0ab12370d2ee-console-serving-cert" pod "a8ec2bd2-73fb-4d63-b6c3-0ab12370d2ee" (UID: "a8ec2bd2-73fb-4d63-b6c3-0ab12370d2ee"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 00:30:35.135738 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:30:35.135693    4284 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z9cll\" (UniqueName: \"kubernetes.io/projected/a8ec2bd2-73fb-4d63-b6c3-0ab12370d2ee-kube-api-access-z9cll\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:30:35.135738 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:30:35.135729    4284 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a8ec2bd2-73fb-4d63-b6c3-0ab12370d2ee-console-serving-cert\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:30:35.135738 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:30:35.135739    4284 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a8ec2bd2-73fb-4d63-b6c3-0ab12370d2ee-console-oauth-config\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:30:35.564418 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:30:35.564380    4284 generic.go:356] "Generic (PLEG): container finished" podID="a8ec2bd2-73fb-4d63-b6c3-0ab12370d2ee" containerID="f8db77a37a8a97a4a603f9884252d9111f670e51f46194fbb40a7c7019f5ccd6" exitCode=0
Oct 31 00:30:35.564618 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:30:35.564450    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7cc6cf47b7-fqmmm"
Oct 31 00:30:35.564618 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:30:35.564474    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7cc6cf47b7-fqmmm" event={"ID":"a8ec2bd2-73fb-4d63-b6c3-0ab12370d2ee","Type":"ContainerDied","Data":"f8db77a37a8a97a4a603f9884252d9111f670e51f46194fbb40a7c7019f5ccd6"}
Oct 31 00:30:35.564618 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:30:35.564563    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7cc6cf47b7-fqmmm" event={"ID":"a8ec2bd2-73fb-4d63-b6c3-0ab12370d2ee","Type":"ContainerDied","Data":"6a933e54f15ea6b00fbf032e87af29f0d02b20518096777fdc71756d1a13fffe"}
Oct 31 00:30:35.564618 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:30:35.564592    4284 scope.go:122] "RemoveContainer" containerID="f8db77a37a8a97a4a603f9884252d9111f670e51f46194fbb40a7c7019f5ccd6"
Oct 31 00:30:35.572416 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:30:35.572398    4284 scope.go:122] "RemoveContainer" containerID="f8db77a37a8a97a4a603f9884252d9111f670e51f46194fbb40a7c7019f5ccd6"
Oct 31 00:30:35.572660 ip-10-0-132-173 kubenswrapper[4284]: E1031 00:30:35.572641    4284 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f8db77a37a8a97a4a603f9884252d9111f670e51f46194fbb40a7c7019f5ccd6\": container with ID starting with f8db77a37a8a97a4a603f9884252d9111f670e51f46194fbb40a7c7019f5ccd6 not found: ID does not exist" containerID="f8db77a37a8a97a4a603f9884252d9111f670e51f46194fbb40a7c7019f5ccd6"
Oct 31 00:30:35.572705 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:30:35.572667    4284 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f8db77a37a8a97a4a603f9884252d9111f670e51f46194fbb40a7c7019f5ccd6"} err="failed to get container status \"f8db77a37a8a97a4a603f9884252d9111f670e51f46194fbb40a7c7019f5ccd6\": rpc error: code = NotFound desc = could not find container \"f8db77a37a8a97a4a603f9884252d9111f670e51f46194fbb40a7c7019f5ccd6\": container with ID starting with f8db77a37a8a97a4a603f9884252d9111f670e51f46194fbb40a7c7019f5ccd6 not found: ID does not exist"
Oct 31 00:30:35.721301 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:30:35.721271    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7cc6cf47b7-fqmmm"]
Oct 31 00:30:35.769244 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:30:35.769194    4284 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7cc6cf47b7-fqmmm"]
Oct 31 00:30:35.960301 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:30:35.960277    4284 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a8ec2bd2-73fb-4d63-b6c3-0ab12370d2ee" path="/var/lib/kubelet/pods/a8ec2bd2-73fb-4d63-b6c3-0ab12370d2ee/volumes"
Oct 31 00:30:47.012793 ip-10-0-132-173 kubenswrapper[4284]: E1031 00:30:47.012752    4284 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fd8sp" containerName="extractor"
Oct 31 00:30:49.644150 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:30:49.644112    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:31:09.802122 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:31:09.802085    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:31:17.013023 ip-10-0-132-173 kubenswrapper[4284]: E1031 00:31:17.012983    4284 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fd8sp" containerName="extractor"
Oct 31 00:31:47.013098 ip-10-0-132-173 kubenswrapper[4284]: E1031 00:31:47.013018    4284 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fd8sp" containerName="extractor"
Oct 31 00:31:49.941382 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:31:49.941339    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:32:17.011880 ip-10-0-132-173 kubenswrapper[4284]: E1031 00:32:17.011837    4284 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fd8sp" containerName="extractor"
Oct 31 00:32:22.146680 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:32:22.146643    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:32:22.937269 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:32:22.937230    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:32:23.300722 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:32:23.300643    4284 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be716xzb2"]
Oct 31 00:32:23.301090 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:32:23.300880    4284 cpu_manager.go:414] "containerMap: removing container" podUID="a8ec2bd2-73fb-4d63-b6c3-0ab12370d2ee" containerName="console"
Oct 31 00:32:23.301090 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:32:23.300889    4284 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a8ec2bd2-73fb-4d63-b6c3-0ab12370d2ee" containerName="console"
Oct 31 00:32:23.301090 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:32:23.300945    4284 memory_manager.go:365] "RemoveStaleState removing state" podUID="a8ec2bd2-73fb-4d63-b6c3-0ab12370d2ee" containerName="console"
Oct 31 00:32:23.303055 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:32:23.303039    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be716xzb2"
Oct 31 00:32:23.317165 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:32:23.317144    4284 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651czj5gg"]
Oct 31 00:32:23.319551 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:32:23.319533    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651czj5gg"
Oct 31 00:32:23.364174 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:32:23.364144    4284 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"e644421527c169d876fdb59235c927986d6bd2-9d2815fc-dockercfg-jx24b\""
Oct 31 00:32:23.364174 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:32:23.364166    4284 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Oct 31 00:32:23.364174 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:32:23.364147    4284 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Oct 31 00:32:23.380511 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:32:23.380478    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7llqf\" (UniqueName: \"kubernetes.io/projected/c45edd46-4015-497b-a442-355ff8ed71ce-kube-api-access-7llqf\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be716xzb2\" (UID: \"c45edd46-4015-497b-a442-355ff8ed71ce\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be716xzb2"
Oct 31 00:32:23.380620 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:32:23.380548    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c45edd46-4015-497b-a442-355ff8ed71ce-bundle\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be716xzb2\" (UID: \"c45edd46-4015-497b-a442-355ff8ed71ce\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be716xzb2"
Oct 31 00:32:23.380659 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:32:23.380640    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c45edd46-4015-497b-a442-355ff8ed71ce-util\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be716xzb2\" (UID: \"c45edd46-4015-497b-a442-355ff8ed71ce\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be716xzb2"
Oct 31 00:32:23.412862 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:32:23.412836    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651czj5gg"]
Oct 31 00:32:23.413628 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:32:23.413609    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be716xzb2"]
Oct 31 00:32:23.453827 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:32:23.453801    4284 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"c7537007944577f64df34960ace3c69136c416-139ec370-dockercfg-rs4k2\""
Oct 31 00:32:23.481098 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:32:23.481064    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pdbdw\" (UniqueName: \"kubernetes.io/projected/af650f19-e5f4-42a6-b8fc-6d0238515044-kube-api-access-pdbdw\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651czj5gg\" (UID: \"af650f19-e5f4-42a6-b8fc-6d0238515044\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651czj5gg"
Oct 31 00:32:23.481296 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:32:23.481107    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7llqf\" (UniqueName: \"kubernetes.io/projected/c45edd46-4015-497b-a442-355ff8ed71ce-kube-api-access-7llqf\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be716xzb2\" (UID: \"c45edd46-4015-497b-a442-355ff8ed71ce\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be716xzb2"
Oct 31 00:32:23.481296 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:32:23.481151    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/af650f19-e5f4-42a6-b8fc-6d0238515044-util\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651czj5gg\" (UID: \"af650f19-e5f4-42a6-b8fc-6d0238515044\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651czj5gg"
Oct 31 00:32:23.481296 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:32:23.481290    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/af650f19-e5f4-42a6-b8fc-6d0238515044-bundle\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651czj5gg\" (UID: \"af650f19-e5f4-42a6-b8fc-6d0238515044\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651czj5gg"
Oct 31 00:32:23.481496 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:32:23.481336    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c45edd46-4015-497b-a442-355ff8ed71ce-bundle\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be716xzb2\" (UID: \"c45edd46-4015-497b-a442-355ff8ed71ce\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be716xzb2"
Oct 31 00:32:23.481496 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:32:23.481364    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c45edd46-4015-497b-a442-355ff8ed71ce-util\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be716xzb2\" (UID: \"c45edd46-4015-497b-a442-355ff8ed71ce\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be716xzb2"
Oct 31 00:32:23.481711 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:32:23.481693    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c45edd46-4015-497b-a442-355ff8ed71ce-util\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be716xzb2\" (UID: \"c45edd46-4015-497b-a442-355ff8ed71ce\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be716xzb2"
Oct 31 00:32:23.481790 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:32:23.481743    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c45edd46-4015-497b-a442-355ff8ed71ce-bundle\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be716xzb2\" (UID: \"c45edd46-4015-497b-a442-355ff8ed71ce\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be716xzb2"
Oct 31 00:32:23.540148 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:32:23.540117    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7llqf\" (UniqueName: \"kubernetes.io/projected/c45edd46-4015-497b-a442-355ff8ed71ce-kube-api-access-7llqf\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be716xzb2\" (UID: \"c45edd46-4015-497b-a442-355ff8ed71ce\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be716xzb2"
Oct 31 00:32:23.582150 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:32:23.582093    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pdbdw\" (UniqueName: \"kubernetes.io/projected/af650f19-e5f4-42a6-b8fc-6d0238515044-kube-api-access-pdbdw\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651czj5gg\" (UID: \"af650f19-e5f4-42a6-b8fc-6d0238515044\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651czj5gg"
Oct 31 00:32:23.582150 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:32:23.582134    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/af650f19-e5f4-42a6-b8fc-6d0238515044-util\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651czj5gg\" (UID: \"af650f19-e5f4-42a6-b8fc-6d0238515044\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651czj5gg"
Oct 31 00:32:23.582312 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:32:23.582153    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/af650f19-e5f4-42a6-b8fc-6d0238515044-bundle\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651czj5gg\" (UID: \"af650f19-e5f4-42a6-b8fc-6d0238515044\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651czj5gg"
Oct 31 00:32:23.582490 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:32:23.582474    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/af650f19-e5f4-42a6-b8fc-6d0238515044-util\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651czj5gg\" (UID: \"af650f19-e5f4-42a6-b8fc-6d0238515044\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651czj5gg"
Oct 31 00:32:23.582527 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:32:23.582492    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/af650f19-e5f4-42a6-b8fc-6d0238515044-bundle\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651czj5gg\" (UID: \"af650f19-e5f4-42a6-b8fc-6d0238515044\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651czj5gg"
Oct 31 00:32:23.616256 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:32:23.616230    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be716xzb2"
Oct 31 00:32:23.640533 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:32:23.640508    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pdbdw\" (UniqueName: \"kubernetes.io/projected/af650f19-e5f4-42a6-b8fc-6d0238515044-kube-api-access-pdbdw\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651czj5gg\" (UID: \"af650f19-e5f4-42a6-b8fc-6d0238515044\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651czj5gg"
Oct 31 00:32:23.807920 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:32:23.807893    4284 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7876bfcb74-876pt"]
Oct 31 00:32:23.810200 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:32:23.810186    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-876pt"
Oct 31 00:32:23.831318 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:32:23.831284    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:32:23.864962 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:32:23.864916    4284 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Oct 31 00:32:23.864962 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:32:23.864920    4284 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Oct 31 00:32:23.874720 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:32:23.874699    4284 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Oct 31 00:32:23.876092 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:32:23.876077    4284 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Oct 31 00:32:23.876192 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:32:23.876178    4284 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Oct 31 00:32:23.883523 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:32:23.883506    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e8c333bb-6008-457c-b8a3-66bd654f012d-istio-token\") pod \"istiod-openshift-gateway-7876bfcb74-876pt\" (UID: \"e8c333bb-6008-457c-b8a3-66bd654f012d\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-876pt"
Oct 31 00:32:23.883603 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:32:23.883539    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e8c333bb-6008-457c-b8a3-66bd654f012d-istio-kubeconfig\") pod \"istiod-openshift-gateway-7876bfcb74-876pt\" (UID: \"e8c333bb-6008-457c-b8a3-66bd654f012d\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-876pt"
Oct 31 00:32:23.883603 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:32:23.883558    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/e8c333bb-6008-457c-b8a3-66bd654f012d-cacerts\") pod \"istiod-openshift-gateway-7876bfcb74-876pt\" (UID: \"e8c333bb-6008-457c-b8a3-66bd654f012d\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-876pt"
Oct 31 00:32:23.883666 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:32:23.883638    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gvvl6\" (UniqueName: \"kubernetes.io/projected/e8c333bb-6008-457c-b8a3-66bd654f012d-kube-api-access-gvvl6\") pod \"istiod-openshift-gateway-7876bfcb74-876pt\" (UID: \"e8c333bb-6008-457c-b8a3-66bd654f012d\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-876pt"
Oct 31 00:32:23.883728 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:32:23.883715    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/e8c333bb-6008-457c-b8a3-66bd654f012d-local-certs\") pod \"istiod-openshift-gateway-7876bfcb74-876pt\" (UID: \"e8c333bb-6008-457c-b8a3-66bd654f012d\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-876pt"
Oct 31 00:32:23.883762 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:32:23.883740    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/e8c333bb-6008-457c-b8a3-66bd654f012d-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7876bfcb74-876pt\" (UID: \"e8c333bb-6008-457c-b8a3-66bd654f012d\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-876pt"
Oct 31 00:32:23.883822 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:32:23.883808    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/e8c333bb-6008-457c-b8a3-66bd654f012d-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7876bfcb74-876pt\" (UID: \"e8c333bb-6008-457c-b8a3-66bd654f012d\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-876pt"
Oct 31 00:32:23.884843 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:32:23.884827    4284 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Oct 31 00:32:23.887862 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:32:23.887840    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7876bfcb74-876pt"]
Oct 31 00:32:23.932061 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:32:23.932034    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651czj5gg"
Oct 31 00:32:23.965279 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:32:23.965250    4284 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-6gfpq\""
Oct 31 00:32:23.979651 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:32:23.979616    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:32:23.985154 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:32:23.985127    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e8c333bb-6008-457c-b8a3-66bd654f012d-istio-token\") pod \"istiod-openshift-gateway-7876bfcb74-876pt\" (UID: \"e8c333bb-6008-457c-b8a3-66bd654f012d\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-876pt"
Oct 31 00:32:23.985727 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:32:23.985706    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e8c333bb-6008-457c-b8a3-66bd654f012d-istio-kubeconfig\") pod \"istiod-openshift-gateway-7876bfcb74-876pt\" (UID: \"e8c333bb-6008-457c-b8a3-66bd654f012d\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-876pt"
Oct 31 00:32:23.986277 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:32:23.986256    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/e8c333bb-6008-457c-b8a3-66bd654f012d-cacerts\") pod \"istiod-openshift-gateway-7876bfcb74-876pt\" (UID: \"e8c333bb-6008-457c-b8a3-66bd654f012d\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-876pt"
Oct 31 00:32:23.986377 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:32:23.986312    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gvvl6\" (UniqueName: \"kubernetes.io/projected/e8c333bb-6008-457c-b8a3-66bd654f012d-kube-api-access-gvvl6\") pod \"istiod-openshift-gateway-7876bfcb74-876pt\" (UID: \"e8c333bb-6008-457c-b8a3-66bd654f012d\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-876pt"
Oct 31 00:32:23.986377 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:32:23.986357    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/e8c333bb-6008-457c-b8a3-66bd654f012d-local-certs\") pod \"istiod-openshift-gateway-7876bfcb74-876pt\" (UID: \"e8c333bb-6008-457c-b8a3-66bd654f012d\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-876pt"
Oct 31 00:32:23.986487 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:32:23.986399    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/e8c333bb-6008-457c-b8a3-66bd654f012d-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7876bfcb74-876pt\" (UID: \"e8c333bb-6008-457c-b8a3-66bd654f012d\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-876pt"
Oct 31 00:32:23.986487 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:32:23.986446    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/e8c333bb-6008-457c-b8a3-66bd654f012d-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7876bfcb74-876pt\" (UID: \"e8c333bb-6008-457c-b8a3-66bd654f012d\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-876pt"
Oct 31 00:32:23.987098 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:32:23.987072    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/e8c333bb-6008-457c-b8a3-66bd654f012d-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7876bfcb74-876pt\" (UID: \"e8c333bb-6008-457c-b8a3-66bd654f012d\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-876pt"
Oct 31 00:32:23.989759 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:32:23.989730    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e8c333bb-6008-457c-b8a3-66bd654f012d-istio-kubeconfig\") pod \"istiod-openshift-gateway-7876bfcb74-876pt\" (UID: \"e8c333bb-6008-457c-b8a3-66bd654f012d\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-876pt"
Oct 31 00:32:23.991360 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:32:23.991335    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/e8c333bb-6008-457c-b8a3-66bd654f012d-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7876bfcb74-876pt\" (UID: \"e8c333bb-6008-457c-b8a3-66bd654f012d\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-876pt"
Oct 31 00:32:23.991624 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:32:23.991606    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/e8c333bb-6008-457c-b8a3-66bd654f012d-cacerts\") pod \"istiod-openshift-gateway-7876bfcb74-876pt\" (UID: \"e8c333bb-6008-457c-b8a3-66bd654f012d\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-876pt"
Oct 31 00:32:23.992387 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:32:23.992370    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/e8c333bb-6008-457c-b8a3-66bd654f012d-local-certs\") pod \"istiod-openshift-gateway-7876bfcb74-876pt\" (UID: \"e8c333bb-6008-457c-b8a3-66bd654f012d\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-876pt"
Oct 31 00:32:23.992817 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:32:23.992781    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be716xzb2"]
Oct 31 00:32:23.996319 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:32:23.996294    4284 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc45edd46_4015_497b_a442_355ff8ed71ce.slice/crio-a3f95acfb1aebdb5f87b2eb82faee0ad168ed16af3d87429a09d23c83b35bab1 WatchSource:0}: Error finding container a3f95acfb1aebdb5f87b2eb82faee0ad168ed16af3d87429a09d23c83b35bab1: Status 404 returned error can't find the container with id a3f95acfb1aebdb5f87b2eb82faee0ad168ed16af3d87429a09d23c83b35bab1
Oct 31 00:32:24.044491 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:32:24.044461    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e8c333bb-6008-457c-b8a3-66bd654f012d-istio-token\") pod \"istiod-openshift-gateway-7876bfcb74-876pt\" (UID: \"e8c333bb-6008-457c-b8a3-66bd654f012d\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-876pt"
Oct 31 00:32:24.045020 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:32:24.045005    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gvvl6\" (UniqueName: \"kubernetes.io/projected/e8c333bb-6008-457c-b8a3-66bd654f012d-kube-api-access-gvvl6\") pod \"istiod-openshift-gateway-7876bfcb74-876pt\" (UID: \"e8c333bb-6008-457c-b8a3-66bd654f012d\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-876pt"
Oct 31 00:32:24.121448 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:32:24.121360    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-876pt"
Oct 31 00:32:24.293343 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:32:24.293322    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651czj5gg"]
Oct 31 00:32:24.295523 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:32:24.295499    4284 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaf650f19_e5f4_42a6_b8fc_6d0238515044.slice/crio-c4f44528da6a32d79c017d2b98e838e22eb8750558884c5e937f98e670635029 WatchSource:0}: Error finding container c4f44528da6a32d79c017d2b98e838e22eb8750558884c5e937f98e670635029: Status 404 returned error can't find the container with id c4f44528da6a32d79c017d2b98e838e22eb8750558884c5e937f98e670635029
Oct 31 00:32:24.487166 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:32:24.487143    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7876bfcb74-876pt"]
Oct 31 00:32:24.489301 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:32:24.489263    4284 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode8c333bb_6008_457c_b8a3_66bd654f012d.slice/crio-deb203a2d3130c6a88d0e9ac6a61ae835a479aa1636792a97150a64cfe3e4f94 WatchSource:0}: Error finding container deb203a2d3130c6a88d0e9ac6a61ae835a479aa1636792a97150a64cfe3e4f94: Status 404 returned error can't find the container with id deb203a2d3130c6a88d0e9ac6a61ae835a479aa1636792a97150a64cfe3e4f94
Oct 31 00:32:24.908066 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:32:24.908028    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651czj5gg" event={"ID":"af650f19-e5f4-42a6-b8fc-6d0238515044","Type":"ContainerStarted","Data":"c4f44528da6a32d79c017d2b98e838e22eb8750558884c5e937f98e670635029"}
Oct 31 00:32:24.909285 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:32:24.909257    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-876pt" event={"ID":"e8c333bb-6008-457c-b8a3-66bd654f012d","Type":"ContainerStarted","Data":"deb203a2d3130c6a88d0e9ac6a61ae835a479aa1636792a97150a64cfe3e4f94"}
Oct 31 00:32:24.910378 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:32:24.910355    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be716xzb2" event={"ID":"c45edd46-4015-497b-a442-355ff8ed71ce","Type":"ContainerStarted","Data":"a3f95acfb1aebdb5f87b2eb82faee0ad168ed16af3d87429a09d23c83b35bab1"}
Oct 31 00:32:25.210901 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:32:25.210816    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:32:29.076678 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:32:29.076649    4284 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 00:32:29.076963 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:32:29.076714    4284 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 00:32:29.934420 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:32:29.934382    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-876pt" event={"ID":"e8c333bb-6008-457c-b8a3-66bd654f012d","Type":"ContainerStarted","Data":"e8e6c2cd8b38f39329b13b03cbb7b4bf9eb1effffd2bcb709b6d426a8b75c55b"}
Oct 31 00:32:29.934603 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:32:29.934491    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-876pt"
Oct 31 00:32:30.098451 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:32:30.098397    4284 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-876pt" podStartSLOduration=2.513740088 podStartE2EDuration="7.098381669s" totalImagesPullingTime="4.584641581s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 00:32:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:32:30.050572913 +0000 UTC m=+264.195579833" watchObservedRunningTime="2026-10-31 00:32:30.098381669 +0000 UTC m=+264.243388584"
Oct 31 00:32:30.348582 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:32:30.348497    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:32:30.569939 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:32:30.569904    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:32:30.941350 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:32:30.941319    4284 patch_prober.go:29] interesting pod/istiod-openshift-gateway-7876bfcb74-876pt container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 31 00:32:30.941529 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:32:30.941373    4284 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-876pt" podUID="e8c333bb-6008-457c-b8a3-66bd654f012d" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 31 00:32:31.758978 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:32:31.758941    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:32:31.970606 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:32:31.970577    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:32:32.116848 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:32:32.116814    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:32:33.940475 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:32:33.940445    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-876pt"
Oct 31 00:32:35.731857 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:32:35.731819    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:32:41.976686 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:32:41.976650    4284 generic.go:356] "Generic (PLEG): container finished" podID="af650f19-e5f4-42a6-b8fc-6d0238515044" containerID="5237c56df0160380788e8d854aa73cc274e178b62b03d7057efde7d7e287ffa8" exitCode=0
Oct 31 00:32:41.977155 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:32:41.976740    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651czj5gg" event={"ID":"af650f19-e5f4-42a6-b8fc-6d0238515044","Type":"ContainerDied","Data":"5237c56df0160380788e8d854aa73cc274e178b62b03d7057efde7d7e287ffa8"}
Oct 31 00:32:41.978234 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:32:41.978188    4284 generic.go:356] "Generic (PLEG): container finished" podID="c45edd46-4015-497b-a442-355ff8ed71ce" containerID="9aca96d6c99b6c531280cf6ce5be2493124e8158773aabf1747342b327c006a5" exitCode=0
Oct 31 00:32:41.978365 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:32:41.978241    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be716xzb2" event={"ID":"c45edd46-4015-497b-a442-355ff8ed71ce","Type":"ContainerDied","Data":"9aca96d6c99b6c531280cf6ce5be2493124e8158773aabf1747342b327c006a5"}
Oct 31 00:32:42.983562 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:32:42.983529    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be716xzb2" event={"ID":"c45edd46-4015-497b-a442-355ff8ed71ce","Type":"ContainerStarted","Data":"403b686ddf12392dd2e9f794a81e692b8b6f007f47c816265d52189558d1bade"}
Oct 31 00:32:43.987981 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:32:43.987947    4284 generic.go:356] "Generic (PLEG): container finished" podID="c45edd46-4015-497b-a442-355ff8ed71ce" containerID="403b686ddf12392dd2e9f794a81e692b8b6f007f47c816265d52189558d1bade" exitCode=0
Oct 31 00:32:43.987981 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:32:43.987985    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be716xzb2" event={"ID":"c45edd46-4015-497b-a442-355ff8ed71ce","Type":"ContainerDied","Data":"403b686ddf12392dd2e9f794a81e692b8b6f007f47c816265d52189558d1bade"}
Oct 31 00:32:47.002591 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:32:47.002520    4284 generic.go:356] "Generic (PLEG): container finished" podID="af650f19-e5f4-42a6-b8fc-6d0238515044" containerID="42a0ab144c0afcf7a51e9fa06b415dea2d9c42d796eb6331b1e425148a29e8c8" exitCode=0
Oct 31 00:32:47.002591 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:32:47.002575    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651czj5gg" event={"ID":"af650f19-e5f4-42a6-b8fc-6d0238515044","Type":"ContainerDied","Data":"42a0ab144c0afcf7a51e9fa06b415dea2d9c42d796eb6331b1e425148a29e8c8"}
Oct 31 00:32:47.016691 ip-10-0-132-173 kubenswrapper[4284]: E1031 00:32:47.016655    4284 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fd8sp" containerName="extractor"
Oct 31 00:32:50.277998 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:32:50.277943    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:32:52.023075 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:32:52.023028    4284 generic.go:356] "Generic (PLEG): container finished" podID="c45edd46-4015-497b-a442-355ff8ed71ce" containerID="2a3f1e583340afb4f1a971947882c5b691c5b9148c15c736d754faf49e257d9c" exitCode=0
Oct 31 00:32:52.023635 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:32:52.023103    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be716xzb2" event={"ID":"c45edd46-4015-497b-a442-355ff8ed71ce","Type":"ContainerDied","Data":"2a3f1e583340afb4f1a971947882c5b691c5b9148c15c736d754faf49e257d9c"}
Oct 31 00:32:53.162292 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:32:53.162266    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be716xzb2"
Oct 31 00:32:53.214797 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:32:53.214760    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c45edd46-4015-497b-a442-355ff8ed71ce-kube-api-access-7llqf\" (UniqueName: \"kubernetes.io/projected/c45edd46-4015-497b-a442-355ff8ed71ce-kube-api-access-7llqf\") pod \"c45edd46-4015-497b-a442-355ff8ed71ce\" (UID: \"c45edd46-4015-497b-a442-355ff8ed71ce\") "
Oct 31 00:32:53.214990 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:32:53.214871    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c45edd46-4015-497b-a442-355ff8ed71ce-bundle\" (UniqueName: \"kubernetes.io/empty-dir/c45edd46-4015-497b-a442-355ff8ed71ce-bundle\") pod \"c45edd46-4015-497b-a442-355ff8ed71ce\" (UID: \"c45edd46-4015-497b-a442-355ff8ed71ce\") "
Oct 31 00:32:53.214990 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:32:53.214904    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c45edd46-4015-497b-a442-355ff8ed71ce-util\" (UniqueName: \"kubernetes.io/empty-dir/c45edd46-4015-497b-a442-355ff8ed71ce-util\") pod \"c45edd46-4015-497b-a442-355ff8ed71ce\" (UID: \"c45edd46-4015-497b-a442-355ff8ed71ce\") "
Oct 31 00:32:53.215630 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:32:53.215600    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c45edd46-4015-497b-a442-355ff8ed71ce-bundle" pod "c45edd46-4015-497b-a442-355ff8ed71ce" (UID: "c45edd46-4015-497b-a442-355ff8ed71ce"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:32:53.217823 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:32:53.217795    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c45edd46-4015-497b-a442-355ff8ed71ce-kube-api-access-7llqf" pod "c45edd46-4015-497b-a442-355ff8ed71ce" (UID: "c45edd46-4015-497b-a442-355ff8ed71ce"). InnerVolumeSpecName "kube-api-access-7llqf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 00:32:53.221553 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:32:53.221525    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c45edd46-4015-497b-a442-355ff8ed71ce-util" pod "c45edd46-4015-497b-a442-355ff8ed71ce" (UID: "c45edd46-4015-497b-a442-355ff8ed71ce"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:32:53.316040 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:32:53.315956    4284 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7llqf\" (UniqueName: \"kubernetes.io/projected/c45edd46-4015-497b-a442-355ff8ed71ce-kube-api-access-7llqf\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:32:53.316040 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:32:53.315985    4284 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c45edd46-4015-497b-a442-355ff8ed71ce-bundle\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:32:53.316040 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:32:53.316000    4284 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c45edd46-4015-497b-a442-355ff8ed71ce-util\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:32:54.033028 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:32:54.032945    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651czj5gg" event={"ID":"af650f19-e5f4-42a6-b8fc-6d0238515044","Type":"ContainerStarted","Data":"c391e30c624b36a59ad44ec82fdd924a792345794850fa6a7a38af6075990ecc"}
Oct 31 00:32:54.034882 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:32:54.034858    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be716xzb2" event={"ID":"c45edd46-4015-497b-a442-355ff8ed71ce","Type":"ContainerDied","Data":"a3f95acfb1aebdb5f87b2eb82faee0ad168ed16af3d87429a09d23c83b35bab1"}
Oct 31 00:32:54.034882 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:32:54.034875    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be716xzb2"
Oct 31 00:32:54.035035 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:32:54.034893    4284 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a3f95acfb1aebdb5f87b2eb82faee0ad168ed16af3d87429a09d23c83b35bab1"
Oct 31 00:32:54.147167 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:32:54.147107    4284 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651czj5gg" podStartSLOduration=2.514458577 podStartE2EDuration="31.147094377s" totalImagesPullingTime="28.6326358s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 00:32:23 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:32:54.14690156 +0000 UTC m=+288.291908486" watchObservedRunningTime="2026-10-31 00:32:54.147094377 +0000 UTC m=+288.292101292"
Oct 31 00:32:55.039684 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:32:55.039653    4284 generic.go:356] "Generic (PLEG): container finished" podID="af650f19-e5f4-42a6-b8fc-6d0238515044" containerID="c391e30c624b36a59ad44ec82fdd924a792345794850fa6a7a38af6075990ecc" exitCode=0
Oct 31 00:32:55.040141 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:32:55.039722    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651czj5gg" event={"ID":"af650f19-e5f4-42a6-b8fc-6d0238515044","Type":"ContainerDied","Data":"c391e30c624b36a59ad44ec82fdd924a792345794850fa6a7a38af6075990ecc"}
Oct 31 00:32:56.215935 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:32:56.215901    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651czj5gg"
Oct 31 00:32:56.237723 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:32:56.237693    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/af650f19-e5f4-42a6-b8fc-6d0238515044-kube-api-access-pdbdw\" (UniqueName: \"kubernetes.io/projected/af650f19-e5f4-42a6-b8fc-6d0238515044-kube-api-access-pdbdw\") pod \"af650f19-e5f4-42a6-b8fc-6d0238515044\" (UID: \"af650f19-e5f4-42a6-b8fc-6d0238515044\") "
Oct 31 00:32:56.237867 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:32:56.237743    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/af650f19-e5f4-42a6-b8fc-6d0238515044-bundle\" (UniqueName: \"kubernetes.io/empty-dir/af650f19-e5f4-42a6-b8fc-6d0238515044-bundle\") pod \"af650f19-e5f4-42a6-b8fc-6d0238515044\" (UID: \"af650f19-e5f4-42a6-b8fc-6d0238515044\") "
Oct 31 00:32:56.237867 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:32:56.237770    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/af650f19-e5f4-42a6-b8fc-6d0238515044-util\" (UniqueName: \"kubernetes.io/empty-dir/af650f19-e5f4-42a6-b8fc-6d0238515044-util\") pod \"af650f19-e5f4-42a6-b8fc-6d0238515044\" (UID: \"af650f19-e5f4-42a6-b8fc-6d0238515044\") "
Oct 31 00:32:56.238445 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:32:56.238411    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/af650f19-e5f4-42a6-b8fc-6d0238515044-bundle" pod "af650f19-e5f4-42a6-b8fc-6d0238515044" (UID: "af650f19-e5f4-42a6-b8fc-6d0238515044"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:32:56.240096 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:32:56.240072    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/af650f19-e5f4-42a6-b8fc-6d0238515044-kube-api-access-pdbdw" pod "af650f19-e5f4-42a6-b8fc-6d0238515044" (UID: "af650f19-e5f4-42a6-b8fc-6d0238515044"). InnerVolumeSpecName "kube-api-access-pdbdw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 00:32:56.242130 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:32:56.242106    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/af650f19-e5f4-42a6-b8fc-6d0238515044-util" pod "af650f19-e5f4-42a6-b8fc-6d0238515044" (UID: "af650f19-e5f4-42a6-b8fc-6d0238515044"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:32:56.338468 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:32:56.338392    4284 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pdbdw\" (UniqueName: \"kubernetes.io/projected/af650f19-e5f4-42a6-b8fc-6d0238515044-kube-api-access-pdbdw\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:32:56.338468 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:32:56.338421    4284 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/af650f19-e5f4-42a6-b8fc-6d0238515044-bundle\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:32:56.338468 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:32:56.338432    4284 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/af650f19-e5f4-42a6-b8fc-6d0238515044-util\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:32:57.048133 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:32:57.048099    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651czj5gg" event={"ID":"af650f19-e5f4-42a6-b8fc-6d0238515044","Type":"ContainerDied","Data":"c4f44528da6a32d79c017d2b98e838e22eb8750558884c5e937f98e670635029"}
Oct 31 00:32:57.048133 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:32:57.048117    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651czj5gg"
Oct 31 00:32:57.048365 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:32:57.048144    4284 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c4f44528da6a32d79c017d2b98e838e22eb8750558884c5e937f98e670635029"
Oct 31 00:33:02.956016 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:33:02.955979    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:33:03.173610 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:33:03.173562    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:33:03.846036 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:33:03.845995    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:33:04.069697 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:33:04.069665    4284 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub-operator-system/opendatahub-operator-controller-manager-c54754479-2t99s"]
Oct 31 00:33:04.070084 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:33:04.069911    4284 cpu_manager.go:414] "containerMap: removing container" podUID="af650f19-e5f4-42a6-b8fc-6d0238515044" containerName="util"
Oct 31 00:33:04.070084 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:33:04.069921    4284 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="af650f19-e5f4-42a6-b8fc-6d0238515044" containerName="util"
Oct 31 00:33:04.070084 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:33:04.069929    4284 cpu_manager.go:414] "containerMap: removing container" podUID="c45edd46-4015-497b-a442-355ff8ed71ce" containerName="pull"
Oct 31 00:33:04.070084 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:33:04.069934    4284 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c45edd46-4015-497b-a442-355ff8ed71ce" containerName="pull"
Oct 31 00:33:04.070084 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:33:04.069948    4284 cpu_manager.go:414] "containerMap: removing container" podUID="c45edd46-4015-497b-a442-355ff8ed71ce" containerName="util"
Oct 31 00:33:04.070084 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:33:04.069952    4284 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c45edd46-4015-497b-a442-355ff8ed71ce" containerName="util"
Oct 31 00:33:04.070084 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:33:04.069960    4284 cpu_manager.go:414] "containerMap: removing container" podUID="af650f19-e5f4-42a6-b8fc-6d0238515044" containerName="pull"
Oct 31 00:33:04.070084 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:33:04.069964    4284 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="af650f19-e5f4-42a6-b8fc-6d0238515044" containerName="pull"
Oct 31 00:33:04.070084 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:33:04.069970    4284 cpu_manager.go:414] "containerMap: removing container" podUID="af650f19-e5f4-42a6-b8fc-6d0238515044" containerName="extract"
Oct 31 00:33:04.070084 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:33:04.069974    4284 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="af650f19-e5f4-42a6-b8fc-6d0238515044" containerName="extract"
Oct 31 00:33:04.070084 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:33:04.069986    4284 cpu_manager.go:414] "containerMap: removing container" podUID="c45edd46-4015-497b-a442-355ff8ed71ce" containerName="extract"
Oct 31 00:33:04.070084 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:33:04.069992    4284 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c45edd46-4015-497b-a442-355ff8ed71ce" containerName="extract"
Oct 31 00:33:04.070084 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:33:04.070030    4284 memory_manager.go:365] "RemoveStaleState removing state" podUID="af650f19-e5f4-42a6-b8fc-6d0238515044" containerName="extract"
Oct 31 00:33:04.070084 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:33:04.070039    4284 memory_manager.go:365] "RemoveStaleState removing state" podUID="c45edd46-4015-497b-a442-355ff8ed71ce" containerName="extract"
Oct 31 00:33:04.072739 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:33:04.072720    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-c54754479-2t99s"
Oct 31 00:33:04.092939 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:33:04.092914    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5845017d-8aab-4e95-96fa-d5097d7b9332-tmp\") pod \"opendatahub-operator-controller-manager-c54754479-2t99s\" (UID: \"5845017d-8aab-4e95-96fa-d5097d7b9332\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-c54754479-2t99s"
Oct 31 00:33:04.093092 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:33:04.092949    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-99p6w\" (UniqueName: \"kubernetes.io/projected/5845017d-8aab-4e95-96fa-d5097d7b9332-kube-api-access-99p6w\") pod \"opendatahub-operator-controller-manager-c54754479-2t99s\" (UID: \"5845017d-8aab-4e95-96fa-d5097d7b9332\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-c54754479-2t99s"
Oct 31 00:33:04.093092 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:33:04.092971    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/5845017d-8aab-4e95-96fa-d5097d7b9332-apiservice-cert\") pod \"opendatahub-operator-controller-manager-c54754479-2t99s\" (UID: \"5845017d-8aab-4e95-96fa-d5097d7b9332\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-c54754479-2t99s"
Oct 31 00:33:04.093193 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:33:04.093086    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/5845017d-8aab-4e95-96fa-d5097d7b9332-manifests\") pod \"opendatahub-operator-controller-manager-c54754479-2t99s\" (UID: \"5845017d-8aab-4e95-96fa-d5097d7b9332\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-c54754479-2t99s"
Oct 31 00:33:04.093193 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:33:04.093122    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/5845017d-8aab-4e95-96fa-d5097d7b9332-metrics-cert\") pod \"opendatahub-operator-controller-manager-c54754479-2t99s\" (UID: \"5845017d-8aab-4e95-96fa-d5097d7b9332\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-c54754479-2t99s"
Oct 31 00:33:04.093193 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:33:04.093152    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5845017d-8aab-4e95-96fa-d5097d7b9332-webhook-cert\") pod \"opendatahub-operator-controller-manager-c54754479-2t99s\" (UID: \"5845017d-8aab-4e95-96fa-d5097d7b9332\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-c54754479-2t99s"
Oct 31 00:33:04.176259 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:33:04.176207    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:33:04.181538 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:33:04.181516    4284 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub-operator-system\"/\"kube-root-ca.crt\""
Oct 31 00:33:04.181613 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:33:04.181554    4284 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-controller-webhook-cert\""
Oct 31 00:33:04.181874 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:33:04.181854    4284 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-controller-manager-service-cert\""
Oct 31 00:33:04.181874 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:33:04.181860    4284 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-controller-manager-dockercfg-km8d9\""
Oct 31 00:33:04.182029 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:33:04.181857    4284 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub-operator-system\"/\"openshift-service-ca.crt\""
Oct 31 00:33:04.182029 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:33:04.181904    4284 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-metrics-tls\""
Oct 31 00:33:04.194354 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:33:04.194324    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/5845017d-8aab-4e95-96fa-d5097d7b9332-manifests\") pod \"opendatahub-operator-controller-manager-c54754479-2t99s\" (UID: \"5845017d-8aab-4e95-96fa-d5097d7b9332\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-c54754479-2t99s"
Oct 31 00:33:04.194407 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:33:04.194367    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/5845017d-8aab-4e95-96fa-d5097d7b9332-metrics-cert\") pod \"opendatahub-operator-controller-manager-c54754479-2t99s\" (UID: \"5845017d-8aab-4e95-96fa-d5097d7b9332\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-c54754479-2t99s"
Oct 31 00:33:04.194451 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:33:04.194412    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5845017d-8aab-4e95-96fa-d5097d7b9332-webhook-cert\") pod \"opendatahub-operator-controller-manager-c54754479-2t99s\" (UID: \"5845017d-8aab-4e95-96fa-d5097d7b9332\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-c54754479-2t99s"
Oct 31 00:33:04.194498 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:33:04.194453    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5845017d-8aab-4e95-96fa-d5097d7b9332-tmp\") pod \"opendatahub-operator-controller-manager-c54754479-2t99s\" (UID: \"5845017d-8aab-4e95-96fa-d5097d7b9332\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-c54754479-2t99s"
Oct 31 00:33:04.194498 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:33:04.194490    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-99p6w\" (UniqueName: \"kubernetes.io/projected/5845017d-8aab-4e95-96fa-d5097d7b9332-kube-api-access-99p6w\") pod \"opendatahub-operator-controller-manager-c54754479-2t99s\" (UID: \"5845017d-8aab-4e95-96fa-d5097d7b9332\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-c54754479-2t99s"
Oct 31 00:33:04.194601 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:33:04.194522    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/5845017d-8aab-4e95-96fa-d5097d7b9332-apiservice-cert\") pod \"opendatahub-operator-controller-manager-c54754479-2t99s\" (UID: \"5845017d-8aab-4e95-96fa-d5097d7b9332\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-c54754479-2t99s"
Oct 31 00:33:04.194827 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:33:04.194795    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/5845017d-8aab-4e95-96fa-d5097d7b9332-manifests\") pod \"opendatahub-operator-controller-manager-c54754479-2t99s\" (UID: \"5845017d-8aab-4e95-96fa-d5097d7b9332\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-c54754479-2t99s"
Oct 31 00:33:04.196897 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:33:04.196867    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5845017d-8aab-4e95-96fa-d5097d7b9332-tmp\") pod \"opendatahub-operator-controller-manager-c54754479-2t99s\" (UID: \"5845017d-8aab-4e95-96fa-d5097d7b9332\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-c54754479-2t99s"
Oct 31 00:33:04.197164 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:33:04.197147    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/5845017d-8aab-4e95-96fa-d5097d7b9332-apiservice-cert\") pod \"opendatahub-operator-controller-manager-c54754479-2t99s\" (UID: \"5845017d-8aab-4e95-96fa-d5097d7b9332\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-c54754479-2t99s"
Oct 31 00:33:04.197235 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:33:04.197202    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/5845017d-8aab-4e95-96fa-d5097d7b9332-metrics-cert\") pod \"opendatahub-operator-controller-manager-c54754479-2t99s\" (UID: \"5845017d-8aab-4e95-96fa-d5097d7b9332\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-c54754479-2t99s"
Oct 31 00:33:04.197311 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:33:04.197290    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5845017d-8aab-4e95-96fa-d5097d7b9332-webhook-cert\") pod \"opendatahub-operator-controller-manager-c54754479-2t99s\" (UID: \"5845017d-8aab-4e95-96fa-d5097d7b9332\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-c54754479-2t99s"
Oct 31 00:33:04.203844 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:33:04.203816    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub-operator-system/opendatahub-operator-controller-manager-c54754479-2t99s"]
Oct 31 00:33:04.254385 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:33:04.254358    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-99p6w\" (UniqueName: \"kubernetes.io/projected/5845017d-8aab-4e95-96fa-d5097d7b9332-kube-api-access-99p6w\") pod \"opendatahub-operator-controller-manager-c54754479-2t99s\" (UID: \"5845017d-8aab-4e95-96fa-d5097d7b9332\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-c54754479-2t99s"
Oct 31 00:33:04.329708 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:33:04.329670    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:33:04.385845 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:33:04.385805    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-c54754479-2t99s"
Oct 31 00:33:04.480543 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:33:04.480502    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:33:04.859711 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:33:04.859689    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub-operator-system/opendatahub-operator-controller-manager-c54754479-2t99s"]
Oct 31 00:33:04.861713 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:33:04.861692    4284 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5845017d_8aab_4e95_96fa_d5097d7b9332.slice/crio-28c91dc2e917a6712e63d9b234f16681158c0e44dc75dc0d6a390ab2a6d2f744 WatchSource:0}: Error finding container 28c91dc2e917a6712e63d9b234f16681158c0e44dc75dc0d6a390ab2a6d2f744: Status 404 returned error can't find the container with id 28c91dc2e917a6712e63d9b234f16681158c0e44dc75dc0d6a390ab2a6d2f744
Oct 31 00:33:05.078024 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:33:05.077983    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-c54754479-2t99s" event={"ID":"5845017d-8aab-4e95-96fa-d5097d7b9332","Type":"ContainerStarted","Data":"28c91dc2e917a6712e63d9b234f16681158c0e44dc75dc0d6a390ab2a6d2f744"}
Oct 31 00:33:06.572472 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:33:06.572429    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:33:07.561206 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:33:07.561164    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:33:07.740090 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:33:07.740055    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:33:09.094695 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:33:09.094609    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-c54754479-2t99s" event={"ID":"5845017d-8aab-4e95-96fa-d5097d7b9332","Type":"ContainerStarted","Data":"953639cb5ec77d93f2c4745d5907edfa5674466505e4ed96dd6822eb7337b02e"}
Oct 31 00:33:09.560719 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:33:09.560671    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:33:09.716597 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:33:09.716552    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:33:10.099892 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:33:10.099858    4284 generic.go:356] "Generic (PLEG): container finished" podID="5845017d-8aab-4e95-96fa-d5097d7b9332" containerID="953639cb5ec77d93f2c4745d5907edfa5674466505e4ed96dd6822eb7337b02e" exitCode=0
Oct 31 00:33:10.100313 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:33:10.099923    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-c54754479-2t99s" event={"ID":"5845017d-8aab-4e95-96fa-d5097d7b9332","Type":"ContainerDied","Data":"953639cb5ec77d93f2c4745d5907edfa5674466505e4ed96dd6822eb7337b02e"}
Oct 31 00:33:10.100516 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:33:10.100499    4284 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 31 00:33:11.106117 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:33:11.106081    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-c54754479-2t99s" event={"ID":"5845017d-8aab-4e95-96fa-d5097d7b9332","Type":"ContainerStarted","Data":"604d86127c2daf91656e4602cfe4addf4dafa9429c9680ac4cf19e93842511f8"}
Oct 31 00:33:11.106587 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:33:11.106206    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-c54754479-2t99s"
Oct 31 00:33:11.336628 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:33:11.336537    4284 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-c54754479-2t99s" podStartSLOduration=3.147659662 podStartE2EDuration="8.336522663s" totalImagesPullingTime="4.784477166s" totalInitContainerRuntime="404.385835ms" isStatefulPod=false podCreationTimestamp="2026-10-31 00:33:03 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:33:11.288953546 +0000 UTC m=+305.433960461" watchObservedRunningTime="2026-10-31 00:33:11.336522663 +0000 UTC m=+305.481529634"
Oct 31 00:33:12.129181 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:33:12.129131    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:33:12.462496 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:33:12.462461    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:33:12.673870 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:33:12.673818    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:33:13.339290 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:33:13.339240    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:33:13.766496 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:33:13.766467    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:33:13.921614 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:33:13.921583    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:33:14.127672 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:33:14.127593    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:33:15.160547 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:33:15.160513    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:33:17.012938 ip-10-0-132-173 kubenswrapper[4284]: E1031 00:33:17.012856    4284 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fd8sp" containerName="extractor"
Oct 31 00:33:17.381359 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:33:17.381277    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:33:17.578362 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:33:17.578333    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:33:19.021682 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:33:19.021647    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:33:19.221309 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:33:19.221277    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:33:21.434772 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:33:21.434739    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:33:21.949614 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:33:21.949578    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:33:22.111643 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:33:22.111610    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-c54754479-2t99s"
Oct 31 00:33:22.556237 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:33:22.556183    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:33:22.769050 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:33:22.769019    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:33:25.505730 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:33:25.505687    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:33:25.811113 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:33:25.811009    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:33:27.225883 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:33:27.225837    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:33:28.209286 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:33:28.209244    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:33:28.476504 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:33:28.476407    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:33:31.810413 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:33:31.810369    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:33:32.030366 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:33:32.030333    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:33:32.552717 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:33:32.552684    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:33:32.708904 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:33:32.708851    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:33:32.867722 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:33:32.867637    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:33:33.150403 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:33:33.150314    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:33:35.172162 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:33:35.172110    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:33:35.379628 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:33:35.379595    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:33:35.734495 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:33:35.734464    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:33:35.948032 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:33:35.948002    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:33:36.403193 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:33:36.403151    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:33:37.604756 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:33:37.604720    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:33:37.682843 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:33:37.682808    4284 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-sl99p"]
Oct 31 00:33:37.685469 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:33:37.685449    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-74dfcfb894-sl99p"
Oct 31 00:33:37.736382 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:33:37.736353    4284 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Oct 31 00:33:37.760762 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:33:37.760730    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mx7p9\" (UniqueName: \"kubernetes.io/projected/4d383179-7c2c-4db9-aae4-c25e97e00620-kube-api-access-mx7p9\") pod \"limitador-limitador-74dfcfb894-sl99p\" (UID: \"4d383179-7c2c-4db9-aae4-c25e97e00620\") " pod="kuadrant-system/limitador-limitador-74dfcfb894-sl99p"
Oct 31 00:33:37.760928 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:33:37.760778    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/4d383179-7c2c-4db9-aae4-c25e97e00620-config-file\") pod \"limitador-limitador-74dfcfb894-sl99p\" (UID: \"4d383179-7c2c-4db9-aae4-c25e97e00620\") " pod="kuadrant-system/limitador-limitador-74dfcfb894-sl99p"
Oct 31 00:33:37.774960 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:33:37.774923    4284 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Oct 31 00:33:37.775097 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:33:37.774980    4284 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Oct 31 00:33:37.783439 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:33:37.783412    4284 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-8x6pf\""
Oct 31 00:33:37.784731 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:33:37.784712    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-sl99p"]
Oct 31 00:33:37.809362 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:33:37.809322    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:33:37.862052 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:33:37.861940    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mx7p9\" (UniqueName: \"kubernetes.io/projected/4d383179-7c2c-4db9-aae4-c25e97e00620-kube-api-access-mx7p9\") pod \"limitador-limitador-74dfcfb894-sl99p\" (UID: \"4d383179-7c2c-4db9-aae4-c25e97e00620\") " pod="kuadrant-system/limitador-limitador-74dfcfb894-sl99p"
Oct 31 00:33:37.862052 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:33:37.861995    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/4d383179-7c2c-4db9-aae4-c25e97e00620-config-file\") pod \"limitador-limitador-74dfcfb894-sl99p\" (UID: \"4d383179-7c2c-4db9-aae4-c25e97e00620\") " pod="kuadrant-system/limitador-limitador-74dfcfb894-sl99p"
Oct 31 00:33:37.862639 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:33:37.862620    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/4d383179-7c2c-4db9-aae4-c25e97e00620-config-file\") pod \"limitador-limitador-74dfcfb894-sl99p\" (UID: \"4d383179-7c2c-4db9-aae4-c25e97e00620\") " pod="kuadrant-system/limitador-limitador-74dfcfb894-sl99p"
Oct 31 00:33:37.870809 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:33:37.870783    4284 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-67865c945c-ffm6r"]
Oct 31 00:33:37.873110 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:33:37.873091    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-67865c945c-ffm6r"
Oct 31 00:33:37.929528 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:33:37.929499    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-sl99p"]
Oct 31 00:33:37.941143 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:33:37.941118    4284 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Oct 31 00:33:37.941561 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:33:37.941543    4284 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Oct 31 00:33:37.941633 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:33:37.941543    4284 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-wp9n9\""
Oct 31 00:33:37.950807 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:33:37.950779    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-67865c945c-ffm6r"]
Oct 31 00:33:37.950958 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:33:37.950882    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mx7p9\" (UniqueName: \"kubernetes.io/projected/4d383179-7c2c-4db9-aae4-c25e97e00620-kube-api-access-mx7p9\") pod \"limitador-limitador-74dfcfb894-sl99p\" (UID: \"4d383179-7c2c-4db9-aae4-c25e97e00620\") " pod="kuadrant-system/limitador-limitador-74dfcfb894-sl99p"
Oct 31 00:33:37.962516 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:33:37.962484    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/241f1b93-92b4-4480-93db-6b4121d8b112-tmp\") pod \"kube-auth-proxy-67865c945c-ffm6r\" (UID: \"241f1b93-92b4-4480-93db-6b4121d8b112\") " pod="openshift-ingress/kube-auth-proxy-67865c945c-ffm6r"
Oct 31 00:33:37.962651 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:33:37.962533    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/241f1b93-92b4-4480-93db-6b4121d8b112-tls-certs\") pod \"kube-auth-proxy-67865c945c-ffm6r\" (UID: \"241f1b93-92b4-4480-93db-6b4121d8b112\") " pod="openshift-ingress/kube-auth-proxy-67865c945c-ffm6r"
Oct 31 00:33:37.962651 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:33:37.962608    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xkzgw\" (UniqueName: \"kubernetes.io/projected/241f1b93-92b4-4480-93db-6b4121d8b112-kube-api-access-xkzgw\") pod \"kube-auth-proxy-67865c945c-ffm6r\" (UID: \"241f1b93-92b4-4480-93db-6b4121d8b112\") " pod="openshift-ingress/kube-auth-proxy-67865c945c-ffm6r"
Oct 31 00:33:37.997369 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:33:37.997334    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-74dfcfb894-sl99p"
Oct 31 00:33:38.063811 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:33:38.063778    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/241f1b93-92b4-4480-93db-6b4121d8b112-tls-certs\") pod \"kube-auth-proxy-67865c945c-ffm6r\" (UID: \"241f1b93-92b4-4480-93db-6b4121d8b112\") " pod="openshift-ingress/kube-auth-proxy-67865c945c-ffm6r"
Oct 31 00:33:38.063955 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:33:38.063821    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xkzgw\" (UniqueName: \"kubernetes.io/projected/241f1b93-92b4-4480-93db-6b4121d8b112-kube-api-access-xkzgw\") pod \"kube-auth-proxy-67865c945c-ffm6r\" (UID: \"241f1b93-92b4-4480-93db-6b4121d8b112\") " pod="openshift-ingress/kube-auth-proxy-67865c945c-ffm6r"
Oct 31 00:33:38.063955 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:33:38.063863    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/241f1b93-92b4-4480-93db-6b4121d8b112-tmp\") pod \"kube-auth-proxy-67865c945c-ffm6r\" (UID: \"241f1b93-92b4-4480-93db-6b4121d8b112\") " pod="openshift-ingress/kube-auth-proxy-67865c945c-ffm6r"
Oct 31 00:33:38.064324 ip-10-0-132-173 kubenswrapper[4284]: E1031 00:33:38.064306    4284 secret.go:189] Couldn't get secret openshift-ingress/kube-auth-proxy-tls: secret "kube-auth-proxy-tls" not found
Oct 31 00:33:38.064397 ip-10-0-132-173 kubenswrapper[4284]: E1031 00:33:38.064387    4284 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/241f1b93-92b4-4480-93db-6b4121d8b112-tls-certs podName:241f1b93-92b4-4480-93db-6b4121d8b112 nodeName:}" failed. No retries permitted until 2026-10-31 00:33:38.564366796 +0000 UTC m=+332.709373709 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/241f1b93-92b4-4480-93db-6b4121d8b112-tls-certs") pod "kube-auth-proxy-67865c945c-ffm6r" (UID: "241f1b93-92b4-4480-93db-6b4121d8b112") : secret "kube-auth-proxy-tls" not found
Oct 31 00:33:38.068510 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:33:38.068480    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/241f1b93-92b4-4480-93db-6b4121d8b112-tmp\") pod \"kube-auth-proxy-67865c945c-ffm6r\" (UID: \"241f1b93-92b4-4480-93db-6b4121d8b112\") " pod="openshift-ingress/kube-auth-proxy-67865c945c-ffm6r"
Oct 31 00:33:38.101398 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:33:38.101353    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:33:38.174159 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:33:38.174129    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xkzgw\" (UniqueName: \"kubernetes.io/projected/241f1b93-92b4-4480-93db-6b4121d8b112-kube-api-access-xkzgw\") pod \"kube-auth-proxy-67865c945c-ffm6r\" (UID: \"241f1b93-92b4-4480-93db-6b4121d8b112\") " pod="openshift-ingress/kube-auth-proxy-67865c945c-ffm6r"
Oct 31 00:33:38.311607 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:33:38.311566    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:33:38.365998 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:33:38.365967    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-sl99p"]
Oct 31 00:33:38.367775 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:33:38.367744    4284 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4d383179_7c2c_4db9_aae4_c25e97e00620.slice/crio-dbb89f8c620ca62a764a9db45b88184bffc37683cd5b60e496d2108f939156a1 WatchSource:0}: Error finding container dbb89f8c620ca62a764a9db45b88184bffc37683cd5b60e496d2108f939156a1: Status 404 returned error can't find the container with id dbb89f8c620ca62a764a9db45b88184bffc37683cd5b60e496d2108f939156a1
Oct 31 00:33:38.495594 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:33:38.495494    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:33:38.568556 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:33:38.568522    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/241f1b93-92b4-4480-93db-6b4121d8b112-tls-certs\") pod \"kube-auth-proxy-67865c945c-ffm6r\" (UID: \"241f1b93-92b4-4480-93db-6b4121d8b112\") " pod="openshift-ingress/kube-auth-proxy-67865c945c-ffm6r"
Oct 31 00:33:38.570980 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:33:38.570952    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/241f1b93-92b4-4480-93db-6b4121d8b112-tls-certs\") pod \"kube-auth-proxy-67865c945c-ffm6r\" (UID: \"241f1b93-92b4-4480-93db-6b4121d8b112\") " pod="openshift-ingress/kube-auth-proxy-67865c945c-ffm6r"
Oct 31 00:33:38.653480 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:33:38.653438    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:33:38.784422 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:33:38.784336    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-67865c945c-ffm6r"
Oct 31 00:33:38.808568 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:33:38.808534    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:33:39.142160 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:33:39.142071    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:33:39.152490 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:33:39.152469    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-67865c945c-ffm6r"]
Oct 31 00:33:39.154475 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:33:39.154448    4284 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod241f1b93_92b4_4480_93db_6b4121d8b112.slice/crio-2e3a9ec8ba3d9bda38e971b55a78a4d0a2725c9ac1db49ae663c53d5e03a0ea8 WatchSource:0}: Error finding container 2e3a9ec8ba3d9bda38e971b55a78a4d0a2725c9ac1db49ae663c53d5e03a0ea8: Status 404 returned error can't find the container with id 2e3a9ec8ba3d9bda38e971b55a78a4d0a2725c9ac1db49ae663c53d5e03a0ea8
Oct 31 00:33:39.205455 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:33:39.205397    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-67865c945c-ffm6r" event={"ID":"241f1b93-92b4-4480-93db-6b4121d8b112","Type":"ContainerStarted","Data":"2e3a9ec8ba3d9bda38e971b55a78a4d0a2725c9ac1db49ae663c53d5e03a0ea8"}
Oct 31 00:33:39.206477 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:33:39.206452    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-74dfcfb894-sl99p" event={"ID":"4d383179-7c2c-4db9-aae4-c25e97e00620","Type":"ContainerStarted","Data":"dbb89f8c620ca62a764a9db45b88184bffc37683cd5b60e496d2108f939156a1"}
Oct 31 00:33:39.299909 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:33:39.299877    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:33:39.444815 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:33:39.444771    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:33:39.595631 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:33:39.595592    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:33:39.769020 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:33:39.768919    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:33:39.924094 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:33:39.924060    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:33:40.469156 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:33:40.469116    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:33:40.643267 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:33:40.643160    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:33:43.226282 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:33:43.226240    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-67865c945c-ffm6r" event={"ID":"241f1b93-92b4-4480-93db-6b4121d8b112","Type":"ContainerStarted","Data":"48c6e8d336f8049ebb9f218576ddbd043ebf7fdb92cc0e40bb97493b9c6a0f0f"}
Oct 31 00:33:43.339889 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:33:43.339836    4284 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-67865c945c-ffm6r" podStartSLOduration=2.618724603 podStartE2EDuration="6.339821954s" totalImagesPullingTime="3.721097351s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 00:33:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:33:43.338728305 +0000 UTC m=+337.483735221" watchObservedRunningTime="2026-10-31 00:33:43.339821954 +0000 UTC m=+337.484828877"
Oct 31 00:33:44.230906 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:33:44.230872    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-74dfcfb894-sl99p" event={"ID":"4d383179-7c2c-4db9-aae4-c25e97e00620","Type":"ContainerStarted","Data":"122e107b37c1686bf06498c87713058ebe0dbb98db320df45e4153b15fef38a6"}
Oct 31 00:33:44.343461 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:33:44.343412    4284 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-74dfcfb894-sl99p" podStartSLOduration=2.322029425 podStartE2EDuration="7.343397676s" totalImagesPullingTime="5.021368251s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 00:33:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:33:44.341802491 +0000 UTC m=+338.486809407" watchObservedRunningTime="2026-10-31 00:33:44.343397676 +0000 UTC m=+338.488404592"
Oct 31 00:33:45.234753 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:33:45.234717    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-74dfcfb894-sl99p"
Oct 31 00:33:45.752651 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:33:45.752612    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:33:45.960371 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:33:45.960184    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:33:47.012636 ip-10-0-132-173 kubenswrapper[4284]: E1031 00:33:47.012600    4284 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fd8sp" containerName="extractor"
Oct 31 00:33:48.199136 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:33:48.199103    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:33:48.399505 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:33:48.399471    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:33:50.600296 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:33:50.600240    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:33:54.385567 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:33:54.385531    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:33:54.587021 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:33:54.586988    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:33:54.737347 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:33:54.737305    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:33:56.241644 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:33:56.241617    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-74dfcfb894-sl99p"
Oct 31 00:33:58.084401 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:33:58.084370    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-sl99p"]
Oct 31 00:33:58.084789 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:33:58.084593    4284 kuberuntime_container.go:898] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-74dfcfb894-sl99p" podUID="4d383179-7c2c-4db9-aae4-c25e97e00620" containerName="limitador" containerID="cri-o://122e107b37c1686bf06498c87713058ebe0dbb98db320df45e4153b15fef38a6" gracePeriod=30
Oct 31 00:33:58.618946 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:33:58.618923    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-74dfcfb894-sl99p"
Oct 31 00:33:58.733234 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:33:58.733180    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4d383179-7c2c-4db9-aae4-c25e97e00620-kube-api-access-mx7p9\" (UniqueName: \"kubernetes.io/projected/4d383179-7c2c-4db9-aae4-c25e97e00620-kube-api-access-mx7p9\") pod \"4d383179-7c2c-4db9-aae4-c25e97e00620\" (UID: \"4d383179-7c2c-4db9-aae4-c25e97e00620\") "
Oct 31 00:33:58.733399 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:33:58.733287    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/4d383179-7c2c-4db9-aae4-c25e97e00620-config-file\" (UniqueName: \"kubernetes.io/configmap/4d383179-7c2c-4db9-aae4-c25e97e00620-config-file\") pod \"4d383179-7c2c-4db9-aae4-c25e97e00620\" (UID: \"4d383179-7c2c-4db9-aae4-c25e97e00620\") "
Oct 31 00:33:58.733674 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:33:58.733641    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4d383179-7c2c-4db9-aae4-c25e97e00620-config-file" pod "4d383179-7c2c-4db9-aae4-c25e97e00620" (UID: "4d383179-7c2c-4db9-aae4-c25e97e00620"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 00:33:58.735675 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:33:58.735650    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4d383179-7c2c-4db9-aae4-c25e97e00620-kube-api-access-mx7p9" pod "4d383179-7c2c-4db9-aae4-c25e97e00620" (UID: "4d383179-7c2c-4db9-aae4-c25e97e00620"). InnerVolumeSpecName "kube-api-access-mx7p9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 00:33:58.834149 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:33:58.834107    4284 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/4d383179-7c2c-4db9-aae4-c25e97e00620-config-file\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:33:58.834149 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:33:58.834141    4284 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mx7p9\" (UniqueName: \"kubernetes.io/projected/4d383179-7c2c-4db9-aae4-c25e97e00620-kube-api-access-mx7p9\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:33:59.291194 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:33:59.291160    4284 generic.go:356] "Generic (PLEG): container finished" podID="4d383179-7c2c-4db9-aae4-c25e97e00620" containerID="122e107b37c1686bf06498c87713058ebe0dbb98db320df45e4153b15fef38a6" exitCode=0
Oct 31 00:33:59.291724 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:33:59.291226    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-74dfcfb894-sl99p"
Oct 31 00:33:59.291724 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:33:59.291253    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-74dfcfb894-sl99p" event={"ID":"4d383179-7c2c-4db9-aae4-c25e97e00620","Type":"ContainerDied","Data":"122e107b37c1686bf06498c87713058ebe0dbb98db320df45e4153b15fef38a6"}
Oct 31 00:33:59.291724 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:33:59.291310    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-74dfcfb894-sl99p" event={"ID":"4d383179-7c2c-4db9-aae4-c25e97e00620","Type":"ContainerDied","Data":"dbb89f8c620ca62a764a9db45b88184bffc37683cd5b60e496d2108f939156a1"}
Oct 31 00:33:59.291724 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:33:59.291334    4284 scope.go:122] "RemoveContainer" containerID="122e107b37c1686bf06498c87713058ebe0dbb98db320df45e4153b15fef38a6"
Oct 31 00:33:59.299668 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:33:59.299647    4284 scope.go:122] "RemoveContainer" containerID="122e107b37c1686bf06498c87713058ebe0dbb98db320df45e4153b15fef38a6"
Oct 31 00:33:59.299951 ip-10-0-132-173 kubenswrapper[4284]: E1031 00:33:59.299933    4284 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"122e107b37c1686bf06498c87713058ebe0dbb98db320df45e4153b15fef38a6\": container with ID starting with 122e107b37c1686bf06498c87713058ebe0dbb98db320df45e4153b15fef38a6 not found: ID does not exist" containerID="122e107b37c1686bf06498c87713058ebe0dbb98db320df45e4153b15fef38a6"
Oct 31 00:33:59.300036 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:33:59.299960    4284 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"122e107b37c1686bf06498c87713058ebe0dbb98db320df45e4153b15fef38a6"} err="failed to get container status \"122e107b37c1686bf06498c87713058ebe0dbb98db320df45e4153b15fef38a6\": rpc error: code = NotFound desc = could not find container \"122e107b37c1686bf06498c87713058ebe0dbb98db320df45e4153b15fef38a6\": container with ID starting with 122e107b37c1686bf06498c87713058ebe0dbb98db320df45e4153b15fef38a6 not found: ID does not exist"
Oct 31 00:33:59.447435 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:33:59.447401    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-sl99p"]
Oct 31 00:33:59.495080 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:33:59.495040    4284 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-sl99p"]
Oct 31 00:33:59.959999 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:33:59.959962    4284 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="4d383179-7c2c-4db9-aae4-c25e97e00620" path="/var/lib/kubelet/pods/4d383179-7c2c-4db9-aae4-c25e97e00620/volumes"
Oct 31 00:34:03.594867 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:34:03.594828    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:34:03.753689 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:34:03.753658    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:34:04.307643 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:34:04.307614    4284 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-7cdbf6dc44-rw68z"]
Oct 31 00:34:04.307929 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:34:04.307916    4284 cpu_manager.go:414] "containerMap: removing container" podUID="4d383179-7c2c-4db9-aae4-c25e97e00620" containerName="limitador"
Oct 31 00:34:04.307983 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:34:04.307931    4284 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4d383179-7c2c-4db9-aae4-c25e97e00620" containerName="limitador"
Oct 31 00:34:04.308018 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:34:04.308003    4284 memory_manager.go:365] "RemoveStaleState removing state" podUID="4d383179-7c2c-4db9-aae4-c25e97e00620" containerName="limitador"
Oct 31 00:34:04.312325 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:34:04.312299    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-rw68z"
Oct 31 00:34:04.372269 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:34:04.372238    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/6e4242f6-ec4d-4d19-a7e2-47611818369a-data\") pod \"postgres-7cdbf6dc44-rw68z\" (UID: \"6e4242f6-ec4d-4d19-a7e2-47611818369a\") " pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-rw68z"
Oct 31 00:34:04.372269 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:34:04.372276    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-986s4\" (UniqueName: \"kubernetes.io/projected/6e4242f6-ec4d-4d19-a7e2-47611818369a-kube-api-access-986s4\") pod \"postgres-7cdbf6dc44-rw68z\" (UID: \"6e4242f6-ec4d-4d19-a7e2-47611818369a\") " pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-rw68z"
Oct 31 00:34:04.398277 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:34:04.398239    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:34:04.410061 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:34:04.410033    4284 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-z46f5\""
Oct 31 00:34:04.410231 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:34:04.410066    4284 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\""
Oct 31 00:34:04.411667 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:34:04.411651    4284 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Oct 31 00:34:04.411763 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:34:04.411746    4284 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Oct 31 00:34:04.412739 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:34:04.412723    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-7cdbf6dc44-rw68z"]
Oct 31 00:34:04.472702 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:34:04.472663    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/6e4242f6-ec4d-4d19-a7e2-47611818369a-data\") pod \"postgres-7cdbf6dc44-rw68z\" (UID: \"6e4242f6-ec4d-4d19-a7e2-47611818369a\") " pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-rw68z"
Oct 31 00:34:04.472702 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:34:04.472701    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-986s4\" (UniqueName: \"kubernetes.io/projected/6e4242f6-ec4d-4d19-a7e2-47611818369a-kube-api-access-986s4\") pod \"postgres-7cdbf6dc44-rw68z\" (UID: \"6e4242f6-ec4d-4d19-a7e2-47611818369a\") " pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-rw68z"
Oct 31 00:34:04.473142 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:34:04.473125    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/6e4242f6-ec4d-4d19-a7e2-47611818369a-data\") pod \"postgres-7cdbf6dc44-rw68z\" (UID: \"6e4242f6-ec4d-4d19-a7e2-47611818369a\") " pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-rw68z"
Oct 31 00:34:04.531720 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:34:04.531676    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-986s4\" (UniqueName: \"kubernetes.io/projected/6e4242f6-ec4d-4d19-a7e2-47611818369a-kube-api-access-986s4\") pod \"postgres-7cdbf6dc44-rw68z\" (UID: \"6e4242f6-ec4d-4d19-a7e2-47611818369a\") " pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-rw68z"
Oct 31 00:34:04.624550 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:34:04.624454    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-rw68z"
Oct 31 00:34:04.890753 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:34:04.886005    4284 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-module-controller-manager-7566cf5fc8-9l4r5"]
Oct 31 00:34:04.892282 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:34:04.892262    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-9l4r5"
Oct 31 00:34:04.942539 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:34:04.942506    4284 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-module-controller-manager-dockercfg-r6pb6\""
Oct 31 00:34:04.942539 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:34:04.942522    4284 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Oct 31 00:34:04.942762 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:34:04.942524    4284 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Oct 31 00:34:04.976273 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:34:04.976207    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e452dfac-477e-46b3-8300-e90197abb7e9-tmp\") pod \"kserve-module-controller-manager-7566cf5fc8-9l4r5\" (UID: \"e452dfac-477e-46b3-8300-e90197abb7e9\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-9l4r5"
Oct 31 00:34:04.976425 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:34:04.976277    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e452dfac-477e-46b3-8300-e90197abb7e9-trusted-ca\") pod \"kserve-module-controller-manager-7566cf5fc8-9l4r5\" (UID: \"e452dfac-477e-46b3-8300-e90197abb7e9\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-9l4r5"
Oct 31 00:34:04.976425 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:34:04.976337    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8cdrm\" (UniqueName: \"kubernetes.io/projected/e452dfac-477e-46b3-8300-e90197abb7e9-kube-api-access-8cdrm\") pod \"kserve-module-controller-manager-7566cf5fc8-9l4r5\" (UID: \"e452dfac-477e-46b3-8300-e90197abb7e9\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-9l4r5"
Oct 31 00:34:04.976425 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:34:04.976414    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/e452dfac-477e-46b3-8300-e90197abb7e9-manifests\") pod \"kserve-module-controller-manager-7566cf5fc8-9l4r5\" (UID: \"e452dfac-477e-46b3-8300-e90197abb7e9\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-9l4r5"
Oct 31 00:34:04.991632 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:34:04.991604    4284 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6e4242f6_ec4d_4d19_a7e2_47611818369a.slice/crio-cc2b81bbd128b65a58523c42941ed25b355a0c88c01397a68dea33b3ab0f7d8b WatchSource:0}: Error finding container cc2b81bbd128b65a58523c42941ed25b355a0c88c01397a68dea33b3ab0f7d8b: Status 404 returned error can't find the container with id cc2b81bbd128b65a58523c42941ed25b355a0c88c01397a68dea33b3ab0f7d8b
Oct 31 00:34:05.077054 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:34:05.077017    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e452dfac-477e-46b3-8300-e90197abb7e9-tmp\") pod \"kserve-module-controller-manager-7566cf5fc8-9l4r5\" (UID: \"e452dfac-477e-46b3-8300-e90197abb7e9\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-9l4r5"
Oct 31 00:34:05.077054 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:34:05.077055    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e452dfac-477e-46b3-8300-e90197abb7e9-trusted-ca\") pod \"kserve-module-controller-manager-7566cf5fc8-9l4r5\" (UID: \"e452dfac-477e-46b3-8300-e90197abb7e9\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-9l4r5"
Oct 31 00:34:05.077312 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:34:05.077151    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8cdrm\" (UniqueName: \"kubernetes.io/projected/e452dfac-477e-46b3-8300-e90197abb7e9-kube-api-access-8cdrm\") pod \"kserve-module-controller-manager-7566cf5fc8-9l4r5\" (UID: \"e452dfac-477e-46b3-8300-e90197abb7e9\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-9l4r5"
Oct 31 00:34:05.077312 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:34:05.077172    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/e452dfac-477e-46b3-8300-e90197abb7e9-manifests\") pod \"kserve-module-controller-manager-7566cf5fc8-9l4r5\" (UID: \"e452dfac-477e-46b3-8300-e90197abb7e9\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-9l4r5"
Oct 31 00:34:05.077540 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:34:05.077524    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/e452dfac-477e-46b3-8300-e90197abb7e9-manifests\") pod \"kserve-module-controller-manager-7566cf5fc8-9l4r5\" (UID: \"e452dfac-477e-46b3-8300-e90197abb7e9\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-9l4r5"
Oct 31 00:34:05.077580 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:34:05.077521    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e452dfac-477e-46b3-8300-e90197abb7e9-tmp\") pod \"kserve-module-controller-manager-7566cf5fc8-9l4r5\" (UID: \"e452dfac-477e-46b3-8300-e90197abb7e9\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-9l4r5"
Oct 31 00:34:05.086026 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:34:05.086002    4284 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"odh-trusted-ca-bundle\""
Oct 31 00:34:05.087295 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:34:05.087269    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-module-controller-manager-7566cf5fc8-9l4r5"]
Oct 31 00:34:05.087860 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:34:05.087843    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e452dfac-477e-46b3-8300-e90197abb7e9-trusted-ca\") pod \"kserve-module-controller-manager-7566cf5fc8-9l4r5\" (UID: \"e452dfac-477e-46b3-8300-e90197abb7e9\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-9l4r5"
Oct 31 00:34:05.088230 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:34:05.088205    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-7cdbf6dc44-rw68z"]
Oct 31 00:34:05.136504 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:34:05.136473    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8cdrm\" (UniqueName: \"kubernetes.io/projected/e452dfac-477e-46b3-8300-e90197abb7e9-kube-api-access-8cdrm\") pod \"kserve-module-controller-manager-7566cf5fc8-9l4r5\" (UID: \"e452dfac-477e-46b3-8300-e90197abb7e9\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-9l4r5"
Oct 31 00:34:05.204684 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:34:05.204650    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-9l4r5"
Oct 31 00:34:05.317134 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:34:05.317097    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-rw68z" event={"ID":"6e4242f6-ec4d-4d19-a7e2-47611818369a","Type":"ContainerStarted","Data":"cc2b81bbd128b65a58523c42941ed25b355a0c88c01397a68dea33b3ab0f7d8b"}
Oct 31 00:34:05.615703 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:34:05.615561    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-module-controller-manager-7566cf5fc8-9l4r5"]
Oct 31 00:34:05.617608 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:34:05.617580    4284 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode452dfac_477e_46b3_8300_e90197abb7e9.slice/crio-f325aa3ed883ef93b4532f815b451e020aefaa16116acd63b10f595d64bc1276 WatchSource:0}: Error finding container f325aa3ed883ef93b4532f815b451e020aefaa16116acd63b10f595d64bc1276: Status 404 returned error can't find the container with id f325aa3ed883ef93b4532f815b451e020aefaa16116acd63b10f595d64bc1276
Oct 31 00:34:06.321440 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:34:06.321362    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-9l4r5" event={"ID":"e452dfac-477e-46b3-8300-e90197abb7e9","Type":"ContainerStarted","Data":"f325aa3ed883ef93b4532f815b451e020aefaa16116acd63b10f595d64bc1276"}
Oct 31 00:34:07.933930 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:34:07.933898    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:34:08.138515 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:34:08.138377    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:34:09.336980 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:34:09.336925    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-9l4r5" event={"ID":"e452dfac-477e-46b3-8300-e90197abb7e9","Type":"ContainerStarted","Data":"987dca80ee650093eedaee0e039c09deb8d882582334ef714972e31c1f828c99"}
Oct 31 00:34:09.337443 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:34:09.337079    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-9l4r5"
Oct 31 00:34:09.503256 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:34:09.503187    4284 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-9l4r5" podStartSLOduration=2.3276809800000002 podStartE2EDuration="5.503174018s" totalImagesPullingTime="3.175493038s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 00:34:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:34:09.502526462 +0000 UTC m=+363.647533378" watchObservedRunningTime="2026-10-31 00:34:09.503174018 +0000 UTC m=+363.648180933"
Oct 31 00:34:12.055067 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:34:12.055031    4284 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\""
Oct 31 00:34:12.350879 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:34:12.350787    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-rw68z" event={"ID":"6e4242f6-ec4d-4d19-a7e2-47611818369a","Type":"ContainerStarted","Data":"df0cd5a7e4e99295c7a25870e6e717c3233ec82c3cbd81bfb907ef43e30b583d"}
Oct 31 00:34:12.351047 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:34:12.351019    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-rw68z"
Oct 31 00:34:12.462438 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:34:12.462388    4284 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-rw68z" podStartSLOduration=1.450697799 podStartE2EDuration="8.462373881s" totalImagesPullingTime="7.011676082s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 00:34:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:34:12.461927149 +0000 UTC m=+366.606934081" watchObservedRunningTime="2026-10-31 00:34:12.462373881 +0000 UTC m=+366.607380797"
Oct 31 00:34:17.012788 ip-10-0-132-173 kubenswrapper[4284]: E1031 00:34:17.012743    4284 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fd8sp" containerName="extractor"
Oct 31 00:34:18.384671 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:34:18.384638    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-rw68z"
Oct 31 00:34:26.028248 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:34:26.028188    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:34:26.180989 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:34:26.180954    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:34:26.338359 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:34:26.338269    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:34:26.517887 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:34:26.517849    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:34:26.722111 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:34:26.722076    4284 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-6685f84c75-8wmg2"]
Oct 31 00:34:26.727795 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:34:26.727774    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6685f84c75-8wmg2"
Oct 31 00:34:26.828346 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:34:26.828317    4284 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-7zs4p\""
Oct 31 00:34:26.828544 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:34:26.828368    4284 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Oct 31 00:34:26.828544 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:34:26.828408    4284 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Oct 31 00:34:26.828544 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:34:26.828481    4284 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Oct 31 00:34:26.829577 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:34:26.829558    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6685f84c75-8wmg2"]
Oct 31 00:34:26.848718 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:34:26.848696    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8b5a139b-f0ff-4d24-b17d-cd6698ff3de5-webhook-cert\") pod \"maas-controller-6685f84c75-8wmg2\" (UID: \"8b5a139b-f0ff-4d24-b17d-cd6698ff3de5\") " pod="opendatahub/maas-controller-6685f84c75-8wmg2"
Oct 31 00:34:26.848866 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:34:26.848725    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8b5a139b-f0ff-4d24-b17d-cd6698ff3de5-metrics-certs\") pod \"maas-controller-6685f84c75-8wmg2\" (UID: \"8b5a139b-f0ff-4d24-b17d-cd6698ff3de5\") " pod="opendatahub/maas-controller-6685f84c75-8wmg2"
Oct 31 00:34:26.848866 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:34:26.848747    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6n27w\" (UniqueName: \"kubernetes.io/projected/8b5a139b-f0ff-4d24-b17d-cd6698ff3de5-kube-api-access-6n27w\") pod \"maas-controller-6685f84c75-8wmg2\" (UID: \"8b5a139b-f0ff-4d24-b17d-cd6698ff3de5\") " pod="opendatahub/maas-controller-6685f84c75-8wmg2"
Oct 31 00:34:26.949149 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:34:26.949118    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8b5a139b-f0ff-4d24-b17d-cd6698ff3de5-webhook-cert\") pod \"maas-controller-6685f84c75-8wmg2\" (UID: \"8b5a139b-f0ff-4d24-b17d-cd6698ff3de5\") " pod="opendatahub/maas-controller-6685f84c75-8wmg2"
Oct 31 00:34:26.949149 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:34:26.949153    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8b5a139b-f0ff-4d24-b17d-cd6698ff3de5-metrics-certs\") pod \"maas-controller-6685f84c75-8wmg2\" (UID: \"8b5a139b-f0ff-4d24-b17d-cd6698ff3de5\") " pod="opendatahub/maas-controller-6685f84c75-8wmg2"
Oct 31 00:34:26.949410 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:34:26.949235    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6n27w\" (UniqueName: \"kubernetes.io/projected/8b5a139b-f0ff-4d24-b17d-cd6698ff3de5-kube-api-access-6n27w\") pod \"maas-controller-6685f84c75-8wmg2\" (UID: \"8b5a139b-f0ff-4d24-b17d-cd6698ff3de5\") " pod="opendatahub/maas-controller-6685f84c75-8wmg2"
Oct 31 00:34:26.951731 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:34:26.951698    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8b5a139b-f0ff-4d24-b17d-cd6698ff3de5-metrics-certs\") pod \"maas-controller-6685f84c75-8wmg2\" (UID: \"8b5a139b-f0ff-4d24-b17d-cd6698ff3de5\") " pod="opendatahub/maas-controller-6685f84c75-8wmg2"
Oct 31 00:34:26.951871 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:34:26.951816    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8b5a139b-f0ff-4d24-b17d-cd6698ff3de5-webhook-cert\") pod \"maas-controller-6685f84c75-8wmg2\" (UID: \"8b5a139b-f0ff-4d24-b17d-cd6698ff3de5\") " pod="opendatahub/maas-controller-6685f84c75-8wmg2"
Oct 31 00:34:27.008416 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:34:27.008337    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6n27w\" (UniqueName: \"kubernetes.io/projected/8b5a139b-f0ff-4d24-b17d-cd6698ff3de5-kube-api-access-6n27w\") pod \"maas-controller-6685f84c75-8wmg2\" (UID: \"8b5a139b-f0ff-4d24-b17d-cd6698ff3de5\") " pod="opendatahub/maas-controller-6685f84c75-8wmg2"
Oct 31 00:34:27.038290 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:34:27.038246    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6685f84c75-8wmg2"
Oct 31 00:34:27.402625 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:34:27.402549    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6685f84c75-8wmg2"]
Oct 31 00:34:27.433515 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:34:27.433486    4284 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8b5a139b_f0ff_4d24_b17d_cd6698ff3de5.slice/crio-29af2f080dce233b56fbcce79ed0abbea0912fbf76304167c6867969739ce565 WatchSource:0}: Error finding container 29af2f080dce233b56fbcce79ed0abbea0912fbf76304167c6867969739ce565: Status 404 returned error can't find the container with id 29af2f080dce233b56fbcce79ed0abbea0912fbf76304167c6867969739ce565
Oct 31 00:34:28.413163 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:34:28.413121    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6685f84c75-8wmg2" event={"ID":"8b5a139b-f0ff-4d24-b17d-cd6698ff3de5","Type":"ContainerStarted","Data":"29af2f080dce233b56fbcce79ed0abbea0912fbf76304167c6867969739ce565"}
Oct 31 00:34:29.806500 ip-10-0-132-173 kubenswrapper[4284]: E1031 00:34:29.806444    4284 log.go:32] "PullImage from image service failed" err="rpc error: code = Unknown desc = unable to pull image: copying system image from manifest list: parsing image configuration: Get \"https://cdn01.quay.io/quayio-production-s3/sha256/3a/3a914942e67880a01f9e852006dce0a26dcad1ee82cb3bbda1129fca49ddd887?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIATAAF2YHTGR23ZTE6%2F20261031%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20261031T003428Z&X-Amz-Expires=600&X-Amz-SignedHeaders=host&X-Amz-Signature=69f052d2fdd4a4042c9a49a39ba3465c837768d213493b7928e6c89d79a4efc3&region=us-east-1&namespace=opendatahub&username=openshift-release-dev+ocm_access_f62e7711b3d44085bacf45dd1a94949f&repo_name=maas-controller&akamai_signature=exp=1793407768~hmac=f8f4d2881e049de712b5f289e4e095e91e284a51539f42e5344895f877d1f8f7\": EOF" image="quay.io/opendatahub/maas-controller@sha256:fe4bd0d7cdd1cfb8cbce19cd8c68adc9b1a09d9153c15cf3222203c943fd7118"
Oct 31 00:34:29.806928 ip-10-0-132-173 kubenswrapper[4284]: E1031 00:34:29.806675    4284 kuberuntime_manager.go:1707] "Unhandled Error" err="container manager start failed in pod maas-controller-6685f84c75-8wmg2_opendatahub(8b5a139b-f0ff-4d24-b17d-cd6698ff3de5): ErrImagePull: unable to pull image: copying system image from manifest list: parsing image configuration: Get \"https://cdn01.quay.io/quayio-production-s3/sha256/3a/3a914942e67880a01f9e852006dce0a26dcad1ee82cb3bbda1129fca49ddd887?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIATAAF2YHTGR23ZTE6%2F20261031%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20261031T003428Z&X-Amz-Expires=600&X-Amz-SignedHeaders=host&X-Amz-Signature=69f052d2fdd4a4042c9a49a39ba3465c837768d213493b7928e6c89d79a4efc3&region=us-east-1&namespace=opendatahub&username=openshift-release-dev+ocm_access_f62e7711b3d44085bacf45dd1a94949f&repo_name=maas-controller&akamai_signature=exp=1793407768~hmac=f8f4d2881e049de712b5f289e4e095e91e284a51539f42e5344895f877d1f8f7\": EOF" logger="UnhandledError"
Oct 31 00:34:29.806928 ip-10-0-132-173 kubenswrapper[4284]: E1031 00:34:29.806726    4284 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with ErrImagePull: \"unable to pull image: copying system image from manifest list: parsing image configuration: Get \\\"https://cdn01.quay.io/quayio-production-s3/sha256/3a/3a914942e67880a01f9e852006dce0a26dcad1ee82cb3bbda1129fca49ddd887?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIATAAF2YHTGR23ZTE6%2F20261031%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20261031T003428Z&X-Amz-Expires=600&X-Amz-SignedHeaders=host&X-Amz-Signature=69f052d2fdd4a4042c9a49a39ba3465c837768d213493b7928e6c89d79a4efc3&region=us-east-1&namespace=opendatahub&username=openshift-release-dev+ocm_access_f62e7711b3d44085bacf45dd1a94949f&repo_name=maas-controller&akamai_signature=exp=1793407768~hmac=f8f4d2881e049de712b5f289e4e095e91e284a51539f42e5344895f877d1f8f7\\\": EOF\"" pod="opendatahub/maas-controller-6685f84c75-8wmg2" podUID="8b5a139b-f0ff-4d24-b17d-cd6698ff3de5"
Oct 31 00:34:30.342474 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:34:30.342445    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-9l4r5"
Oct 31 00:34:30.421159 ip-10-0-132-173 kubenswrapper[4284]: E1031 00:34:30.421119    4284 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/opendatahub/maas-controller@sha256:fe4bd0d7cdd1cfb8cbce19cd8c68adc9b1a09d9153c15cf3222203c943fd7118\\\": ErrImagePull: unable to pull image: copying system image from manifest list: parsing image configuration: Get \\\"https://cdn01.quay.io/quayio-production-s3/sha256/3a/3a914942e67880a01f9e852006dce0a26dcad1ee82cb3bbda1129fca49ddd887?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIATAAF2YHTGR23ZTE6%2F20261031%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20261031T003428Z&X-Amz-Expires=600&X-Amz-SignedHeaders=host&X-Amz-Signature=69f052d2fdd4a4042c9a49a39ba3465c837768d213493b7928e6c89d79a4efc3&region=us-east-1&namespace=opendatahub&username=openshift-release-dev+ocm_access_f62e7711b3d44085bacf45dd1a94949f&repo_name=maas-controller&akamai_signature=exp=1793407768~hmac=f8f4d2881e049de712b5f289e4e095e91e284a51539f42e5344895f877d1f8f7\\\": EOF\"" pod="opendatahub/maas-controller-6685f84c75-8wmg2" podUID="8b5a139b-f0ff-4d24-b17d-cd6698ff3de5"
Oct 31 00:34:34.465559 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:34:34.465523    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:34:34.648029 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:34:34.647990    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:34:37.621930 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:34:37.621893    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:34:37.818450 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:34:37.818415    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:34:46.485005 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:34:46.484969    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6685f84c75-8wmg2" event={"ID":"8b5a139b-f0ff-4d24-b17d-cd6698ff3de5","Type":"ContainerStarted","Data":"07caa3cb94501e06a4b3b2f4ff0a4b1eed0e40b5e9e0059844f61758137722d8"}
Oct 31 00:34:46.485380 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:34:46.485238    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-6685f84c75-8wmg2"
Oct 31 00:34:46.644912 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:34:46.644852    4284 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/maas-controller-6685f84c75-8wmg2" podStartSLOduration=1.652699831 podStartE2EDuration="20.64483911s" totalImagesPullingTime="18.992139279s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 00:34:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=1 observedRunningTime="2026-10-31 00:34:46.644536966 +0000 UTC m=+400.789543882" watchObservedRunningTime="2026-10-31 00:34:46.64483911 +0000 UTC m=+400.789846025"
Oct 31 00:34:47.016018 ip-10-0-132-173 kubenswrapper[4284]: E1031 00:34:47.015973    4284 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fd8sp" containerName="extractor"
Oct 31 00:34:50.942951 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:34:50.942917    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:34:57.491673 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:34:57.491640    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-6685f84c75-8wmg2"
Oct 31 00:35:07.004975 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:35:07.004942    4284 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-bzn9w"]
Oct 31 00:35:07.009648 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:35:07.009624    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-bzn9w"
Oct 31 00:35:07.079100 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:35:07.079063    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-bzn9w"]
Oct 31 00:35:07.079677 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:35:07.079653    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/746ed5e5-2723-4768-8824-63b135583812-cacerts\") pod \"istiod-openshift-gateway-7ddd9cfdc8-bzn9w\" (UID: \"746ed5e5-2723-4768-8824-63b135583812\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-bzn9w"
Oct 31 00:35:07.079813 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:35:07.079687    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/746ed5e5-2723-4768-8824-63b135583812-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7ddd9cfdc8-bzn9w\" (UID: \"746ed5e5-2723-4768-8824-63b135583812\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-bzn9w"
Oct 31 00:35:07.079813 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:35:07.079745    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/746ed5e5-2723-4768-8824-63b135583812-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7ddd9cfdc8-bzn9w\" (UID: \"746ed5e5-2723-4768-8824-63b135583812\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-bzn9w"
Oct 31 00:35:07.079813 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:35:07.079782    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jrrmz\" (UniqueName: \"kubernetes.io/projected/746ed5e5-2723-4768-8824-63b135583812-kube-api-access-jrrmz\") pod \"istiod-openshift-gateway-7ddd9cfdc8-bzn9w\" (UID: \"746ed5e5-2723-4768-8824-63b135583812\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-bzn9w"
Oct 31 00:35:07.079948 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:35:07.079894    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/746ed5e5-2723-4768-8824-63b135583812-local-certs\") pod \"istiod-openshift-gateway-7ddd9cfdc8-bzn9w\" (UID: \"746ed5e5-2723-4768-8824-63b135583812\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-bzn9w"
Oct 31 00:35:07.080052 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:35:07.080027    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/746ed5e5-2723-4768-8824-63b135583812-istio-token\") pod \"istiod-openshift-gateway-7ddd9cfdc8-bzn9w\" (UID: \"746ed5e5-2723-4768-8824-63b135583812\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-bzn9w"
Oct 31 00:35:07.080098 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:35:07.080080    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/746ed5e5-2723-4768-8824-63b135583812-istio-kubeconfig\") pod \"istiod-openshift-gateway-7ddd9cfdc8-bzn9w\" (UID: \"746ed5e5-2723-4768-8824-63b135583812\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-bzn9w"
Oct 31 00:35:07.180735 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:35:07.180696    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/746ed5e5-2723-4768-8824-63b135583812-istio-token\") pod \"istiod-openshift-gateway-7ddd9cfdc8-bzn9w\" (UID: \"746ed5e5-2723-4768-8824-63b135583812\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-bzn9w"
Oct 31 00:35:07.180735 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:35:07.180743    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/746ed5e5-2723-4768-8824-63b135583812-istio-kubeconfig\") pod \"istiod-openshift-gateway-7ddd9cfdc8-bzn9w\" (UID: \"746ed5e5-2723-4768-8824-63b135583812\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-bzn9w"
Oct 31 00:35:07.181003 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:35:07.180798    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/746ed5e5-2723-4768-8824-63b135583812-cacerts\") pod \"istiod-openshift-gateway-7ddd9cfdc8-bzn9w\" (UID: \"746ed5e5-2723-4768-8824-63b135583812\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-bzn9w"
Oct 31 00:35:07.181003 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:35:07.180822    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/746ed5e5-2723-4768-8824-63b135583812-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7ddd9cfdc8-bzn9w\" (UID: \"746ed5e5-2723-4768-8824-63b135583812\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-bzn9w"
Oct 31 00:35:07.181003 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:35:07.180862    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/746ed5e5-2723-4768-8824-63b135583812-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7ddd9cfdc8-bzn9w\" (UID: \"746ed5e5-2723-4768-8824-63b135583812\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-bzn9w"
Oct 31 00:35:07.181003 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:35:07.180894    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jrrmz\" (UniqueName: \"kubernetes.io/projected/746ed5e5-2723-4768-8824-63b135583812-kube-api-access-jrrmz\") pod \"istiod-openshift-gateway-7ddd9cfdc8-bzn9w\" (UID: \"746ed5e5-2723-4768-8824-63b135583812\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-bzn9w"
Oct 31 00:35:07.181003 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:35:07.180917    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/746ed5e5-2723-4768-8824-63b135583812-local-certs\") pod \"istiod-openshift-gateway-7ddd9cfdc8-bzn9w\" (UID: \"746ed5e5-2723-4768-8824-63b135583812\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-bzn9w"
Oct 31 00:35:07.181646 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:35:07.181619    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/746ed5e5-2723-4768-8824-63b135583812-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7ddd9cfdc8-bzn9w\" (UID: \"746ed5e5-2723-4768-8824-63b135583812\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-bzn9w"
Oct 31 00:35:07.183387 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:35:07.183356    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/746ed5e5-2723-4768-8824-63b135583812-local-certs\") pod \"istiod-openshift-gateway-7ddd9cfdc8-bzn9w\" (UID: \"746ed5e5-2723-4768-8824-63b135583812\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-bzn9w"
Oct 31 00:35:07.183501 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:35:07.183486    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/746ed5e5-2723-4768-8824-63b135583812-cacerts\") pod \"istiod-openshift-gateway-7ddd9cfdc8-bzn9w\" (UID: \"746ed5e5-2723-4768-8824-63b135583812\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-bzn9w"
Oct 31 00:35:07.183558 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:35:07.183533    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/746ed5e5-2723-4768-8824-63b135583812-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7ddd9cfdc8-bzn9w\" (UID: \"746ed5e5-2723-4768-8824-63b135583812\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-bzn9w"
Oct 31 00:35:07.183632 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:35:07.183610    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/746ed5e5-2723-4768-8824-63b135583812-istio-kubeconfig\") pod \"istiod-openshift-gateway-7ddd9cfdc8-bzn9w\" (UID: \"746ed5e5-2723-4768-8824-63b135583812\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-bzn9w"
Oct 31 00:35:07.239750 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:35:07.239719    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/746ed5e5-2723-4768-8824-63b135583812-istio-token\") pod \"istiod-openshift-gateway-7ddd9cfdc8-bzn9w\" (UID: \"746ed5e5-2723-4768-8824-63b135583812\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-bzn9w"
Oct 31 00:35:07.239887 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:35:07.239856    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jrrmz\" (UniqueName: \"kubernetes.io/projected/746ed5e5-2723-4768-8824-63b135583812-kube-api-access-jrrmz\") pod \"istiod-openshift-gateway-7ddd9cfdc8-bzn9w\" (UID: \"746ed5e5-2723-4768-8824-63b135583812\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-bzn9w"
Oct 31 00:35:07.321037 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:35:07.320946    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-bzn9w"
Oct 31 00:35:07.744873 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:35:07.744845    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-bzn9w"]
Oct 31 00:35:07.746508 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:35:07.746483    4284 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod746ed5e5_2723_4768_8824_63b135583812.slice/crio-e40acf13a9f3ee3e7c64d46b1318513ef8aa19c285e14a6d74a42c3635c82365 WatchSource:0}: Error finding container e40acf13a9f3ee3e7c64d46b1318513ef8aa19c285e14a6d74a42c3635c82365: Status 404 returned error can't find the container with id e40acf13a9f3ee3e7c64d46b1318513ef8aa19c285e14a6d74a42c3635c82365
Oct 31 00:35:07.748837 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:35:07.748800    4284 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 00:35:07.748939 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:35:07.748880    4284 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 00:35:08.578052 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:35:08.578001    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-bzn9w" event={"ID":"746ed5e5-2723-4768-8824-63b135583812","Type":"ContainerStarted","Data":"93d436649382ec7232b6493d8a0357f17108d3fad041c4c9928e554d26c8ce2d"}
Oct 31 00:35:08.578592 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:35:08.578091    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-bzn9w"
Oct 31 00:35:08.578592 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:35:08.578107    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-bzn9w" event={"ID":"746ed5e5-2723-4768-8824-63b135583812","Type":"ContainerStarted","Data":"e40acf13a9f3ee3e7c64d46b1318513ef8aa19c285e14a6d74a42c3635c82365"}
Oct 31 00:35:08.579888 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:35:08.579861    4284 patch_prober.go:29] interesting pod/istiod-openshift-gateway-7ddd9cfdc8-bzn9w container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 31 00:35:08.579964 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:35:08.579913    4284 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-bzn9w" podUID="746ed5e5-2723-4768-8824-63b135583812" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 31 00:35:08.699750 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:35:08.699699    4284 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-bzn9w" podStartSLOduration=2.6996858379999997 podStartE2EDuration="2.699685838s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 00:35:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:35:08.699032317 +0000 UTC m=+422.844039233" watchObservedRunningTime="2026-10-31 00:35:08.699685838 +0000 UTC m=+422.844692753"
Oct 31 00:35:09.582938 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:35:09.582910    4284 patch_prober.go:29] interesting pod/istiod-openshift-gateway-7ddd9cfdc8-bzn9w container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 31 00:35:09.583360 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:35:09.582959    4284 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-bzn9w" podUID="746ed5e5-2723-4768-8824-63b135583812" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 31 00:35:11.807191 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:35:11.807153    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:35:12.025227 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:35:12.025178    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:35:12.184341 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:35:12.184312    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:35:12.330455 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:35:12.330424    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:35:12.508834 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:35:12.508718    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:35:12.583683 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:35:12.583653    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-bzn9w"
Oct 31 00:35:12.667991 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:35:12.667949    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:35:12.785106 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:35:12.785036    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7876bfcb74-876pt"]
Oct 31 00:35:12.785304 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:35:12.785278    4284 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-876pt" podUID="e8c333bb-6008-457c-b8a3-66bd654f012d" containerName="discovery" containerID="cri-o://e8e6c2cd8b38f39329b13b03cbb7b4bf9eb1effffd2bcb709b6d426a8b75c55b" gracePeriod=30
Oct 31 00:35:12.828462 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:35:12.828386    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:35:12.939976 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:35:12.939939    4284 patch_prober.go:29] interesting pod/istiod-openshift-gateway-7876bfcb74-876pt container/discovery namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.20:8080/ready\": dial tcp 10.134.0.20:8080: connect: connection refused" start-of-body=
Oct 31 00:35:12.940159 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:35:12.939996    4284 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-876pt" podUID="e8c333bb-6008-457c-b8a3-66bd654f012d" containerName="discovery" probeResult="failure" output="Get \"http://10.134.0.20:8080/ready\": dial tcp 10.134.0.20:8080: connect: connection refused"
Oct 31 00:35:13.024709 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:35:13.024687    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-876pt"
Oct 31 00:35:13.126376 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:35:13.126291    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/e8c333bb-6008-457c-b8a3-66bd654f012d-istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/e8c333bb-6008-457c-b8a3-66bd654f012d-istio-csr-dns-cert\") pod \"e8c333bb-6008-457c-b8a3-66bd654f012d\" (UID: \"e8c333bb-6008-457c-b8a3-66bd654f012d\") "
Oct 31 00:35:13.126376 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:35:13.126343    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e8c333bb-6008-457c-b8a3-66bd654f012d-istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/e8c333bb-6008-457c-b8a3-66bd654f012d-istio-csr-ca-configmap\") pod \"e8c333bb-6008-457c-b8a3-66bd654f012d\" (UID: \"e8c333bb-6008-457c-b8a3-66bd654f012d\") "
Oct 31 00:35:13.126376 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:35:13.126374    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e8c333bb-6008-457c-b8a3-66bd654f012d-kube-api-access-gvvl6\" (UniqueName: \"kubernetes.io/projected/e8c333bb-6008-457c-b8a3-66bd654f012d-kube-api-access-gvvl6\") pod \"e8c333bb-6008-457c-b8a3-66bd654f012d\" (UID: \"e8c333bb-6008-457c-b8a3-66bd654f012d\") "
Oct 31 00:35:13.126651 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:35:13.126395    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/e8c333bb-6008-457c-b8a3-66bd654f012d-cacerts\" (UniqueName: \"kubernetes.io/secret/e8c333bb-6008-457c-b8a3-66bd654f012d-cacerts\") pod \"e8c333bb-6008-457c-b8a3-66bd654f012d\" (UID: \"e8c333bb-6008-457c-b8a3-66bd654f012d\") "
Oct 31 00:35:13.126651 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:35:13.126427    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e8c333bb-6008-457c-b8a3-66bd654f012d-istio-token\" (UniqueName: \"kubernetes.io/projected/e8c333bb-6008-457c-b8a3-66bd654f012d-istio-token\") pod \"e8c333bb-6008-457c-b8a3-66bd654f012d\" (UID: \"e8c333bb-6008-457c-b8a3-66bd654f012d\") "
Oct 31 00:35:13.126651 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:35:13.126445    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e8c333bb-6008-457c-b8a3-66bd654f012d-local-certs\" (UniqueName: \"kubernetes.io/empty-dir/e8c333bb-6008-457c-b8a3-66bd654f012d-local-certs\") pod \"e8c333bb-6008-457c-b8a3-66bd654f012d\" (UID: \"e8c333bb-6008-457c-b8a3-66bd654f012d\") "
Oct 31 00:35:13.126651 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:35:13.126492    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/e8c333bb-6008-457c-b8a3-66bd654f012d-istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e8c333bb-6008-457c-b8a3-66bd654f012d-istio-kubeconfig\") pod \"e8c333bb-6008-457c-b8a3-66bd654f012d\" (UID: \"e8c333bb-6008-457c-b8a3-66bd654f012d\") "
Oct 31 00:35:13.126858 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:35:13.126810    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e8c333bb-6008-457c-b8a3-66bd654f012d-istio-csr-ca-configmap" pod "e8c333bb-6008-457c-b8a3-66bd654f012d" (UID: "e8c333bb-6008-457c-b8a3-66bd654f012d"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 00:35:13.129012 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:35:13.128975    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e8c333bb-6008-457c-b8a3-66bd654f012d-istio-csr-dns-cert" pod "e8c333bb-6008-457c-b8a3-66bd654f012d" (UID: "e8c333bb-6008-457c-b8a3-66bd654f012d"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 00:35:13.129144 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:35:13.129043    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e8c333bb-6008-457c-b8a3-66bd654f012d-local-certs" pod "e8c333bb-6008-457c-b8a3-66bd654f012d" (UID: "e8c333bb-6008-457c-b8a3-66bd654f012d"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:35:13.129199 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:35:13.129144    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e8c333bb-6008-457c-b8a3-66bd654f012d-istio-kubeconfig" pod "e8c333bb-6008-457c-b8a3-66bd654f012d" (UID: "e8c333bb-6008-457c-b8a3-66bd654f012d"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 00:35:13.129199 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:35:13.129170    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e8c333bb-6008-457c-b8a3-66bd654f012d-istio-token" pod "e8c333bb-6008-457c-b8a3-66bd654f012d" (UID: "e8c333bb-6008-457c-b8a3-66bd654f012d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 00:35:13.129432 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:35:13.129411    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e8c333bb-6008-457c-b8a3-66bd654f012d-cacerts" pod "e8c333bb-6008-457c-b8a3-66bd654f012d" (UID: "e8c333bb-6008-457c-b8a3-66bd654f012d"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 00:35:13.129548 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:35:13.129527    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e8c333bb-6008-457c-b8a3-66bd654f012d-kube-api-access-gvvl6" pod "e8c333bb-6008-457c-b8a3-66bd654f012d" (UID: "e8c333bb-6008-457c-b8a3-66bd654f012d"). InnerVolumeSpecName "kube-api-access-gvvl6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 00:35:13.227707 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:35:13.227668    4284 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/e8c333bb-6008-457c-b8a3-66bd654f012d-istio-csr-ca-configmap\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:35:13.227707 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:35:13.227698    4284 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gvvl6\" (UniqueName: \"kubernetes.io/projected/e8c333bb-6008-457c-b8a3-66bd654f012d-kube-api-access-gvvl6\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:35:13.227707 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:35:13.227708    4284 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/e8c333bb-6008-457c-b8a3-66bd654f012d-cacerts\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:35:13.227707 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:35:13.227718    4284 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e8c333bb-6008-457c-b8a3-66bd654f012d-istio-token\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:35:13.227986 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:35:13.227727    4284 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/e8c333bb-6008-457c-b8a3-66bd654f012d-local-certs\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:35:13.227986 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:35:13.227735    4284 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e8c333bb-6008-457c-b8a3-66bd654f012d-istio-kubeconfig\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:35:13.227986 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:35:13.227744    4284 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/e8c333bb-6008-457c-b8a3-66bd654f012d-istio-csr-dns-cert\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:35:13.599496 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:35:13.599459    4284 generic.go:356] "Generic (PLEG): container finished" podID="e8c333bb-6008-457c-b8a3-66bd654f012d" containerID="e8e6c2cd8b38f39329b13b03cbb7b4bf9eb1effffd2bcb709b6d426a8b75c55b" exitCode=0
Oct 31 00:35:13.599696 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:35:13.599514    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-876pt"
Oct 31 00:35:13.599696 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:35:13.599541    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-876pt" event={"ID":"e8c333bb-6008-457c-b8a3-66bd654f012d","Type":"ContainerDied","Data":"e8e6c2cd8b38f39329b13b03cbb7b4bf9eb1effffd2bcb709b6d426a8b75c55b"}
Oct 31 00:35:13.599696 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:35:13.599604    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-876pt" event={"ID":"e8c333bb-6008-457c-b8a3-66bd654f012d","Type":"ContainerDied","Data":"deb203a2d3130c6a88d0e9ac6a61ae835a479aa1636792a97150a64cfe3e4f94"}
Oct 31 00:35:13.599696 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:35:13.599624    4284 scope.go:122] "RemoveContainer" containerID="e8e6c2cd8b38f39329b13b03cbb7b4bf9eb1effffd2bcb709b6d426a8b75c55b"
Oct 31 00:35:13.610602 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:35:13.610570    4284 scope.go:122] "RemoveContainer" containerID="e8e6c2cd8b38f39329b13b03cbb7b4bf9eb1effffd2bcb709b6d426a8b75c55b"
Oct 31 00:35:13.610864 ip-10-0-132-173 kubenswrapper[4284]: E1031 00:35:13.610845    4284 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e8e6c2cd8b38f39329b13b03cbb7b4bf9eb1effffd2bcb709b6d426a8b75c55b\": container with ID starting with e8e6c2cd8b38f39329b13b03cbb7b4bf9eb1effffd2bcb709b6d426a8b75c55b not found: ID does not exist" containerID="e8e6c2cd8b38f39329b13b03cbb7b4bf9eb1effffd2bcb709b6d426a8b75c55b"
Oct 31 00:35:13.610916 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:35:13.610873    4284 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e8e6c2cd8b38f39329b13b03cbb7b4bf9eb1effffd2bcb709b6d426a8b75c55b"} err="failed to get container status \"e8e6c2cd8b38f39329b13b03cbb7b4bf9eb1effffd2bcb709b6d426a8b75c55b\": rpc error: code = NotFound desc = could not find container \"e8e6c2cd8b38f39329b13b03cbb7b4bf9eb1effffd2bcb709b6d426a8b75c55b\": container with ID starting with e8e6c2cd8b38f39329b13b03cbb7b4bf9eb1effffd2bcb709b6d426a8b75c55b not found: ID does not exist"
Oct 31 00:35:13.757106 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:35:13.757070    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7876bfcb74-876pt"]
Oct 31 00:35:13.804744 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:35:13.804708    4284 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7876bfcb74-876pt"]
Oct 31 00:35:13.961051 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:35:13.961016    4284 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e8c333bb-6008-457c-b8a3-66bd654f012d" path="/var/lib/kubelet/pods/e8c333bb-6008-457c-b8a3-66bd654f012d/volumes"
Oct 31 00:35:13.980232 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:35:13.980180    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:35:17.013948 ip-10-0-132-173 kubenswrapper[4284]: E1031 00:35:17.013911    4284 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fd8sp" containerName="extractor"
Oct 31 00:35:19.086626 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:35:19.086592    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:35:19.309690 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:35:19.309645    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:35:19.489290 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:35:19.489261    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:35:19.643982 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:35:19.643947    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:35:19.844449 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:35:19.844343    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:35:20.039245 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:35:20.039192    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:35:20.200261 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:35:20.200228    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:35:20.350150 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:35:20.350122    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:35:21.602589 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:35:21.602556    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:35:21.802381 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:35:21.802346    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:35:21.955414 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:35:21.955382    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:35:22.179723 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:35:22.179693    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:35:22.390810 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:35:22.390722    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:35:22.539198 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:35:22.539162    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:35:23.015643 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:35:23.015617    4284 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-66c8cb554b-c6nqg"]
Oct 31 00:35:23.016053 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:35:23.015898    4284 cpu_manager.go:414] "containerMap: removing container" podUID="e8c333bb-6008-457c-b8a3-66bd654f012d" containerName="discovery"
Oct 31 00:35:23.016053 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:35:23.015908    4284 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e8c333bb-6008-457c-b8a3-66bd654f012d" containerName="discovery"
Oct 31 00:35:23.016053 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:35:23.015980    4284 memory_manager.go:365] "RemoveStaleState removing state" podUID="e8c333bb-6008-457c-b8a3-66bd654f012d" containerName="discovery"
Oct 31 00:35:23.022318 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:35:23.022300    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-66c8cb554b-c6nqg"
Oct 31 00:35:23.114147 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:35:23.114115    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/365ce84c-b0d4-4fb6-9951-895a3529cf66-maas-api-tls\") pod \"maas-api-66c8cb554b-c6nqg\" (UID: \"365ce84c-b0d4-4fb6-9951-895a3529cf66\") " pod="odh-ai-gateway-infra/maas-api-66c8cb554b-c6nqg"
Oct 31 00:35:23.114147 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:35:23.114151    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/365ce84c-b0d4-4fb6-9951-895a3529cf66-openshift-service-ca\") pod \"maas-api-66c8cb554b-c6nqg\" (UID: \"365ce84c-b0d4-4fb6-9951-895a3529cf66\") " pod="odh-ai-gateway-infra/maas-api-66c8cb554b-c6nqg"
Oct 31 00:35:23.114413 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:35:23.114179    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kl5b7\" (UniqueName: \"kubernetes.io/projected/365ce84c-b0d4-4fb6-9951-895a3529cf66-kube-api-access-kl5b7\") pod \"maas-api-66c8cb554b-c6nqg\" (UID: \"365ce84c-b0d4-4fb6-9951-895a3529cf66\") " pod="odh-ai-gateway-infra/maas-api-66c8cb554b-c6nqg"
Oct 31 00:35:23.114413 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:35:23.114397    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/365ce84c-b0d4-4fb6-9951-895a3529cf66-metrics-certs\") pod \"maas-api-66c8cb554b-c6nqg\" (UID: \"365ce84c-b0d4-4fb6-9951-895a3529cf66\") " pod="odh-ai-gateway-infra/maas-api-66c8cb554b-c6nqg"
Oct 31 00:35:23.121036 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:35:23.121018    4284 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Oct 31 00:35:23.121136 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:35:23.121068    4284 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-f72zx\""
Oct 31 00:35:23.121136 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:35:23.121083    4284 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Oct 31 00:35:23.122377 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:35:23.122358    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-66c8cb554b-c6nqg"]
Oct 31 00:35:23.215158 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:35:23.215116    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/365ce84c-b0d4-4fb6-9951-895a3529cf66-openshift-service-ca\") pod \"maas-api-66c8cb554b-c6nqg\" (UID: \"365ce84c-b0d4-4fb6-9951-895a3529cf66\") " pod="odh-ai-gateway-infra/maas-api-66c8cb554b-c6nqg"
Oct 31 00:35:23.215362 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:35:23.215170    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kl5b7\" (UniqueName: \"kubernetes.io/projected/365ce84c-b0d4-4fb6-9951-895a3529cf66-kube-api-access-kl5b7\") pod \"maas-api-66c8cb554b-c6nqg\" (UID: \"365ce84c-b0d4-4fb6-9951-895a3529cf66\") " pod="odh-ai-gateway-infra/maas-api-66c8cb554b-c6nqg"
Oct 31 00:35:23.215362 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:35:23.215248    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/365ce84c-b0d4-4fb6-9951-895a3529cf66-metrics-certs\") pod \"maas-api-66c8cb554b-c6nqg\" (UID: \"365ce84c-b0d4-4fb6-9951-895a3529cf66\") " pod="odh-ai-gateway-infra/maas-api-66c8cb554b-c6nqg"
Oct 31 00:35:23.215362 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:35:23.215275    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/365ce84c-b0d4-4fb6-9951-895a3529cf66-maas-api-tls\") pod \"maas-api-66c8cb554b-c6nqg\" (UID: \"365ce84c-b0d4-4fb6-9951-895a3529cf66\") " pod="odh-ai-gateway-infra/maas-api-66c8cb554b-c6nqg"
Oct 31 00:35:23.215971 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:35:23.215932    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/365ce84c-b0d4-4fb6-9951-895a3529cf66-openshift-service-ca\") pod \"maas-api-66c8cb554b-c6nqg\" (UID: \"365ce84c-b0d4-4fb6-9951-895a3529cf66\") " pod="odh-ai-gateway-infra/maas-api-66c8cb554b-c6nqg"
Oct 31 00:35:23.217849 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:35:23.217823    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/365ce84c-b0d4-4fb6-9951-895a3529cf66-maas-api-tls\") pod \"maas-api-66c8cb554b-c6nqg\" (UID: \"365ce84c-b0d4-4fb6-9951-895a3529cf66\") " pod="odh-ai-gateway-infra/maas-api-66c8cb554b-c6nqg"
Oct 31 00:35:23.217993 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:35:23.217971    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/365ce84c-b0d4-4fb6-9951-895a3529cf66-metrics-certs\") pod \"maas-api-66c8cb554b-c6nqg\" (UID: \"365ce84c-b0d4-4fb6-9951-895a3529cf66\") " pod="odh-ai-gateway-infra/maas-api-66c8cb554b-c6nqg"
Oct 31 00:35:23.274125 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:35:23.274042    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kl5b7\" (UniqueName: \"kubernetes.io/projected/365ce84c-b0d4-4fb6-9951-895a3529cf66-kube-api-access-kl5b7\") pod \"maas-api-66c8cb554b-c6nqg\" (UID: \"365ce84c-b0d4-4fb6-9951-895a3529cf66\") " pod="odh-ai-gateway-infra/maas-api-66c8cb554b-c6nqg"
Oct 31 00:35:23.334159 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:35:23.334125    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-66c8cb554b-c6nqg"
Oct 31 00:35:23.700719 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:35:23.700698    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-66c8cb554b-c6nqg"]
Oct 31 00:35:23.702725 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:35:23.702700    4284 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod365ce84c_b0d4_4fb6_9951_895a3529cf66.slice/crio-5bda42c14a34208c18a295b8da9105e847ae9af7f5c4490868622fd7177b8c9b WatchSource:0}: Error finding container 5bda42c14a34208c18a295b8da9105e847ae9af7f5c4490868622fd7177b8c9b: Status 404 returned error can't find the container with id 5bda42c14a34208c18a295b8da9105e847ae9af7f5c4490868622fd7177b8c9b
Oct 31 00:35:24.641794 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:35:24.641767    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-66c8cb554b-c6nqg" event={"ID":"365ce84c-b0d4-4fb6-9951-895a3529cf66","Type":"ContainerStarted","Data":"5bda42c14a34208c18a295b8da9105e847ae9af7f5c4490868622fd7177b8c9b"}
Oct 31 00:35:29.661452 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:35:29.661416    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-66c8cb554b-c6nqg" event={"ID":"365ce84c-b0d4-4fb6-9951-895a3529cf66","Type":"ContainerStarted","Data":"2282dffe7f4da11a20972d9257d6bf2a364429d56503fa8fabc27af565ddc181"}
Oct 31 00:35:29.661878 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:35:29.661547    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-66c8cb554b-c6nqg"
Oct 31 00:35:29.775522 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:35:29.775428    4284 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-66c8cb554b-c6nqg" podStartSLOduration=2.056297037 podStartE2EDuration="7.77541487s" totalImagesPullingTime="5.719117833s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 00:35:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:35:29.773925071 +0000 UTC m=+443.918931986" watchObservedRunningTime="2026-10-31 00:35:29.77541487 +0000 UTC m=+443.920421786"
Oct 31 00:35:35.671533 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:35:35.671506    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-66c8cb554b-c6nqg"
Oct 31 00:35:36.262384 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:35:36.262343    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:35:36.511620 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:35:36.511586    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:35:36.734151 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:35:36.734117    4284 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-759ffb85c5-kdc86"]
Oct 31 00:35:36.737554 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:35:36.737534    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-759ffb85c5-kdc86"
Oct 31 00:35:36.825274 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:35:36.825243    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d3cb3add-09fd-4bea-9441-d720542fbda1-maas-api-tls\") pod \"maas-api-759ffb85c5-kdc86\" (UID: \"d3cb3add-09fd-4bea-9441-d720542fbda1\") " pod="odh-ai-gateway-infra/maas-api-759ffb85c5-kdc86"
Oct 31 00:35:36.825456 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:35:36.825289    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j9zlq\" (UniqueName: \"kubernetes.io/projected/d3cb3add-09fd-4bea-9441-d720542fbda1-kube-api-access-j9zlq\") pod \"maas-api-759ffb85c5-kdc86\" (UID: \"d3cb3add-09fd-4bea-9441-d720542fbda1\") " pod="odh-ai-gateway-infra/maas-api-759ffb85c5-kdc86"
Oct 31 00:35:36.825456 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:35:36.825328    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d3cb3add-09fd-4bea-9441-d720542fbda1-openshift-service-ca\") pod \"maas-api-759ffb85c5-kdc86\" (UID: \"d3cb3add-09fd-4bea-9441-d720542fbda1\") " pod="odh-ai-gateway-infra/maas-api-759ffb85c5-kdc86"
Oct 31 00:35:36.825548 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:35:36.825469    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d3cb3add-09fd-4bea-9441-d720542fbda1-metrics-certs\") pod \"maas-api-759ffb85c5-kdc86\" (UID: \"d3cb3add-09fd-4bea-9441-d720542fbda1\") " pod="odh-ai-gateway-infra/maas-api-759ffb85c5-kdc86"
Oct 31 00:35:36.836334 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:35:36.836312    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-759ffb85c5-kdc86"]
Oct 31 00:35:36.926352 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:35:36.926320    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j9zlq\" (UniqueName: \"kubernetes.io/projected/d3cb3add-09fd-4bea-9441-d720542fbda1-kube-api-access-j9zlq\") pod \"maas-api-759ffb85c5-kdc86\" (UID: \"d3cb3add-09fd-4bea-9441-d720542fbda1\") " pod="odh-ai-gateway-infra/maas-api-759ffb85c5-kdc86"
Oct 31 00:35:36.926547 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:35:36.926371    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d3cb3add-09fd-4bea-9441-d720542fbda1-openshift-service-ca\") pod \"maas-api-759ffb85c5-kdc86\" (UID: \"d3cb3add-09fd-4bea-9441-d720542fbda1\") " pod="odh-ai-gateway-infra/maas-api-759ffb85c5-kdc86"
Oct 31 00:35:36.926547 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:35:36.926397    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d3cb3add-09fd-4bea-9441-d720542fbda1-metrics-certs\") pod \"maas-api-759ffb85c5-kdc86\" (UID: \"d3cb3add-09fd-4bea-9441-d720542fbda1\") " pod="odh-ai-gateway-infra/maas-api-759ffb85c5-kdc86"
Oct 31 00:35:36.926547 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:35:36.926425    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d3cb3add-09fd-4bea-9441-d720542fbda1-maas-api-tls\") pod \"maas-api-759ffb85c5-kdc86\" (UID: \"d3cb3add-09fd-4bea-9441-d720542fbda1\") " pod="odh-ai-gateway-infra/maas-api-759ffb85c5-kdc86"
Oct 31 00:35:36.927066 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:35:36.927044    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d3cb3add-09fd-4bea-9441-d720542fbda1-openshift-service-ca\") pod \"maas-api-759ffb85c5-kdc86\" (UID: \"d3cb3add-09fd-4bea-9441-d720542fbda1\") " pod="odh-ai-gateway-infra/maas-api-759ffb85c5-kdc86"
Oct 31 00:35:36.929095 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:35:36.929073    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d3cb3add-09fd-4bea-9441-d720542fbda1-metrics-certs\") pod \"maas-api-759ffb85c5-kdc86\" (UID: \"d3cb3add-09fd-4bea-9441-d720542fbda1\") " pod="odh-ai-gateway-infra/maas-api-759ffb85c5-kdc86"
Oct 31 00:35:36.929178 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:35:36.929123    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d3cb3add-09fd-4bea-9441-d720542fbda1-maas-api-tls\") pod \"maas-api-759ffb85c5-kdc86\" (UID: \"d3cb3add-09fd-4bea-9441-d720542fbda1\") " pod="odh-ai-gateway-infra/maas-api-759ffb85c5-kdc86"
Oct 31 00:35:36.985656 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:35:36.985575    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j9zlq\" (UniqueName: \"kubernetes.io/projected/d3cb3add-09fd-4bea-9441-d720542fbda1-kube-api-access-j9zlq\") pod \"maas-api-759ffb85c5-kdc86\" (UID: \"d3cb3add-09fd-4bea-9441-d720542fbda1\") " pod="odh-ai-gateway-infra/maas-api-759ffb85c5-kdc86"
Oct 31 00:35:37.048562 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:35:37.048528    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-759ffb85c5-kdc86"
Oct 31 00:35:37.415743 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:35:37.415721    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-759ffb85c5-kdc86"]
Oct 31 00:35:37.417451 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:35:37.417423    4284 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd3cb3add_09fd_4bea_9441_d720542fbda1.slice/crio-5cedfd6fd5904351d060f6a1dd079c967e4535831d0b576f9c418774ab8feb39 WatchSource:0}: Error finding container 5cedfd6fd5904351d060f6a1dd079c967e4535831d0b576f9c418774ab8feb39: Status 404 returned error can't find the container with id 5cedfd6fd5904351d060f6a1dd079c967e4535831d0b576f9c418774ab8feb39
Oct 31 00:35:37.694309 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:35:37.694190    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-759ffb85c5-kdc86" event={"ID":"d3cb3add-09fd-4bea-9441-d720542fbda1","Type":"ContainerStarted","Data":"5cedfd6fd5904351d060f6a1dd079c967e4535831d0b576f9c418774ab8feb39"}
Oct 31 00:35:38.707417 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:35:38.707367    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-759ffb85c5-kdc86" event={"ID":"d3cb3add-09fd-4bea-9441-d720542fbda1","Type":"ContainerStarted","Data":"53f78e36f88a83c36ac168d0e3c90a99ef4ef75c1497e53ff22c86ef00d8100f"}
Oct 31 00:35:38.707823 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:35:38.707510    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-759ffb85c5-kdc86"
Oct 31 00:35:38.821513 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:35:38.821460    4284 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-759ffb85c5-kdc86" podStartSLOduration=1.930057829 podStartE2EDuration="2.821446455s" totalImagesPullingTime="891.388626ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 00:35:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:35:38.821162868 +0000 UTC m=+452.966169784" watchObservedRunningTime="2026-10-31 00:35:38.821446455 +0000 UTC m=+452.966453372"
Oct 31 00:35:44.717123 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:35:44.717095    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-759ffb85c5-kdc86"
Oct 31 00:35:44.879694 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:35:44.879655    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-66c8cb554b-c6nqg"]
Oct 31 00:35:44.879964 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:35:44.879916    4284 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-66c8cb554b-c6nqg" podUID="365ce84c-b0d4-4fb6-9951-895a3529cf66" containerName="maas-api" containerID="cri-o://2282dffe7f4da11a20972d9257d6bf2a364429d56503fa8fabc27af565ddc181" gracePeriod=30
Oct 31 00:35:45.121337 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:35:45.121305    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-66c8cb554b-c6nqg"
Oct 31 00:35:45.201089 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:35:45.201055    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/365ce84c-b0d4-4fb6-9951-895a3529cf66-kube-api-access-kl5b7\" (UniqueName: \"kubernetes.io/projected/365ce84c-b0d4-4fb6-9951-895a3529cf66-kube-api-access-kl5b7\") pod \"365ce84c-b0d4-4fb6-9951-895a3529cf66\" (UID: \"365ce84c-b0d4-4fb6-9951-895a3529cf66\") "
Oct 31 00:35:45.201311 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:35:45.201154    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/365ce84c-b0d4-4fb6-9951-895a3529cf66-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/365ce84c-b0d4-4fb6-9951-895a3529cf66-maas-api-tls\") pod \"365ce84c-b0d4-4fb6-9951-895a3529cf66\" (UID: \"365ce84c-b0d4-4fb6-9951-895a3529cf66\") "
Oct 31 00:35:45.201311 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:35:45.201185    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/365ce84c-b0d4-4fb6-9951-895a3529cf66-metrics-certs\" (UniqueName: \"kubernetes.io/secret/365ce84c-b0d4-4fb6-9951-895a3529cf66-metrics-certs\") pod \"365ce84c-b0d4-4fb6-9951-895a3529cf66\" (UID: \"365ce84c-b0d4-4fb6-9951-895a3529cf66\") "
Oct 31 00:35:45.201311 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:35:45.201248    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/365ce84c-b0d4-4fb6-9951-895a3529cf66-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/365ce84c-b0d4-4fb6-9951-895a3529cf66-openshift-service-ca\") pod \"365ce84c-b0d4-4fb6-9951-895a3529cf66\" (UID: \"365ce84c-b0d4-4fb6-9951-895a3529cf66\") "
Oct 31 00:35:45.201674 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:35:45.201645    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/365ce84c-b0d4-4fb6-9951-895a3529cf66-openshift-service-ca" pod "365ce84c-b0d4-4fb6-9951-895a3529cf66" (UID: "365ce84c-b0d4-4fb6-9951-895a3529cf66"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 00:35:45.203536 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:35:45.203512    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/365ce84c-b0d4-4fb6-9951-895a3529cf66-metrics-certs" pod "365ce84c-b0d4-4fb6-9951-895a3529cf66" (UID: "365ce84c-b0d4-4fb6-9951-895a3529cf66"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 00:35:45.203640 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:35:45.203562    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/365ce84c-b0d4-4fb6-9951-895a3529cf66-kube-api-access-kl5b7" pod "365ce84c-b0d4-4fb6-9951-895a3529cf66" (UID: "365ce84c-b0d4-4fb6-9951-895a3529cf66"). InnerVolumeSpecName "kube-api-access-kl5b7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 00:35:45.203640 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:35:45.203562    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/365ce84c-b0d4-4fb6-9951-895a3529cf66-maas-api-tls" pod "365ce84c-b0d4-4fb6-9951-895a3529cf66" (UID: "365ce84c-b0d4-4fb6-9951-895a3529cf66"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 00:35:45.302364 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:35:45.302278    4284 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kl5b7\" (UniqueName: \"kubernetes.io/projected/365ce84c-b0d4-4fb6-9951-895a3529cf66-kube-api-access-kl5b7\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:35:45.302364 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:35:45.302317    4284 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/365ce84c-b0d4-4fb6-9951-895a3529cf66-maas-api-tls\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:35:45.302364 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:35:45.302357    4284 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/365ce84c-b0d4-4fb6-9951-895a3529cf66-metrics-certs\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:35:45.302579 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:35:45.302374    4284 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/365ce84c-b0d4-4fb6-9951-895a3529cf66-openshift-service-ca\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:35:45.734749 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:35:45.734720    4284 generic.go:356] "Generic (PLEG): container finished" podID="365ce84c-b0d4-4fb6-9951-895a3529cf66" containerID="2282dffe7f4da11a20972d9257d6bf2a364429d56503fa8fabc27af565ddc181" exitCode=0
Oct 31 00:35:45.735154 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:35:45.734766    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-66c8cb554b-c6nqg"
Oct 31 00:35:45.735154 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:35:45.734769    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-66c8cb554b-c6nqg" event={"ID":"365ce84c-b0d4-4fb6-9951-895a3529cf66","Type":"ContainerDied","Data":"2282dffe7f4da11a20972d9257d6bf2a364429d56503fa8fabc27af565ddc181"}
Oct 31 00:35:45.735154 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:35:45.734895    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-66c8cb554b-c6nqg" event={"ID":"365ce84c-b0d4-4fb6-9951-895a3529cf66","Type":"ContainerDied","Data":"5bda42c14a34208c18a295b8da9105e847ae9af7f5c4490868622fd7177b8c9b"}
Oct 31 00:35:45.735154 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:35:45.734924    4284 scope.go:122] "RemoveContainer" containerID="2282dffe7f4da11a20972d9257d6bf2a364429d56503fa8fabc27af565ddc181"
Oct 31 00:35:45.743672 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:35:45.743653    4284 scope.go:122] "RemoveContainer" containerID="2282dffe7f4da11a20972d9257d6bf2a364429d56503fa8fabc27af565ddc181"
Oct 31 00:35:45.743944 ip-10-0-132-173 kubenswrapper[4284]: E1031 00:35:45.743927    4284 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2282dffe7f4da11a20972d9257d6bf2a364429d56503fa8fabc27af565ddc181\": container with ID starting with 2282dffe7f4da11a20972d9257d6bf2a364429d56503fa8fabc27af565ddc181 not found: ID does not exist" containerID="2282dffe7f4da11a20972d9257d6bf2a364429d56503fa8fabc27af565ddc181"
Oct 31 00:35:45.744009 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:35:45.743957    4284 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2282dffe7f4da11a20972d9257d6bf2a364429d56503fa8fabc27af565ddc181"} err="failed to get container status \"2282dffe7f4da11a20972d9257d6bf2a364429d56503fa8fabc27af565ddc181\": rpc error: code = NotFound desc = could not find container \"2282dffe7f4da11a20972d9257d6bf2a364429d56503fa8fabc27af565ddc181\": container with ID starting with 2282dffe7f4da11a20972d9257d6bf2a364429d56503fa8fabc27af565ddc181 not found: ID does not exist"
Oct 31 00:35:45.891144 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:35:45.891111    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-66c8cb554b-c6nqg"]
Oct 31 00:35:45.894811 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:35:45.894791    4284 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-66c8cb554b-c6nqg"]
Oct 31 00:35:45.962124 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:35:45.962085    4284 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="365ce84c-b0d4-4fb6-9951-895a3529cf66" path="/var/lib/kubelet/pods/365ce84c-b0d4-4fb6-9951-895a3529cf66/volumes"
Oct 31 00:35:47.013358 ip-10-0-132-173 kubenswrapper[4284]: E1031 00:35:47.013320    4284 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fd8sp" containerName="extractor"
Oct 31 00:35:51.283777 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:35:51.283745    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:36:17.013011 ip-10-0-132-173 kubenswrapper[4284]: E1031 00:36:17.012935    4284 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fd8sp" containerName="extractor"
Oct 31 00:36:46.914638 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:36:46.914598    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:36:47.014434 ip-10-0-132-173 kubenswrapper[4284]: E1031 00:36:47.014397    4284 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fd8sp" containerName="extractor"
Oct 31 00:36:51.643488 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:36:51.643441    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:36:57.208199 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:36:57.208161    4284 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/ai-gateway-controller-7d85675f98-lh7sz"]
Oct 31 00:36:57.208697 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:36:57.208470    4284 cpu_manager.go:414] "containerMap: removing container" podUID="365ce84c-b0d4-4fb6-9951-895a3529cf66" containerName="maas-api"
Oct 31 00:36:57.208697 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:36:57.208480    4284 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="365ce84c-b0d4-4fb6-9951-895a3529cf66" containerName="maas-api"
Oct 31 00:36:57.208697 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:36:57.208540    4284 memory_manager.go:365] "RemoveStaleState removing state" podUID="365ce84c-b0d4-4fb6-9951-895a3529cf66" containerName="maas-api"
Oct 31 00:36:57.211476 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:36:57.211456    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/ai-gateway-controller-7d85675f98-lh7sz"
Oct 31 00:36:57.240787 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:36:57.240751    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:36:57.247705 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:36:57.247671    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e84b35f2-bb19-4256-b1a0-241ce42ed7df-metrics-certs\") pod \"ai-gateway-controller-7d85675f98-lh7sz\" (UID: \"e84b35f2-bb19-4256-b1a0-241ce42ed7df\") " pod="opendatahub/ai-gateway-controller-7d85675f98-lh7sz"
Oct 31 00:36:57.247705 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:36:57.247707    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w9zx9\" (UniqueName: \"kubernetes.io/projected/e84b35f2-bb19-4256-b1a0-241ce42ed7df-kube-api-access-w9zx9\") pod \"ai-gateway-controller-7d85675f98-lh7sz\" (UID: \"e84b35f2-bb19-4256-b1a0-241ce42ed7df\") " pod="opendatahub/ai-gateway-controller-7d85675f98-lh7sz"
Oct 31 00:36:57.261428 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:36:57.261388    4284 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"ai-gateway-controller-metrics-tls\""
Oct 31 00:36:57.308972 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:36:57.308923    4284 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"ai-gateway-controller-parameters\""
Oct 31 00:36:57.308972 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:36:57.308932    4284 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"ai-gateway-controller-dockercfg-nnc68\""
Oct 31 00:36:57.310362 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:36:57.310335    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/ai-gateway-controller-7d85675f98-lh7sz"]
Oct 31 00:36:57.348237 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:36:57.348178    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e84b35f2-bb19-4256-b1a0-241ce42ed7df-metrics-certs\") pod \"ai-gateway-controller-7d85675f98-lh7sz\" (UID: \"e84b35f2-bb19-4256-b1a0-241ce42ed7df\") " pod="opendatahub/ai-gateway-controller-7d85675f98-lh7sz"
Oct 31 00:36:57.348413 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:36:57.348269    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w9zx9\" (UniqueName: \"kubernetes.io/projected/e84b35f2-bb19-4256-b1a0-241ce42ed7df-kube-api-access-w9zx9\") pod \"ai-gateway-controller-7d85675f98-lh7sz\" (UID: \"e84b35f2-bb19-4256-b1a0-241ce42ed7df\") " pod="opendatahub/ai-gateway-controller-7d85675f98-lh7sz"
Oct 31 00:36:57.350799 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:36:57.350774    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e84b35f2-bb19-4256-b1a0-241ce42ed7df-metrics-certs\") pod \"ai-gateway-controller-7d85675f98-lh7sz\" (UID: \"e84b35f2-bb19-4256-b1a0-241ce42ed7df\") " pod="opendatahub/ai-gateway-controller-7d85675f98-lh7sz"
Oct 31 00:36:57.407383 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:36:57.407346    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w9zx9\" (UniqueName: \"kubernetes.io/projected/e84b35f2-bb19-4256-b1a0-241ce42ed7df-kube-api-access-w9zx9\") pod \"ai-gateway-controller-7d85675f98-lh7sz\" (UID: \"e84b35f2-bb19-4256-b1a0-241ce42ed7df\") " pod="opendatahub/ai-gateway-controller-7d85675f98-lh7sz"
Oct 31 00:36:57.408917 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:36:57.408890    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:36:57.524202 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:36:57.524118    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/ai-gateway-controller-7d85675f98-lh7sz"
Oct 31 00:36:57.887462 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:36:57.887429    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/ai-gateway-controller-7d85675f98-lh7sz"]
Oct 31 00:36:57.889809 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:36:57.889780    4284 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode84b35f2_bb19_4256_b1a0_241ce42ed7df.slice/crio-da0ea772cba086058eff904fb2334d53398d054b024aee1c4997f87410fa101f WatchSource:0}: Error finding container da0ea772cba086058eff904fb2334d53398d054b024aee1c4997f87410fa101f: Status 404 returned error can't find the container with id da0ea772cba086058eff904fb2334d53398d054b024aee1c4997f87410fa101f
Oct 31 00:36:58.014697 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:36:58.014656    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/ai-gateway-controller-7d85675f98-lh7sz" event={"ID":"e84b35f2-bb19-4256-b1a0-241ce42ed7df","Type":"ContainerStarted","Data":"da0ea772cba086058eff904fb2334d53398d054b024aee1c4997f87410fa101f"}
Oct 31 00:37:01.030061 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:37:01.030025    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/ai-gateway-controller-7d85675f98-lh7sz" event={"ID":"e84b35f2-bb19-4256-b1a0-241ce42ed7df","Type":"ContainerStarted","Data":"9ec52d60b4b891bfbb888f8f917a128a45ed0f9cb7f8b615c1ef8d91dae9ef5f"}
Oct 31 00:37:01.030481 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:37:01.030165    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/ai-gateway-controller-7d85675f98-lh7sz"
Oct 31 00:37:01.141871 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:37:01.141814    4284 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/ai-gateway-controller-7d85675f98-lh7sz" podStartSLOduration=1.6360472640000001 podStartE2EDuration="4.141799921s" totalImagesPullingTime="2.505752657s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 00:36:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:37:01.141132074 +0000 UTC m=+535.286138989" watchObservedRunningTime="2026-10-31 00:37:01.141799921 +0000 UTC m=+535.286806836"
Oct 31 00:37:04.234231 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:37:04.234179    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:37:04.433406 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:37:04.433368    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:37:04.641109 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:37:04.641027    4284 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-7b98bb678c-9xcfc"]
Oct 31 00:37:04.644582 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:37:04.644559    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7b98bb678c-9xcfc"
Oct 31 00:37:04.716004 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:37:04.715970    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1d75c55d-1352-44c8-8199-cbd0381ac2cf-plugins-config-volume\") pod \"payload-pre-processing-7b98bb678c-9xcfc\" (UID: \"1d75c55d-1352-44c8-8199-cbd0381ac2cf\") " pod="openshift-ingress/payload-pre-processing-7b98bb678c-9xcfc"
Oct 31 00:37:04.716178 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:37:04.716009    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gvlxl\" (UniqueName: \"kubernetes.io/projected/1d75c55d-1352-44c8-8199-cbd0381ac2cf-kube-api-access-gvlxl\") pod \"payload-pre-processing-7b98bb678c-9xcfc\" (UID: \"1d75c55d-1352-44c8-8199-cbd0381ac2cf\") " pod="openshift-ingress/payload-pre-processing-7b98bb678c-9xcfc"
Oct 31 00:37:04.744782 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:37:04.744746    4284 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Oct 31 00:37:04.744940 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:37:04.744786    4284 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-5jxcj\""
Oct 31 00:37:04.746182 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:37:04.746158    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7b98bb678c-9xcfc"]
Oct 31 00:37:04.783998 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:37:04.783967    4284 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-55fbbbf9dc-j9vgx"]
Oct 31 00:37:04.787402 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:37:04.787381    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-55fbbbf9dc-j9vgx"
Oct 31 00:37:04.816901 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:37:04.816861    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1d75c55d-1352-44c8-8199-cbd0381ac2cf-plugins-config-volume\") pod \"payload-pre-processing-7b98bb678c-9xcfc\" (UID: \"1d75c55d-1352-44c8-8199-cbd0381ac2cf\") " pod="openshift-ingress/payload-pre-processing-7b98bb678c-9xcfc"
Oct 31 00:37:04.816901 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:37:04.816900    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gvlxl\" (UniqueName: \"kubernetes.io/projected/1d75c55d-1352-44c8-8199-cbd0381ac2cf-kube-api-access-gvlxl\") pod \"payload-pre-processing-7b98bb678c-9xcfc\" (UID: \"1d75c55d-1352-44c8-8199-cbd0381ac2cf\") " pod="openshift-ingress/payload-pre-processing-7b98bb678c-9xcfc"
Oct 31 00:37:04.817627 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:37:04.817606    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1d75c55d-1352-44c8-8199-cbd0381ac2cf-plugins-config-volume\") pod \"payload-pre-processing-7b98bb678c-9xcfc\" (UID: \"1d75c55d-1352-44c8-8199-cbd0381ac2cf\") " pod="openshift-ingress/payload-pre-processing-7b98bb678c-9xcfc"
Oct 31 00:37:04.842149 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:37:04.842110    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-55fbbbf9dc-j9vgx"]
Oct 31 00:37:04.876330 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:37:04.876293    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gvlxl\" (UniqueName: \"kubernetes.io/projected/1d75c55d-1352-44c8-8199-cbd0381ac2cf-kube-api-access-gvlxl\") pod \"payload-pre-processing-7b98bb678c-9xcfc\" (UID: \"1d75c55d-1352-44c8-8199-cbd0381ac2cf\") " pod="openshift-ingress/payload-pre-processing-7b98bb678c-9xcfc"
Oct 31 00:37:04.917534 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:37:04.917498    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5fa890d5-a2f9-495a-972b-06b8db658c88-plugins-config-volume\") pod \"payload-processing-55fbbbf9dc-j9vgx\" (UID: \"5fa890d5-a2f9-495a-972b-06b8db658c88\") " pod="openshift-ingress/payload-processing-55fbbbf9dc-j9vgx"
Oct 31 00:37:04.917719 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:37:04.917627    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n6jsl\" (UniqueName: \"kubernetes.io/projected/5fa890d5-a2f9-495a-972b-06b8db658c88-kube-api-access-n6jsl\") pod \"payload-processing-55fbbbf9dc-j9vgx\" (UID: \"5fa890d5-a2f9-495a-972b-06b8db658c88\") " pod="openshift-ingress/payload-processing-55fbbbf9dc-j9vgx"
Oct 31 00:37:04.956460 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:37:04.956429    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7b98bb678c-9xcfc"
Oct 31 00:37:05.019070 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:37:05.019028    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n6jsl\" (UniqueName: \"kubernetes.io/projected/5fa890d5-a2f9-495a-972b-06b8db658c88-kube-api-access-n6jsl\") pod \"payload-processing-55fbbbf9dc-j9vgx\" (UID: \"5fa890d5-a2f9-495a-972b-06b8db658c88\") " pod="openshift-ingress/payload-processing-55fbbbf9dc-j9vgx"
Oct 31 00:37:05.019277 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:37:05.019106    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5fa890d5-a2f9-495a-972b-06b8db658c88-plugins-config-volume\") pod \"payload-processing-55fbbbf9dc-j9vgx\" (UID: \"5fa890d5-a2f9-495a-972b-06b8db658c88\") " pod="openshift-ingress/payload-processing-55fbbbf9dc-j9vgx"
Oct 31 00:37:05.019994 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:37:05.019946    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5fa890d5-a2f9-495a-972b-06b8db658c88-plugins-config-volume\") pod \"payload-processing-55fbbbf9dc-j9vgx\" (UID: \"5fa890d5-a2f9-495a-972b-06b8db658c88\") " pod="openshift-ingress/payload-processing-55fbbbf9dc-j9vgx"
Oct 31 00:37:05.078605 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:37:05.078567    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n6jsl\" (UniqueName: \"kubernetes.io/projected/5fa890d5-a2f9-495a-972b-06b8db658c88-kube-api-access-n6jsl\") pod \"payload-processing-55fbbbf9dc-j9vgx\" (UID: \"5fa890d5-a2f9-495a-972b-06b8db658c88\") " pod="openshift-ingress/payload-processing-55fbbbf9dc-j9vgx"
Oct 31 00:37:05.101227 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:37:05.101185    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-55fbbbf9dc-j9vgx"
Oct 31 00:37:05.322122 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:37:05.322099    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7b98bb678c-9xcfc"]
Oct 31 00:37:05.324142 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:37:05.324116    4284 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1d75c55d_1352_44c8_8199_cbd0381ac2cf.slice/crio-3d4b8b91f3918a8eea75dca3144691d15a61c01ef3479fcbc2bec3368c60c5f8 WatchSource:0}: Error finding container 3d4b8b91f3918a8eea75dca3144691d15a61c01ef3479fcbc2bec3368c60c5f8: Status 404 returned error can't find the container with id 3d4b8b91f3918a8eea75dca3144691d15a61c01ef3479fcbc2bec3368c60c5f8
Oct 31 00:37:05.467363 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:37:05.467337    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-55fbbbf9dc-j9vgx"]
Oct 31 00:37:05.468944 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:37:05.468915    4284 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5fa890d5_a2f9_495a_972b_06b8db658c88.slice/crio-d38b450ef74582e181ae67368d7a39f5fe6950ca098ed0abcf1315bbc27fe3d9 WatchSource:0}: Error finding container d38b450ef74582e181ae67368d7a39f5fe6950ca098ed0abcf1315bbc27fe3d9: Status 404 returned error can't find the container with id d38b450ef74582e181ae67368d7a39f5fe6950ca098ed0abcf1315bbc27fe3d9
Oct 31 00:37:06.050341 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:37:06.050303    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7b98bb678c-9xcfc" event={"ID":"1d75c55d-1352-44c8-8199-cbd0381ac2cf","Type":"ContainerStarted","Data":"3d4b8b91f3918a8eea75dca3144691d15a61c01ef3479fcbc2bec3368c60c5f8"}
Oct 31 00:37:06.051351 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:37:06.051332    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-55fbbbf9dc-j9vgx" event={"ID":"5fa890d5-a2f9-495a-972b-06b8db658c88","Type":"ContainerStarted","Data":"d38b450ef74582e181ae67368d7a39f5fe6950ca098ed0abcf1315bbc27fe3d9"}
Oct 31 00:37:09.069098 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:37:09.068995    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7b98bb678c-9xcfc" event={"ID":"1d75c55d-1352-44c8-8199-cbd0381ac2cf","Type":"ContainerStarted","Data":"eecfb858099e7a1d2fe5681d4144d4134bc57aa29514dcac26e4ff91f08db2dc"}
Oct 31 00:37:09.069586 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:37:09.069173    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-7b98bb678c-9xcfc"
Oct 31 00:37:09.070662 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:37:09.070640    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-55fbbbf9dc-j9vgx" event={"ID":"5fa890d5-a2f9-495a-972b-06b8db658c88","Type":"ContainerStarted","Data":"ab2a502d07c72fc38b666c1aa9b3c5cddc8f5708da6f8bcd7789b34aeb163c9e"}
Oct 31 00:37:09.070748 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:37:09.070682    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-55fbbbf9dc-j9vgx"
Oct 31 00:37:09.180374 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:37:09.180325    4284 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-7b98bb678c-9xcfc" podStartSLOduration=2.001864558 podStartE2EDuration="5.180312277s" totalImagesPullingTime="3.178447719s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 00:37:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:37:09.179718733 +0000 UTC m=+543.324725649" watchObservedRunningTime="2026-10-31 00:37:09.180312277 +0000 UTC m=+543.325319192"
Oct 31 00:37:12.036901 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:37:12.036875    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/ai-gateway-controller-7d85675f98-lh7sz"
Oct 31 00:37:12.149877 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:37:12.149831    4284 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-55fbbbf9dc-j9vgx" podStartSLOduration=4.8882168870000005 podStartE2EDuration="8.149817259s" totalImagesPullingTime="3.261600372s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 00:37:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:37:09.29161601 +0000 UTC m=+543.436622925" watchObservedRunningTime="2026-10-31 00:37:12.149817259 +0000 UTC m=+546.294824185"
Oct 31 00:37:12.379632 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:37:12.379554    4284 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/ai-gateway-controller-d6d549b95-zmcsl"]
Oct 31 00:37:12.383026 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:37:12.382998    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/ai-gateway-controller-d6d549b95-zmcsl"
Oct 31 00:37:12.437777 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:37:12.437742    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/ai-gateway-controller-d6d549b95-zmcsl"]
Oct 31 00:37:12.477597 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:37:12.477564    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3b3b88b3-5e11-44e1-9e0c-084e86e20d96-metrics-certs\") pod \"ai-gateway-controller-d6d549b95-zmcsl\" (UID: \"3b3b88b3-5e11-44e1-9e0c-084e86e20d96\") " pod="opendatahub/ai-gateway-controller-d6d549b95-zmcsl"
Oct 31 00:37:12.477784 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:37:12.477603    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bkgqn\" (UniqueName: \"kubernetes.io/projected/3b3b88b3-5e11-44e1-9e0c-084e86e20d96-kube-api-access-bkgqn\") pod \"ai-gateway-controller-d6d549b95-zmcsl\" (UID: \"3b3b88b3-5e11-44e1-9e0c-084e86e20d96\") " pod="opendatahub/ai-gateway-controller-d6d549b95-zmcsl"
Oct 31 00:37:12.578933 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:37:12.578896    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3b3b88b3-5e11-44e1-9e0c-084e86e20d96-metrics-certs\") pod \"ai-gateway-controller-d6d549b95-zmcsl\" (UID: \"3b3b88b3-5e11-44e1-9e0c-084e86e20d96\") " pod="opendatahub/ai-gateway-controller-d6d549b95-zmcsl"
Oct 31 00:37:12.579124 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:37:12.578946    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bkgqn\" (UniqueName: \"kubernetes.io/projected/3b3b88b3-5e11-44e1-9e0c-084e86e20d96-kube-api-access-bkgqn\") pod \"ai-gateway-controller-d6d549b95-zmcsl\" (UID: \"3b3b88b3-5e11-44e1-9e0c-084e86e20d96\") " pod="opendatahub/ai-gateway-controller-d6d549b95-zmcsl"
Oct 31 00:37:12.581646 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:37:12.581622    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3b3b88b3-5e11-44e1-9e0c-084e86e20d96-metrics-certs\") pod \"ai-gateway-controller-d6d549b95-zmcsl\" (UID: \"3b3b88b3-5e11-44e1-9e0c-084e86e20d96\") " pod="opendatahub/ai-gateway-controller-d6d549b95-zmcsl"
Oct 31 00:37:12.637577 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:37:12.637502    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bkgqn\" (UniqueName: \"kubernetes.io/projected/3b3b88b3-5e11-44e1-9e0c-084e86e20d96-kube-api-access-bkgqn\") pod \"ai-gateway-controller-d6d549b95-zmcsl\" (UID: \"3b3b88b3-5e11-44e1-9e0c-084e86e20d96\") " pod="opendatahub/ai-gateway-controller-d6d549b95-zmcsl"
Oct 31 00:37:12.694646 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:37:12.694603    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/ai-gateway-controller-d6d549b95-zmcsl"
Oct 31 00:37:13.058019 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:37:13.057983    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/ai-gateway-controller-d6d549b95-zmcsl"]
Oct 31 00:37:13.059625 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:37:13.059597    4284 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3b3b88b3_5e11_44e1_9e0c_084e86e20d96.slice/crio-dd4b0201a34bf90c1b45c08736f126c2ee4628ac4899491bc9f48cf9d33f48fe WatchSource:0}: Error finding container dd4b0201a34bf90c1b45c08736f126c2ee4628ac4899491bc9f48cf9d33f48fe: Status 404 returned error can't find the container with id dd4b0201a34bf90c1b45c08736f126c2ee4628ac4899491bc9f48cf9d33f48fe
Oct 31 00:37:13.087369 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:37:13.087340    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/ai-gateway-controller-d6d549b95-zmcsl" event={"ID":"3b3b88b3-5e11-44e1-9e0c-084e86e20d96","Type":"ContainerStarted","Data":"dd4b0201a34bf90c1b45c08736f126c2ee4628ac4899491bc9f48cf9d33f48fe"}
Oct 31 00:37:15.097340 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:37:15.097308    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/ai-gateway-controller-d6d549b95-zmcsl" event={"ID":"3b3b88b3-5e11-44e1-9e0c-084e86e20d96","Type":"ContainerStarted","Data":"3f00cfc7f14174fabf61f4026e3e7582acca1054bdfe3ea831a2b15f7b103644"}
Oct 31 00:37:15.097775 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:37:15.097407    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/ai-gateway-controller-d6d549b95-zmcsl"
Oct 31 00:37:15.209691 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:37:15.209635    4284 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/ai-gateway-controller-d6d549b95-zmcsl" podStartSLOduration=2.219894377 podStartE2EDuration="3.209609854s" totalImagesPullingTime="989.715477ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 00:37:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:37:15.208568942 +0000 UTC m=+549.353575862" watchObservedRunningTime="2026-10-31 00:37:15.209609854 +0000 UTC m=+549.354616770"
Oct 31 00:37:17.013524 ip-10-0-132-173 kubenswrapper[4284]: E1031 00:37:17.013488    4284 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fd8sp" containerName="extractor"
Oct 31 00:37:17.071702 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:37:17.071648    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:37:17.489884 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:37:17.489855    4284 prober.go:122] "Probe failed" probeType="Readiness" pod="opendatahub/maas-controller-6685f84c75-8wmg2" podUID="8b5a139b-f0ff-4d24-b17d-cd6698ff3de5" containerName="manager" probeResult="failure" output="Get \"http://10.134.0.26:8081/readyz\": dial tcp 10.134.0.26:8081: connect: connection refused"
Oct 31 00:37:18.111713 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:37:18.111676    4284 generic.go:356] "Generic (PLEG): container finished" podID="8b5a139b-f0ff-4d24-b17d-cd6698ff3de5" containerID="07caa3cb94501e06a4b3b2f4ff0a4b1eed0e40b5e9e0059844f61758137722d8" exitCode=1
Oct 31 00:37:18.112115 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:37:18.111747    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6685f84c75-8wmg2" event={"ID":"8b5a139b-f0ff-4d24-b17d-cd6698ff3de5","Type":"ContainerDied","Data":"07caa3cb94501e06a4b3b2f4ff0a4b1eed0e40b5e9e0059844f61758137722d8"}
Oct 31 00:37:18.112115 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:37:18.112084    4284 scope.go:122] "RemoveContainer" containerID="07caa3cb94501e06a4b3b2f4ff0a4b1eed0e40b5e9e0059844f61758137722d8"
Oct 31 00:37:20.078104 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:37:20.078074    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-7b98bb678c-9xcfc"
Oct 31 00:37:20.078674 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:37:20.078151    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-55fbbbf9dc-j9vgx"
Oct 31 00:37:20.120967 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:37:20.120935    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6685f84c75-8wmg2" event={"ID":"8b5a139b-f0ff-4d24-b17d-cd6698ff3de5","Type":"ContainerStarted","Data":"e419e5f966d64471e2dde43d25ebfcc503e02277c95ef211d9814e054d9b2fbc"}
Oct 31 00:37:20.121152 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:37:20.121136    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-6685f84c75-8wmg2"
Oct 31 00:37:26.104491 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:37:26.104448    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/ai-gateway-controller-d6d549b95-zmcsl"
Oct 31 00:37:26.250541 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:37:26.250506    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/ai-gateway-controller-7d85675f98-lh7sz"]
Oct 31 00:37:26.250773 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:37:26.250748    4284 kuberuntime_container.go:898] "Killing container with a grace period" pod="opendatahub/ai-gateway-controller-7d85675f98-lh7sz" podUID="e84b35f2-bb19-4256-b1a0-241ce42ed7df" containerName="manager" containerID="cri-o://9ec52d60b4b891bfbb888f8f917a128a45ed0f9cb7f8b615c1ef8d91dae9ef5f" gracePeriod=30
Oct 31 00:37:26.499354 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:37:26.499327    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/ai-gateway-controller-7d85675f98-lh7sz"
Oct 31 00:37:26.596564 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:37:26.596526    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e84b35f2-bb19-4256-b1a0-241ce42ed7df-kube-api-access-w9zx9\" (UniqueName: \"kubernetes.io/projected/e84b35f2-bb19-4256-b1a0-241ce42ed7df-kube-api-access-w9zx9\") pod \"e84b35f2-bb19-4256-b1a0-241ce42ed7df\" (UID: \"e84b35f2-bb19-4256-b1a0-241ce42ed7df\") "
Oct 31 00:37:26.596740 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:37:26.596587    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/e84b35f2-bb19-4256-b1a0-241ce42ed7df-metrics-certs\" (UniqueName: \"kubernetes.io/secret/e84b35f2-bb19-4256-b1a0-241ce42ed7df-metrics-certs\") pod \"e84b35f2-bb19-4256-b1a0-241ce42ed7df\" (UID: \"e84b35f2-bb19-4256-b1a0-241ce42ed7df\") "
Oct 31 00:37:26.598908 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:37:26.598874    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e84b35f2-bb19-4256-b1a0-241ce42ed7df-metrics-certs" pod "e84b35f2-bb19-4256-b1a0-241ce42ed7df" (UID: "e84b35f2-bb19-4256-b1a0-241ce42ed7df"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 00:37:26.599021 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:37:26.598930    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e84b35f2-bb19-4256-b1a0-241ce42ed7df-kube-api-access-w9zx9" pod "e84b35f2-bb19-4256-b1a0-241ce42ed7df" (UID: "e84b35f2-bb19-4256-b1a0-241ce42ed7df"). InnerVolumeSpecName "kube-api-access-w9zx9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 00:37:26.698132 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:37:26.698093    4284 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w9zx9\" (UniqueName: \"kubernetes.io/projected/e84b35f2-bb19-4256-b1a0-241ce42ed7df-kube-api-access-w9zx9\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:37:26.698132 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:37:26.698126    4284 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e84b35f2-bb19-4256-b1a0-241ce42ed7df-metrics-certs\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:37:27.150752 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:37:27.150663    4284 generic.go:356] "Generic (PLEG): container finished" podID="e84b35f2-bb19-4256-b1a0-241ce42ed7df" containerID="9ec52d60b4b891bfbb888f8f917a128a45ed0f9cb7f8b615c1ef8d91dae9ef5f" exitCode=0
Oct 31 00:37:27.150752 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:37:27.150726    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/ai-gateway-controller-7d85675f98-lh7sz"
Oct 31 00:37:27.151287 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:37:27.150759    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/ai-gateway-controller-7d85675f98-lh7sz" event={"ID":"e84b35f2-bb19-4256-b1a0-241ce42ed7df","Type":"ContainerDied","Data":"9ec52d60b4b891bfbb888f8f917a128a45ed0f9cb7f8b615c1ef8d91dae9ef5f"}
Oct 31 00:37:27.151287 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:37:27.150825    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/ai-gateway-controller-7d85675f98-lh7sz" event={"ID":"e84b35f2-bb19-4256-b1a0-241ce42ed7df","Type":"ContainerDied","Data":"da0ea772cba086058eff904fb2334d53398d054b024aee1c4997f87410fa101f"}
Oct 31 00:37:27.151287 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:37:27.150856    4284 scope.go:122] "RemoveContainer" containerID="9ec52d60b4b891bfbb888f8f917a128a45ed0f9cb7f8b615c1ef8d91dae9ef5f"
Oct 31 00:37:27.159585 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:37:27.159564    4284 scope.go:122] "RemoveContainer" containerID="9ec52d60b4b891bfbb888f8f917a128a45ed0f9cb7f8b615c1ef8d91dae9ef5f"
Oct 31 00:37:27.159892 ip-10-0-132-173 kubenswrapper[4284]: E1031 00:37:27.159876    4284 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9ec52d60b4b891bfbb888f8f917a128a45ed0f9cb7f8b615c1ef8d91dae9ef5f\": container with ID starting with 9ec52d60b4b891bfbb888f8f917a128a45ed0f9cb7f8b615c1ef8d91dae9ef5f not found: ID does not exist" containerID="9ec52d60b4b891bfbb888f8f917a128a45ed0f9cb7f8b615c1ef8d91dae9ef5f"
Oct 31 00:37:27.159932 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:37:27.159902    4284 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9ec52d60b4b891bfbb888f8f917a128a45ed0f9cb7f8b615c1ef8d91dae9ef5f"} err="failed to get container status \"9ec52d60b4b891bfbb888f8f917a128a45ed0f9cb7f8b615c1ef8d91dae9ef5f\": rpc error: code = NotFound desc = could not find container \"9ec52d60b4b891bfbb888f8f917a128a45ed0f9cb7f8b615c1ef8d91dae9ef5f\": container with ID starting with 9ec52d60b4b891bfbb888f8f917a128a45ed0f9cb7f8b615c1ef8d91dae9ef5f not found: ID does not exist"
Oct 31 00:37:27.307987 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:37:27.307955    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/ai-gateway-controller-7d85675f98-lh7sz"]
Oct 31 00:37:27.313884 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:37:27.313854    4284 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["opendatahub/ai-gateway-controller-7d85675f98-lh7sz"]
Oct 31 00:37:27.954272 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:37:27.954239    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:37:27.961174 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:37:27.961146    4284 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e84b35f2-bb19-4256-b1a0-241ce42ed7df" path="/var/lib/kubelet/pods/e84b35f2-bb19-4256-b1a0-241ce42ed7df/volumes"
Oct 31 00:37:28.856687 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:37:28.856657    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:37:29.747946 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:37:29.747906    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:37:29.973679 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:37:29.973642    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:37:31.128636 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:37:31.128609    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-6685f84c75-8wmg2"
Oct 31 00:37:35.923989 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:37:35.923952    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:37:38.813130 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:37:38.813093    4284 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-cbd4f5586-jqgc5"]
Oct 31 00:37:38.813671 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:37:38.813651    4284 cpu_manager.go:414] "containerMap: removing container" podUID="e84b35f2-bb19-4256-b1a0-241ce42ed7df" containerName="manager"
Oct 31 00:37:38.813770 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:37:38.813674    4284 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e84b35f2-bb19-4256-b1a0-241ce42ed7df" containerName="manager"
Oct 31 00:37:38.813814 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:37:38.813777    4284 memory_manager.go:365] "RemoveStaleState removing state" podUID="e84b35f2-bb19-4256-b1a0-241ce42ed7df" containerName="manager"
Oct 31 00:37:38.816970 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:37:38.816951    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-jqgc5"
Oct 31 00:37:38.888867 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:37:38.888838    4284 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Oct 31 00:37:38.890537 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:37:38.890511    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-cbd4f5586-jqgc5"]
Oct 31 00:37:38.890659 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:37:38.890567    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3e84fbe8-43ba-445f-90f9-8d6e899db62f-model-cache\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-jqgc5\" (UID: \"3e84fbe8-43ba-445f-90f9-8d6e899db62f\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-jqgc5"
Oct 31 00:37:38.890659 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:37:38.890603    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3e84fbe8-43ba-445f-90f9-8d6e899db62f-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-jqgc5\" (UID: \"3e84fbe8-43ba-445f-90f9-8d6e899db62f\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-jqgc5"
Oct 31 00:37:38.890749 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:37:38.890699    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c6l57\" (UniqueName: \"kubernetes.io/projected/3e84fbe8-43ba-445f-90f9-8d6e899db62f-kube-api-access-c6l57\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-jqgc5\" (UID: \"3e84fbe8-43ba-445f-90f9-8d6e899db62f\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-jqgc5"
Oct 31 00:37:38.890796 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:37:38.890759    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3e84fbe8-43ba-445f-90f9-8d6e899db62f-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-jqgc5\" (UID: \"3e84fbe8-43ba-445f-90f9-8d6e899db62f\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-jqgc5"
Oct 31 00:37:38.890838 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:37:38.890812    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3e84fbe8-43ba-445f-90f9-8d6e899db62f-home\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-jqgc5\" (UID: \"3e84fbe8-43ba-445f-90f9-8d6e899db62f\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-jqgc5"
Oct 31 00:37:38.890893 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:37:38.890842    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3e84fbe8-43ba-445f-90f9-8d6e899db62f-dshm\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-jqgc5\" (UID: \"3e84fbe8-43ba-445f-90f9-8d6e899db62f\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-jqgc5"
Oct 31 00:37:38.918014 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:37:38.917982    4284 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-pzv4k\""
Oct 31 00:37:38.918014 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:37:38.917997    4284 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\""
Oct 31 00:37:38.918262 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:37:38.918034    4284 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Oct 31 00:37:38.920398 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:37:38.920372    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:37:38.991288 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:37:38.991258    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3e84fbe8-43ba-445f-90f9-8d6e899db62f-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-jqgc5\" (UID: \"3e84fbe8-43ba-445f-90f9-8d6e899db62f\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-jqgc5"
Oct 31 00:37:38.991495 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:37:38.991302    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3e84fbe8-43ba-445f-90f9-8d6e899db62f-home\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-jqgc5\" (UID: \"3e84fbe8-43ba-445f-90f9-8d6e899db62f\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-jqgc5"
Oct 31 00:37:38.991495 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:37:38.991334    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3e84fbe8-43ba-445f-90f9-8d6e899db62f-dshm\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-jqgc5\" (UID: \"3e84fbe8-43ba-445f-90f9-8d6e899db62f\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-jqgc5"
Oct 31 00:37:38.991495 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:37:38.991372    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3e84fbe8-43ba-445f-90f9-8d6e899db62f-model-cache\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-jqgc5\" (UID: \"3e84fbe8-43ba-445f-90f9-8d6e899db62f\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-jqgc5"
Oct 31 00:37:38.991495 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:37:38.991387    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3e84fbe8-43ba-445f-90f9-8d6e899db62f-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-jqgc5\" (UID: \"3e84fbe8-43ba-445f-90f9-8d6e899db62f\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-jqgc5"
Oct 31 00:37:38.991728 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:37:38.991568    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c6l57\" (UniqueName: \"kubernetes.io/projected/3e84fbe8-43ba-445f-90f9-8d6e899db62f-kube-api-access-c6l57\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-jqgc5\" (UID: \"3e84fbe8-43ba-445f-90f9-8d6e899db62f\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-jqgc5"
Oct 31 00:37:38.991836 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:37:38.991787    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3e84fbe8-43ba-445f-90f9-8d6e899db62f-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-jqgc5\" (UID: \"3e84fbe8-43ba-445f-90f9-8d6e899db62f\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-jqgc5"
Oct 31 00:37:38.991918 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:37:38.991852    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3e84fbe8-43ba-445f-90f9-8d6e899db62f-model-cache\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-jqgc5\" (UID: \"3e84fbe8-43ba-445f-90f9-8d6e899db62f\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-jqgc5"
Oct 31 00:37:38.991918 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:37:38.991878    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3e84fbe8-43ba-445f-90f9-8d6e899db62f-home\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-jqgc5\" (UID: \"3e84fbe8-43ba-445f-90f9-8d6e899db62f\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-jqgc5"
Oct 31 00:37:38.994028 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:37:38.994005    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3e84fbe8-43ba-445f-90f9-8d6e899db62f-dshm\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-jqgc5\" (UID: \"3e84fbe8-43ba-445f-90f9-8d6e899db62f\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-jqgc5"
Oct 31 00:37:38.994188 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:37:38.994172    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3e84fbe8-43ba-445f-90f9-8d6e899db62f-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-jqgc5\" (UID: \"3e84fbe8-43ba-445f-90f9-8d6e899db62f\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-jqgc5"
Oct 31 00:37:39.050550 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:37:39.050512    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c6l57\" (UniqueName: \"kubernetes.io/projected/3e84fbe8-43ba-445f-90f9-8d6e899db62f-kube-api-access-c6l57\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-jqgc5\" (UID: \"3e84fbe8-43ba-445f-90f9-8d6e899db62f\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-jqgc5"
Oct 31 00:37:39.128842 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:37:39.128744    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-jqgc5"
Oct 31 00:37:39.496565 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:37:39.496526    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-cbd4f5586-jqgc5"]
Oct 31 00:37:39.498334 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:37:39.498309    4284 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3e84fbe8_43ba_445f_90f9_8d6e899db62f.slice/crio-6b21c62fccda16174319f427c0e4298b35439db8b3631f89d35cf280be0b0c78 WatchSource:0}: Error finding container 6b21c62fccda16174319f427c0e4298b35439db8b3631f89d35cf280be0b0c78: Status 404 returned error can't find the container with id 6b21c62fccda16174319f427c0e4298b35439db8b3631f89d35cf280be0b0c78
Oct 31 00:37:40.208324 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:37:40.208282    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-jqgc5" event={"ID":"3e84fbe8-43ba-445f-90f9-8d6e899db62f","Type":"ContainerStarted","Data":"6b21c62fccda16174319f427c0e4298b35439db8b3631f89d35cf280be0b0c78"}
Oct 31 00:37:42.219086 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:37:42.218667    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-jqgc5" event={"ID":"3e84fbe8-43ba-445f-90f9-8d6e899db62f","Type":"ContainerStarted","Data":"75bad69a0410ac4ab92c1ff5e84abe9656deab571a8984af23ac3e0bebcd9bde"}
Oct 31 00:37:42.356717 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:37:42.356643    4284 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-jqgc5" podStartSLOduration=2.47226632 podStartE2EDuration="4.356626105s" totalImagesPullingTime="1.884359785s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 00:37:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:37:42.332352253 +0000 UTC m=+576.477359170" watchObservedRunningTime="2026-10-31 00:37:42.356626105 +0000 UTC m=+576.501633002"
Oct 31 00:37:43.129321 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:37:43.129283    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-jqgc5"
Oct 31 00:37:47.013465 ip-10-0-132-173 kubenswrapper[4284]: E1031 00:37:47.013381    4284 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fd8sp" containerName="extractor"
Oct 31 00:37:47.415344 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:37:47.415314    4284 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-7b998b79d9-zdlfm"]
Oct 31 00:37:47.417998 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:37:47.417983    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-zdlfm"
Oct 31 00:37:47.451273 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:37:47.451239    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a691a8c5-a962-488d-9b76-41f83166cd8c-model-cache\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-zdlfm\" (UID: \"a691a8c5-a962-488d-9b76-41f83166cd8c\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-zdlfm"
Oct 31 00:37:47.451429 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:37:47.451289    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a691a8c5-a962-488d-9b76-41f83166cd8c-tmp-dir\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-zdlfm\" (UID: \"a691a8c5-a962-488d-9b76-41f83166cd8c\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-zdlfm"
Oct 31 00:37:47.451429 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:37:47.451337    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hc82d\" (UniqueName: \"kubernetes.io/projected/a691a8c5-a962-488d-9b76-41f83166cd8c-kube-api-access-hc82d\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-zdlfm\" (UID: \"a691a8c5-a962-488d-9b76-41f83166cd8c\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-zdlfm"
Oct 31 00:37:47.451539 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:37:47.451433    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a691a8c5-a962-488d-9b76-41f83166cd8c-home\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-zdlfm\" (UID: \"a691a8c5-a962-488d-9b76-41f83166cd8c\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-zdlfm"
Oct 31 00:37:47.451539 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:37:47.451459    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a691a8c5-a962-488d-9b76-41f83166cd8c-tls-certs\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-zdlfm\" (UID: \"a691a8c5-a962-488d-9b76-41f83166cd8c\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-zdlfm"
Oct 31 00:37:47.451539 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:37:47.451511    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a691a8c5-a962-488d-9b76-41f83166cd8c-dshm\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-zdlfm\" (UID: \"a691a8c5-a962-488d-9b76-41f83166cd8c\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-zdlfm"
Oct 31 00:37:47.515697 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:37:47.515659    4284 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\""
Oct 31 00:37:47.517157 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:37:47.517136    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-7b998b79d9-zdlfm"]
Oct 31 00:37:47.529677 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:37:47.529656    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:37:47.552653 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:37:47.552621    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a691a8c5-a962-488d-9b76-41f83166cd8c-tmp-dir\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-zdlfm\" (UID: \"a691a8c5-a962-488d-9b76-41f83166cd8c\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-zdlfm"
Oct 31 00:37:47.552653 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:37:47.552651    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hc82d\" (UniqueName: \"kubernetes.io/projected/a691a8c5-a962-488d-9b76-41f83166cd8c-kube-api-access-hc82d\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-zdlfm\" (UID: \"a691a8c5-a962-488d-9b76-41f83166cd8c\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-zdlfm"
Oct 31 00:37:47.552832 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:37:47.552692    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a691a8c5-a962-488d-9b76-41f83166cd8c-home\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-zdlfm\" (UID: \"a691a8c5-a962-488d-9b76-41f83166cd8c\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-zdlfm"
Oct 31 00:37:47.552832 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:37:47.552714    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a691a8c5-a962-488d-9b76-41f83166cd8c-tls-certs\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-zdlfm\" (UID: \"a691a8c5-a962-488d-9b76-41f83166cd8c\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-zdlfm"
Oct 31 00:37:47.552832 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:37:47.552756    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a691a8c5-a962-488d-9b76-41f83166cd8c-dshm\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-zdlfm\" (UID: \"a691a8c5-a962-488d-9b76-41f83166cd8c\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-zdlfm"
Oct 31 00:37:47.552983 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:37:47.552833    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a691a8c5-a962-488d-9b76-41f83166cd8c-model-cache\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-zdlfm\" (UID: \"a691a8c5-a962-488d-9b76-41f83166cd8c\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-zdlfm"
Oct 31 00:37:47.553133 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:37:47.553076    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a691a8c5-a962-488d-9b76-41f83166cd8c-home\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-zdlfm\" (UID: \"a691a8c5-a962-488d-9b76-41f83166cd8c\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-zdlfm"
Oct 31 00:37:47.553133 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:37:47.553105    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a691a8c5-a962-488d-9b76-41f83166cd8c-tmp-dir\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-zdlfm\" (UID: \"a691a8c5-a962-488d-9b76-41f83166cd8c\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-zdlfm"
Oct 31 00:37:47.553289 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:37:47.553266    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a691a8c5-a962-488d-9b76-41f83166cd8c-model-cache\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-zdlfm\" (UID: \"a691a8c5-a962-488d-9b76-41f83166cd8c\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-zdlfm"
Oct 31 00:37:47.555405 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:37:47.555383    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a691a8c5-a962-488d-9b76-41f83166cd8c-dshm\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-zdlfm\" (UID: \"a691a8c5-a962-488d-9b76-41f83166cd8c\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-zdlfm"
Oct 31 00:37:47.555981 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:37:47.555959    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a691a8c5-a962-488d-9b76-41f83166cd8c-tls-certs\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-zdlfm\" (UID: \"a691a8c5-a962-488d-9b76-41f83166cd8c\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-zdlfm"
Oct 31 00:37:47.614277 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:37:47.614202    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hc82d\" (UniqueName: \"kubernetes.io/projected/a691a8c5-a962-488d-9b76-41f83166cd8c-kube-api-access-hc82d\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-zdlfm\" (UID: \"a691a8c5-a962-488d-9b76-41f83166cd8c\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-zdlfm"
Oct 31 00:37:47.729285 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:37:47.729155    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-zdlfm"
Oct 31 00:37:48.096864 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:37:48.096832    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-7b998b79d9-zdlfm"]
Oct 31 00:37:48.097299 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:37:48.097172    4284 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda691a8c5_a962_488d_9b76_41f83166cd8c.slice/crio-960a65cffea54b430e5e7bdbecb228cfa8bbaaa295c18528400c82cd3f65f2d7 WatchSource:0}: Error finding container 960a65cffea54b430e5e7bdbecb228cfa8bbaaa295c18528400c82cd3f65f2d7: Status 404 returned error can't find the container with id 960a65cffea54b430e5e7bdbecb228cfa8bbaaa295c18528400c82cd3f65f2d7
Oct 31 00:37:48.245469 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:37:48.245430    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-zdlfm" event={"ID":"a691a8c5-a962-488d-9b76-41f83166cd8c","Type":"ContainerStarted","Data":"960a65cffea54b430e5e7bdbecb228cfa8bbaaa295c18528400c82cd3f65f2d7"}
Oct 31 00:37:49.129832 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:37:49.129742    4284 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-jqgc5"
Oct 31 00:37:49.142998 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:37:49.142974    4284 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-jqgc5"
Oct 31 00:37:49.251494 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:37:49.251457    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-zdlfm" event={"ID":"a691a8c5-a962-488d-9b76-41f83166cd8c","Type":"ContainerStarted","Data":"9069ed81d29bfa412bf010def70697638df9325d44842ed064727d363c65706a"}
Oct 31 00:37:49.264000 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:37:49.263968    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-jqgc5"
Oct 31 00:37:49.369845 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:37:49.369788    4284 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-zdlfm" podStartSLOduration=1.772197368 podStartE2EDuration="2.369774468s" totalImagesPullingTime="597.5771ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 00:37:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:37:49.36906739 +0000 UTC m=+583.514074306" watchObservedRunningTime="2026-10-31 00:37:49.369774468 +0000 UTC m=+583.514781383"
Oct 31 00:37:49.730307 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:37:49.730268    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-zdlfm"
Oct 31 00:37:51.067463 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:37:51.067428    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:37:51.917672 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:37:51.917638    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:37:53.933142 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:37:53.933108    4284 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-69b588897d-zk45d"]
Oct 31 00:37:53.935758 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:37:53.935742    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-zk45d"
Oct 31 00:37:54.008581 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:37:54.008543    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bf572ad9-cd2a-43ad-a627-6a1f0dc0536f-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-zk45d\" (UID: \"bf572ad9-cd2a-43ad-a627-6a1f0dc0536f\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-zk45d"
Oct 31 00:37:54.008769 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:37:54.008601    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bf572ad9-cd2a-43ad-a627-6a1f0dc0536f-home\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-zk45d\" (UID: \"bf572ad9-cd2a-43ad-a627-6a1f0dc0536f\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-zk45d"
Oct 31 00:37:54.008769 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:37:54.008683    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bf572ad9-cd2a-43ad-a627-6a1f0dc0536f-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-zk45d\" (UID: \"bf572ad9-cd2a-43ad-a627-6a1f0dc0536f\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-zk45d"
Oct 31 00:37:54.008891 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:37:54.008766    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bf572ad9-cd2a-43ad-a627-6a1f0dc0536f-model-cache\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-zk45d\" (UID: \"bf572ad9-cd2a-43ad-a627-6a1f0dc0536f\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-zk45d"
Oct 31 00:37:54.008891 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:37:54.008842    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n55m4\" (UniqueName: \"kubernetes.io/projected/bf572ad9-cd2a-43ad-a627-6a1f0dc0536f-kube-api-access-n55m4\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-zk45d\" (UID: \"bf572ad9-cd2a-43ad-a627-6a1f0dc0536f\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-zk45d"
Oct 31 00:37:54.009012 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:37:54.008940    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bf572ad9-cd2a-43ad-a627-6a1f0dc0536f-dshm\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-zk45d\" (UID: \"bf572ad9-cd2a-43ad-a627-6a1f0dc0536f\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-zk45d"
Oct 31 00:37:54.028770 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:37:54.028730    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:37:54.033628 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:37:54.033607    4284 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\""
Oct 31 00:37:54.035129 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:37:54.035112    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-69b588897d-zk45d"]
Oct 31 00:37:54.109864 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:37:54.109822    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bf572ad9-cd2a-43ad-a627-6a1f0dc0536f-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-zk45d\" (UID: \"bf572ad9-cd2a-43ad-a627-6a1f0dc0536f\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-zk45d"
Oct 31 00:37:54.110025 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:37:54.109891    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bf572ad9-cd2a-43ad-a627-6a1f0dc0536f-model-cache\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-zk45d\" (UID: \"bf572ad9-cd2a-43ad-a627-6a1f0dc0536f\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-zk45d"
Oct 31 00:37:54.110025 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:37:54.109928    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n55m4\" (UniqueName: \"kubernetes.io/projected/bf572ad9-cd2a-43ad-a627-6a1f0dc0536f-kube-api-access-n55m4\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-zk45d\" (UID: \"bf572ad9-cd2a-43ad-a627-6a1f0dc0536f\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-zk45d"
Oct 31 00:37:54.110025 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:37:54.109974    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bf572ad9-cd2a-43ad-a627-6a1f0dc0536f-dshm\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-zk45d\" (UID: \"bf572ad9-cd2a-43ad-a627-6a1f0dc0536f\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-zk45d"
Oct 31 00:37:54.110025 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:37:54.110017    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bf572ad9-cd2a-43ad-a627-6a1f0dc0536f-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-zk45d\" (UID: \"bf572ad9-cd2a-43ad-a627-6a1f0dc0536f\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-zk45d"
Oct 31 00:37:54.110204 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:37:54.110047    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bf572ad9-cd2a-43ad-a627-6a1f0dc0536f-home\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-zk45d\" (UID: \"bf572ad9-cd2a-43ad-a627-6a1f0dc0536f\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-zk45d"
Oct 31 00:37:54.110448 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:37:54.110423    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bf572ad9-cd2a-43ad-a627-6a1f0dc0536f-home\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-zk45d\" (UID: \"bf572ad9-cd2a-43ad-a627-6a1f0dc0536f\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-zk45d"
Oct 31 00:37:54.110507 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:37:54.110458    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bf572ad9-cd2a-43ad-a627-6a1f0dc0536f-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-zk45d\" (UID: \"bf572ad9-cd2a-43ad-a627-6a1f0dc0536f\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-zk45d"
Oct 31 00:37:54.110568 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:37:54.110554    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bf572ad9-cd2a-43ad-a627-6a1f0dc0536f-model-cache\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-zk45d\" (UID: \"bf572ad9-cd2a-43ad-a627-6a1f0dc0536f\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-zk45d"
Oct 31 00:37:54.112504 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:37:54.112484    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bf572ad9-cd2a-43ad-a627-6a1f0dc0536f-dshm\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-zk45d\" (UID: \"bf572ad9-cd2a-43ad-a627-6a1f0dc0536f\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-zk45d"
Oct 31 00:37:54.112668 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:37:54.112638    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bf572ad9-cd2a-43ad-a627-6a1f0dc0536f-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-zk45d\" (UID: \"bf572ad9-cd2a-43ad-a627-6a1f0dc0536f\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-zk45d"
Oct 31 00:37:54.169118 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:37:54.169081    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n55m4\" (UniqueName: \"kubernetes.io/projected/bf572ad9-cd2a-43ad-a627-6a1f0dc0536f-kube-api-access-n55m4\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-zk45d\" (UID: \"bf572ad9-cd2a-43ad-a627-6a1f0dc0536f\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-zk45d"
Oct 31 00:37:54.247099 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:37:54.247008    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-zk45d"
Oct 31 00:37:54.613543 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:37:54.613516    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-69b588897d-zk45d"]
Oct 31 00:37:54.615377 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:37:54.615352    4284 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbf572ad9_cd2a_43ad_a627_6a1f0dc0536f.slice/crio-b4ab65939463c2bae6ea7ebef2f5355f6301fc1e33b77f52bce542a403277cf3 WatchSource:0}: Error finding container b4ab65939463c2bae6ea7ebef2f5355f6301fc1e33b77f52bce542a403277cf3: Status 404 returned error can't find the container with id b4ab65939463c2bae6ea7ebef2f5355f6301fc1e33b77f52bce542a403277cf3
Oct 31 00:37:55.278856 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:37:55.278829    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-zk45d" event={"ID":"bf572ad9-cd2a-43ad-a627-6a1f0dc0536f","Type":"ContainerStarted","Data":"b4ab65939463c2bae6ea7ebef2f5355f6301fc1e33b77f52bce542a403277cf3"}
Oct 31 00:37:56.283921 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:37:56.283885    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-zk45d" event={"ID":"bf572ad9-cd2a-43ad-a627-6a1f0dc0536f","Type":"ContainerStarted","Data":"5ebd84fa20dd03524c7cc9c40b13e041679d8420373e3e92290a9e8b13b4fea8"}
Oct 31 00:37:56.396550 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:37:56.396500    4284 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-zk45d" podStartSLOduration=2.7803409439999998 podStartE2EDuration="3.396485786s" totalImagesPullingTime="616.144842ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 00:37:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:37:56.396229955 +0000 UTC m=+590.541236861" watchObservedRunningTime="2026-10-31 00:37:56.396485786 +0000 UTC m=+590.541492701"
Oct 31 00:37:56.634583 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:37:56.634502    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:37:56.787984 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:37:56.787950    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:37:57.248058 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:37:57.248020    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-zk45d"
Oct 31 00:37:57.371196 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:37:57.371161    4284 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b82bqnm"]
Oct 31 00:37:57.374170 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:37:57.374149    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b82bqnm"
Oct 31 00:37:57.424607 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:37:57.424579    4284 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\""
Oct 31 00:37:57.438956 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:37:57.438924    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/543969c3-5246-4037-9101-7c566b535e9d-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b82bqnm\" (UID: \"543969c3-5246-4037-9101-7c566b535e9d\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b82bqnm"
Oct 31 00:37:57.438956 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:37:57.438959    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/543969c3-5246-4037-9101-7c566b535e9d-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b82bqnm\" (UID: \"543969c3-5246-4037-9101-7c566b535e9d\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b82bqnm"
Oct 31 00:37:57.439274 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:37:57.438998    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/543969c3-5246-4037-9101-7c566b535e9d-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b82bqnm\" (UID: \"543969c3-5246-4037-9101-7c566b535e9d\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b82bqnm"
Oct 31 00:37:57.439274 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:37:57.439100    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/543969c3-5246-4037-9101-7c566b535e9d-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b82bqnm\" (UID: \"543969c3-5246-4037-9101-7c566b535e9d\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b82bqnm"
Oct 31 00:37:57.439274 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:37:57.439138    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ptvm2\" (UniqueName: \"kubernetes.io/projected/543969c3-5246-4037-9101-7c566b535e9d-kube-api-access-ptvm2\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b82bqnm\" (UID: \"543969c3-5246-4037-9101-7c566b535e9d\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b82bqnm"
Oct 31 00:37:57.439274 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:37:57.439157    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/543969c3-5246-4037-9101-7c566b535e9d-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b82bqnm\" (UID: \"543969c3-5246-4037-9101-7c566b535e9d\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b82bqnm"
Oct 31 00:37:57.470729 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:37:57.470695    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:37:57.473559 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:37:57.473527    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b82bqnm"]
Oct 31 00:37:57.539744 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:37:57.539640    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/543969c3-5246-4037-9101-7c566b535e9d-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b82bqnm\" (UID: \"543969c3-5246-4037-9101-7c566b535e9d\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b82bqnm"
Oct 31 00:37:57.539744 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:37:57.539686    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/543969c3-5246-4037-9101-7c566b535e9d-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b82bqnm\" (UID: \"543969c3-5246-4037-9101-7c566b535e9d\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b82bqnm"
Oct 31 00:37:57.539744 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:37:57.539720    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/543969c3-5246-4037-9101-7c566b535e9d-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b82bqnm\" (UID: \"543969c3-5246-4037-9101-7c566b535e9d\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b82bqnm"
Oct 31 00:37:57.539744 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:37:57.539746    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ptvm2\" (UniqueName: \"kubernetes.io/projected/543969c3-5246-4037-9101-7c566b535e9d-kube-api-access-ptvm2\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b82bqnm\" (UID: \"543969c3-5246-4037-9101-7c566b535e9d\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b82bqnm"
Oct 31 00:37:57.540085 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:37:57.539763    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/543969c3-5246-4037-9101-7c566b535e9d-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b82bqnm\" (UID: \"543969c3-5246-4037-9101-7c566b535e9d\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b82bqnm"
Oct 31 00:37:57.540085 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:37:57.539797    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/543969c3-5246-4037-9101-7c566b535e9d-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b82bqnm\" (UID: \"543969c3-5246-4037-9101-7c566b535e9d\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b82bqnm"
Oct 31 00:37:57.540196 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:37:57.540101    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/543969c3-5246-4037-9101-7c566b535e9d-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b82bqnm\" (UID: \"543969c3-5246-4037-9101-7c566b535e9d\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b82bqnm"
Oct 31 00:37:57.540196 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:37:57.540115    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/543969c3-5246-4037-9101-7c566b535e9d-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b82bqnm\" (UID: \"543969c3-5246-4037-9101-7c566b535e9d\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b82bqnm"
Oct 31 00:37:57.540489 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:37:57.540194    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/543969c3-5246-4037-9101-7c566b535e9d-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b82bqnm\" (UID: \"543969c3-5246-4037-9101-7c566b535e9d\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b82bqnm"
Oct 31 00:37:57.542250 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:37:57.542226    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/543969c3-5246-4037-9101-7c566b535e9d-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b82bqnm\" (UID: \"543969c3-5246-4037-9101-7c566b535e9d\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b82bqnm"
Oct 31 00:37:57.542365 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:37:57.542332    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/543969c3-5246-4037-9101-7c566b535e9d-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b82bqnm\" (UID: \"543969c3-5246-4037-9101-7c566b535e9d\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b82bqnm"
Oct 31 00:37:57.599124 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:37:57.599091    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ptvm2\" (UniqueName: \"kubernetes.io/projected/543969c3-5246-4037-9101-7c566b535e9d-kube-api-access-ptvm2\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b82bqnm\" (UID: \"543969c3-5246-4037-9101-7c566b535e9d\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b82bqnm"
Oct 31 00:37:57.685618 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:37:57.685589    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b82bqnm"
Oct 31 00:37:57.730243 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:37:57.730105    4284 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-zdlfm"
Oct 31 00:37:57.748919 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:37:57.748891    4284 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-zdlfm"
Oct 31 00:37:58.051168 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:37:58.051148    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b82bqnm"]
Oct 31 00:37:58.053037 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:37:58.053014    4284 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod543969c3_5246_4037_9101_7c566b535e9d.slice/crio-b468d9a8b66c33205f5fd7f1b001897a08c0c70e561813e5669b240e7202137e WatchSource:0}: Error finding container b468d9a8b66c33205f5fd7f1b001897a08c0c70e561813e5669b240e7202137e: Status 404 returned error can't find the container with id b468d9a8b66c33205f5fd7f1b001897a08c0c70e561813e5669b240e7202137e
Oct 31 00:37:58.293369 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:37:58.293331    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b82bqnm" event={"ID":"543969c3-5246-4037-9101-7c566b535e9d","Type":"ContainerStarted","Data":"b468d9a8b66c33205f5fd7f1b001897a08c0c70e561813e5669b240e7202137e"}
Oct 31 00:37:58.304940 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:37:58.304863    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-zdlfm"
Oct 31 00:37:59.300197 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:37:59.300162    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b82bqnm" event={"ID":"543969c3-5246-4037-9101-7c566b535e9d","Type":"ContainerStarted","Data":"a85aa71110624a07e681d1627591e2693b9c996f175d7dc2ed2c967b39e53928"}
Oct 31 00:37:59.417621 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:37:59.417562    4284 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b82bqnm" podStartSLOduration=1.811110724 podStartE2EDuration="2.417547718s" totalImagesPullingTime="606.436994ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 00:37:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:37:59.415344677 +0000 UTC m=+593.560351604" watchObservedRunningTime="2026-10-31 00:37:59.417547718 +0000 UTC m=+593.562554634"
Oct 31 00:37:59.686447 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:37:59.686414    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b82bqnm"
Oct 31 00:38:01.777387 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:38:01.777346    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:38:04.247421 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:38:04.247381    4284 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-zk45d"
Oct 31 00:38:04.259784 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:38:04.259761    4284 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-zk45d"
Oct 31 00:38:04.331796 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:38:04.331770    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-zk45d"
Oct 31 00:38:05.080587 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:38:05.080555    4284 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-84b4ff656c-8mnf5"]
Oct 31 00:38:05.083131 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:38:05.083110    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-8mnf5"
Oct 31 00:38:05.150889 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:38:05.150859    4284 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\""
Oct 31 00:38:05.182207 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:38:05.182169    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-84b4ff656c-8mnf5"]
Oct 31 00:38:05.212701 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:38:05.212667    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0f2982ea-8a19-4b39-b859-f52896b34b5b-dshm\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-8mnf5\" (UID: \"0f2982ea-8a19-4b39-b859-f52896b34b5b\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-8mnf5"
Oct 31 00:38:05.212701 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:38:05.212708    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0f2982ea-8a19-4b39-b859-f52896b34b5b-home\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-8mnf5\" (UID: \"0f2982ea-8a19-4b39-b859-f52896b34b5b\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-8mnf5"
Oct 31 00:38:05.212970 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:38:05.212726    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0f2982ea-8a19-4b39-b859-f52896b34b5b-model-cache\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-8mnf5\" (UID: \"0f2982ea-8a19-4b39-b859-f52896b34b5b\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-8mnf5"
Oct 31 00:38:05.212970 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:38:05.212806    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qhnjz\" (UniqueName: \"kubernetes.io/projected/0f2982ea-8a19-4b39-b859-f52896b34b5b-kube-api-access-qhnjz\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-8mnf5\" (UID: \"0f2982ea-8a19-4b39-b859-f52896b34b5b\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-8mnf5"
Oct 31 00:38:05.212970 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:38:05.212961    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0f2982ea-8a19-4b39-b859-f52896b34b5b-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-8mnf5\" (UID: \"0f2982ea-8a19-4b39-b859-f52896b34b5b\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-8mnf5"
Oct 31 00:38:05.213089 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:38:05.213005    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0f2982ea-8a19-4b39-b859-f52896b34b5b-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-8mnf5\" (UID: \"0f2982ea-8a19-4b39-b859-f52896b34b5b\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-8mnf5"
Oct 31 00:38:05.272388 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:38:05.272342    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:38:05.313799 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:38:05.313766    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0f2982ea-8a19-4b39-b859-f52896b34b5b-dshm\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-8mnf5\" (UID: \"0f2982ea-8a19-4b39-b859-f52896b34b5b\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-8mnf5"
Oct 31 00:38:05.313799 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:38:05.313805    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0f2982ea-8a19-4b39-b859-f52896b34b5b-home\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-8mnf5\" (UID: \"0f2982ea-8a19-4b39-b859-f52896b34b5b\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-8mnf5"
Oct 31 00:38:05.314016 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:38:05.313830    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0f2982ea-8a19-4b39-b859-f52896b34b5b-model-cache\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-8mnf5\" (UID: \"0f2982ea-8a19-4b39-b859-f52896b34b5b\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-8mnf5"
Oct 31 00:38:05.314016 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:38:05.313856    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qhnjz\" (UniqueName: \"kubernetes.io/projected/0f2982ea-8a19-4b39-b859-f52896b34b5b-kube-api-access-qhnjz\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-8mnf5\" (UID: \"0f2982ea-8a19-4b39-b859-f52896b34b5b\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-8mnf5"
Oct 31 00:38:05.314016 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:38:05.313903    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0f2982ea-8a19-4b39-b859-f52896b34b5b-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-8mnf5\" (UID: \"0f2982ea-8a19-4b39-b859-f52896b34b5b\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-8mnf5"
Oct 31 00:38:05.314016 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:38:05.313950    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0f2982ea-8a19-4b39-b859-f52896b34b5b-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-8mnf5\" (UID: \"0f2982ea-8a19-4b39-b859-f52896b34b5b\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-8mnf5"
Oct 31 00:38:05.314259 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:38:05.314236    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0f2982ea-8a19-4b39-b859-f52896b34b5b-home\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-8mnf5\" (UID: \"0f2982ea-8a19-4b39-b859-f52896b34b5b\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-8mnf5"
Oct 31 00:38:05.314315 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:38:05.314270    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0f2982ea-8a19-4b39-b859-f52896b34b5b-model-cache\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-8mnf5\" (UID: \"0f2982ea-8a19-4b39-b859-f52896b34b5b\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-8mnf5"
Oct 31 00:38:05.314398 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:38:05.314381    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0f2982ea-8a19-4b39-b859-f52896b34b5b-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-8mnf5\" (UID: \"0f2982ea-8a19-4b39-b859-f52896b34b5b\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-8mnf5"
Oct 31 00:38:05.316400 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:38:05.316377    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0f2982ea-8a19-4b39-b859-f52896b34b5b-dshm\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-8mnf5\" (UID: \"0f2982ea-8a19-4b39-b859-f52896b34b5b\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-8mnf5"
Oct 31 00:38:05.316696 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:38:05.316677    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0f2982ea-8a19-4b39-b859-f52896b34b5b-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-8mnf5\" (UID: \"0f2982ea-8a19-4b39-b859-f52896b34b5b\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-8mnf5"
Oct 31 00:38:05.373448 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:38:05.373370    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qhnjz\" (UniqueName: \"kubernetes.io/projected/0f2982ea-8a19-4b39-b859-f52896b34b5b-kube-api-access-qhnjz\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-8mnf5\" (UID: \"0f2982ea-8a19-4b39-b859-f52896b34b5b\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-8mnf5"
Oct 31 00:38:05.394359 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:38:05.394331    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-8mnf5"
Oct 31 00:38:05.762488 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:38:05.762464    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-84b4ff656c-8mnf5"]
Oct 31 00:38:05.764444 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:38:05.764416    4284 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0f2982ea_8a19_4b39_b859_f52896b34b5b.slice/crio-261fa10e228fa65e3924fd6b04e59f51ef6b82c6bcc5b39322cf8d0624860fdb WatchSource:0}: Error finding container 261fa10e228fa65e3924fd6b04e59f51ef6b82c6bcc5b39322cf8d0624860fdb: Status 404 returned error can't find the container with id 261fa10e228fa65e3924fd6b04e59f51ef6b82c6bcc5b39322cf8d0624860fdb
Oct 31 00:38:06.332852 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:38:06.332815    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-8mnf5" event={"ID":"0f2982ea-8a19-4b39-b859-f52896b34b5b","Type":"ContainerStarted","Data":"261fa10e228fa65e3924fd6b04e59f51ef6b82c6bcc5b39322cf8d0624860fdb"}
Oct 31 00:38:07.338719 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:38:07.338688    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-8mnf5" event={"ID":"0f2982ea-8a19-4b39-b859-f52896b34b5b","Type":"ContainerStarted","Data":"08c87c8c8db9938f204b4fb56f03dc1256acdaaad9436adf56cfe99a03511d8d"}
Oct 31 00:38:07.394850 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:38:07.394820    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-8mnf5"
Oct 31 00:38:07.453069 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:38:07.453020    4284 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-8mnf5" podStartSLOduration=1.8576180899999999 podStartE2EDuration="2.453006075s" totalImagesPullingTime="595.387985ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 00:38:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:38:07.451913555 +0000 UTC m=+601.596920472" watchObservedRunningTime="2026-10-31 00:38:07.453006075 +0000 UTC m=+601.598012990"
Oct 31 00:38:07.686293 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:38:07.686255    4284 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b82bqnm"
Oct 31 00:38:07.699232 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:38:07.699194    4284 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b82bqnm"
Oct 31 00:38:08.357317 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:38:08.357288    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b82bqnm"
Oct 31 00:38:08.448849 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:38:08.448816    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:38:15.394969 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:38:15.394934    4284 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-8mnf5"
Oct 31 00:38:15.407539 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:38:15.407510    4284 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-8mnf5"
Oct 31 00:38:16.390262 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:38:16.390230    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-8mnf5"
Oct 31 00:38:17.015915 ip-10-0-132-173 kubenswrapper[4284]: E1031 00:38:17.015881    4284 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fd8sp" containerName="extractor"
Oct 31 00:38:19.486580 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:38:19.486548    4284 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-rr5k6"]
Oct 31 00:38:19.490042 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:38:19.490021    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-rr5k6"
Oct 31 00:38:19.587807 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:38:19.587777    4284 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-5wx8d\""
Oct 31 00:38:19.590127 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:38:19.590099    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-rr5k6"]
Oct 31 00:38:19.645055 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:38:19.645009    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/27f8cf21-0f47-4397-831a-bf716741dc67-workload-certs\") pod \"maas-default-gateway-openshift-default-6cc458d887-rr5k6\" (UID: \"27f8cf21-0f47-4397-831a-bf716741dc67\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-rr5k6"
Oct 31 00:38:19.645055 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:38:19.645055    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/27f8cf21-0f47-4397-831a-bf716741dc67-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-6cc458d887-rr5k6\" (UID: \"27f8cf21-0f47-4397-831a-bf716741dc67\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-rr5k6"
Oct 31 00:38:19.645304 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:38:19.645084    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/27f8cf21-0f47-4397-831a-bf716741dc67-istio-envoy\") pod \"maas-default-gateway-openshift-default-6cc458d887-rr5k6\" (UID: \"27f8cf21-0f47-4397-831a-bf716741dc67\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-rr5k6"
Oct 31 00:38:19.645304 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:38:19.645205    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/27f8cf21-0f47-4397-831a-bf716741dc67-istio-token\") pod \"maas-default-gateway-openshift-default-6cc458d887-rr5k6\" (UID: \"27f8cf21-0f47-4397-831a-bf716741dc67\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-rr5k6"
Oct 31 00:38:19.645304 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:38:19.645253    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/27f8cf21-0f47-4397-831a-bf716741dc67-istio-podinfo\") pod \"maas-default-gateway-openshift-default-6cc458d887-rr5k6\" (UID: \"27f8cf21-0f47-4397-831a-bf716741dc67\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-rr5k6"
Oct 31 00:38:19.645304 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:38:19.645272    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-slzp9\" (UniqueName: \"kubernetes.io/projected/27f8cf21-0f47-4397-831a-bf716741dc67-kube-api-access-slzp9\") pod \"maas-default-gateway-openshift-default-6cc458d887-rr5k6\" (UID: \"27f8cf21-0f47-4397-831a-bf716741dc67\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-rr5k6"
Oct 31 00:38:19.645511 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:38:19.645402    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/27f8cf21-0f47-4397-831a-bf716741dc67-workload-socket\") pod \"maas-default-gateway-openshift-default-6cc458d887-rr5k6\" (UID: \"27f8cf21-0f47-4397-831a-bf716741dc67\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-rr5k6"
Oct 31 00:38:19.645511 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:38:19.645452    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/27f8cf21-0f47-4397-831a-bf716741dc67-credential-socket\") pod \"maas-default-gateway-openshift-default-6cc458d887-rr5k6\" (UID: \"27f8cf21-0f47-4397-831a-bf716741dc67\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-rr5k6"
Oct 31 00:38:19.645590 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:38:19.645534    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/27f8cf21-0f47-4397-831a-bf716741dc67-istio-data\") pod \"maas-default-gateway-openshift-default-6cc458d887-rr5k6\" (UID: \"27f8cf21-0f47-4397-831a-bf716741dc67\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-rr5k6"
Oct 31 00:38:19.746282 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:38:19.746173    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/27f8cf21-0f47-4397-831a-bf716741dc67-istio-token\") pod \"maas-default-gateway-openshift-default-6cc458d887-rr5k6\" (UID: \"27f8cf21-0f47-4397-831a-bf716741dc67\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-rr5k6"
Oct 31 00:38:19.746282 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:38:19.746232    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/27f8cf21-0f47-4397-831a-bf716741dc67-istio-podinfo\") pod \"maas-default-gateway-openshift-default-6cc458d887-rr5k6\" (UID: \"27f8cf21-0f47-4397-831a-bf716741dc67\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-rr5k6"
Oct 31 00:38:19.746282 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:38:19.746250    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-slzp9\" (UniqueName: \"kubernetes.io/projected/27f8cf21-0f47-4397-831a-bf716741dc67-kube-api-access-slzp9\") pod \"maas-default-gateway-openshift-default-6cc458d887-rr5k6\" (UID: \"27f8cf21-0f47-4397-831a-bf716741dc67\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-rr5k6"
Oct 31 00:38:19.746282 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:38:19.746281    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/27f8cf21-0f47-4397-831a-bf716741dc67-workload-socket\") pod \"maas-default-gateway-openshift-default-6cc458d887-rr5k6\" (UID: \"27f8cf21-0f47-4397-831a-bf716741dc67\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-rr5k6"
Oct 31 00:38:19.746619 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:38:19.746300    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/27f8cf21-0f47-4397-831a-bf716741dc67-credential-socket\") pod \"maas-default-gateway-openshift-default-6cc458d887-rr5k6\" (UID: \"27f8cf21-0f47-4397-831a-bf716741dc67\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-rr5k6"
Oct 31 00:38:19.746619 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:38:19.746319    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/27f8cf21-0f47-4397-831a-bf716741dc67-istio-data\") pod \"maas-default-gateway-openshift-default-6cc458d887-rr5k6\" (UID: \"27f8cf21-0f47-4397-831a-bf716741dc67\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-rr5k6"
Oct 31 00:38:19.746619 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:38:19.746351    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/27f8cf21-0f47-4397-831a-bf716741dc67-workload-certs\") pod \"maas-default-gateway-openshift-default-6cc458d887-rr5k6\" (UID: \"27f8cf21-0f47-4397-831a-bf716741dc67\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-rr5k6"
Oct 31 00:38:19.746619 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:38:19.746370    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/27f8cf21-0f47-4397-831a-bf716741dc67-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-6cc458d887-rr5k6\" (UID: \"27f8cf21-0f47-4397-831a-bf716741dc67\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-rr5k6"
Oct 31 00:38:19.746619 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:38:19.746399    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/27f8cf21-0f47-4397-831a-bf716741dc67-istio-envoy\") pod \"maas-default-gateway-openshift-default-6cc458d887-rr5k6\" (UID: \"27f8cf21-0f47-4397-831a-bf716741dc67\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-rr5k6"
Oct 31 00:38:19.746873 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:38:19.746748    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/27f8cf21-0f47-4397-831a-bf716741dc67-workload-socket\") pod \"maas-default-gateway-openshift-default-6cc458d887-rr5k6\" (UID: \"27f8cf21-0f47-4397-831a-bf716741dc67\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-rr5k6"
Oct 31 00:38:19.746873 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:38:19.746791    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/27f8cf21-0f47-4397-831a-bf716741dc67-credential-socket\") pod \"maas-default-gateway-openshift-default-6cc458d887-rr5k6\" (UID: \"27f8cf21-0f47-4397-831a-bf716741dc67\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-rr5k6"
Oct 31 00:38:19.746978 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:38:19.746902    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/27f8cf21-0f47-4397-831a-bf716741dc67-istio-data\") pod \"maas-default-gateway-openshift-default-6cc458d887-rr5k6\" (UID: \"27f8cf21-0f47-4397-831a-bf716741dc67\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-rr5k6"
Oct 31 00:38:19.746978 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:38:19.746953    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/27f8cf21-0f47-4397-831a-bf716741dc67-workload-certs\") pod \"maas-default-gateway-openshift-default-6cc458d887-rr5k6\" (UID: \"27f8cf21-0f47-4397-831a-bf716741dc67\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-rr5k6"
Oct 31 00:38:19.747264 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:38:19.747245    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/27f8cf21-0f47-4397-831a-bf716741dc67-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-6cc458d887-rr5k6\" (UID: \"27f8cf21-0f47-4397-831a-bf716741dc67\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-rr5k6"
Oct 31 00:38:19.748743 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:38:19.748724    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/27f8cf21-0f47-4397-831a-bf716741dc67-istio-envoy\") pod \"maas-default-gateway-openshift-default-6cc458d887-rr5k6\" (UID: \"27f8cf21-0f47-4397-831a-bf716741dc67\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-rr5k6"
Oct 31 00:38:19.748890 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:38:19.748875    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/27f8cf21-0f47-4397-831a-bf716741dc67-istio-podinfo\") pod \"maas-default-gateway-openshift-default-6cc458d887-rr5k6\" (UID: \"27f8cf21-0f47-4397-831a-bf716741dc67\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-rr5k6"
Oct 31 00:38:19.805076 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:38:19.805046    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/27f8cf21-0f47-4397-831a-bf716741dc67-istio-token\") pod \"maas-default-gateway-openshift-default-6cc458d887-rr5k6\" (UID: \"27f8cf21-0f47-4397-831a-bf716741dc67\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-rr5k6"
Oct 31 00:38:19.805265 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:38:19.805245    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-slzp9\" (UniqueName: \"kubernetes.io/projected/27f8cf21-0f47-4397-831a-bf716741dc67-kube-api-access-slzp9\") pod \"maas-default-gateway-openshift-default-6cc458d887-rr5k6\" (UID: \"27f8cf21-0f47-4397-831a-bf716741dc67\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-rr5k6"
Oct 31 00:38:20.102093 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:38:20.102015    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-rr5k6"
Oct 31 00:38:20.520801 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:38:20.520779    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-rr5k6"]
Oct 31 00:38:20.522282 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:38:20.522260    4284 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod27f8cf21_0f47_4397_831a_bf716741dc67.slice/crio-0e1c9de255ad075e1e2b93bb23ee5c402a2286a895ec280c3a83f08725e8ad4d WatchSource:0}: Error finding container 0e1c9de255ad075e1e2b93bb23ee5c402a2286a895ec280c3a83f08725e8ad4d: Status 404 returned error can't find the container with id 0e1c9de255ad075e1e2b93bb23ee5c402a2286a895ec280c3a83f08725e8ad4d
Oct 31 00:38:20.524305 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:38:20.524288    4284 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 31 00:38:21.403076 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:38:21.403047    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-rr5k6" event={"ID":"27f8cf21-0f47-4397-831a-bf716741dc67","Type":"ContainerStarted","Data":"0e1c9de255ad075e1e2b93bb23ee5c402a2286a895ec280c3a83f08725e8ad4d"}
Oct 31 00:38:25.262154 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:38:25.262122    4284 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 00:38:25.262539 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:38:25.262187    4284 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 00:38:25.262539 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:38:25.262233    4284 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 00:38:25.423197 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:38:25.423162    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-rr5k6" event={"ID":"27f8cf21-0f47-4397-831a-bf716741dc67","Type":"ContainerStarted","Data":"35067cce7b522d2302a926596827c13d7fb4d828f291d22318b7fbcccaa5c162"}
Oct 31 00:38:25.588324 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:38:25.588199    4284 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-rr5k6" podStartSLOduration=1.851371307 podStartE2EDuration="6.588184974s" totalImagesPullingTime="4.736813667s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 00:38:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:38:25.586752064 +0000 UTC m=+619.731758992" watchObservedRunningTime="2026-10-31 00:38:25.588184974 +0000 UTC m=+619.733191889"
Oct 31 00:38:26.106048 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:38:26.105969    4284 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-rr5k6"
Oct 31 00:38:26.106799 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:38:26.106772    4284 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6cc458d887-rr5k6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" start-of-body=
Oct 31 00:38:26.107014 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:38:26.106990    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-rr5k6" podUID="27f8cf21-0f47-4397-831a-bf716741dc67" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused"
Oct 31 00:38:27.102605 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:38:27.102570    4284 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6cc458d887-rr5k6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" start-of-body=
Oct 31 00:38:27.103177 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:38:27.103147    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-rr5k6" podUID="27f8cf21-0f47-4397-831a-bf716741dc67" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused"
Oct 31 00:38:28.109371 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:38:28.109328    4284 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6cc458d887-rr5k6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" start-of-body=
Oct 31 00:38:28.109778 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:38:28.109402    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-rr5k6" podUID="27f8cf21-0f47-4397-831a-bf716741dc67" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused"
Oct 31 00:38:29.109252 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:38:29.105607    4284 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6cc458d887-rr5k6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" start-of-body=
Oct 31 00:38:29.109252 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:38:29.105671    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-rr5k6" podUID="27f8cf21-0f47-4397-831a-bf716741dc67" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused"
Oct 31 00:38:30.103655 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:38:30.103615    4284 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6cc458d887-rr5k6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" start-of-body=
Oct 31 00:38:30.103984 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:38:30.103954    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-rr5k6" podUID="27f8cf21-0f47-4397-831a-bf716741dc67" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused"
Oct 31 00:38:31.105592 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:38:31.105553    4284 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6cc458d887-rr5k6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" start-of-body=
Oct 31 00:38:31.106192 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:38:31.106161    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-rr5k6" podUID="27f8cf21-0f47-4397-831a-bf716741dc67" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused"
Oct 31 00:38:32.102675 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:38:32.102639    4284 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6cc458d887-rr5k6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" start-of-body=
Oct 31 00:38:32.102933 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:38:32.102695    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-rr5k6" podUID="27f8cf21-0f47-4397-831a-bf716741dc67" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused"
Oct 31 00:38:33.551049 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:38:33.551014    4284 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-rr5k6"
Oct 31 00:38:33.551562 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:38:33.551489    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-rr5k6"
Oct 31 00:38:33.552276 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:38:33.552227    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-rr5k6"
Oct 31 00:38:35.183939 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:38:35.183907    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:38:35.338035 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:38:35.337991    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:38:36.815055 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:38:36.815013    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:38:37.022504 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:38:37.022466    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:38:40.579226 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:38:40.579192    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:38:40.736441 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:38:40.736394    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:38:47.013974 ip-10-0-132-173 kubenswrapper[4284]: E1031 00:38:47.013923    4284 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fd8sp" containerName="extractor"
Oct 31 00:38:52.249225 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:38:52.249171    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:39:14.043416 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:14.043373    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:39:14.249020 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:14.248977    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:39:14.568456 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:14.568416    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:39:14.838037 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:14.837936    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:39:14.992050 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:14.992013    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:39:15.037352 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:15.037324    4284 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-d6a47bf7-openshift-default-6bd8b956bc-5lwbr"]
Oct 31 00:39:15.040810 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:15.040788    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-d6a47bf7-openshift-default-6bd8b956bc-5lwbr"
Oct 31 00:39:15.125197 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:15.125117    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-d6a47bf7-openshift-default-6bd8b956bc-5lwbr"]
Oct 31 00:39:15.139768 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:15.139736    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b5a5a3d3-2cfe-4853-bdc3-1401e2b055a2-istio-data\") pod \"e2e-ait-d6a47bf7-openshift-default-6bd8b956bc-5lwbr\" (UID: \"b5a5a3d3-2cfe-4853-bdc3-1401e2b055a2\") " pod="openshift-ingress/e2e-ait-d6a47bf7-openshift-default-6bd8b956bc-5lwbr"
Oct 31 00:39:15.139768 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:15.139770    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b5a5a3d3-2cfe-4853-bdc3-1401e2b055a2-credential-socket\") pod \"e2e-ait-d6a47bf7-openshift-default-6bd8b956bc-5lwbr\" (UID: \"b5a5a3d3-2cfe-4853-bdc3-1401e2b055a2\") " pod="openshift-ingress/e2e-ait-d6a47bf7-openshift-default-6bd8b956bc-5lwbr"
Oct 31 00:39:15.140004 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:15.139789    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b5a5a3d3-2cfe-4853-bdc3-1401e2b055a2-istio-token\") pod \"e2e-ait-d6a47bf7-openshift-default-6bd8b956bc-5lwbr\" (UID: \"b5a5a3d3-2cfe-4853-bdc3-1401e2b055a2\") " pod="openshift-ingress/e2e-ait-d6a47bf7-openshift-default-6bd8b956bc-5lwbr"
Oct 31 00:39:15.140004 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:15.139872    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b5a5a3d3-2cfe-4853-bdc3-1401e2b055a2-workload-certs\") pod \"e2e-ait-d6a47bf7-openshift-default-6bd8b956bc-5lwbr\" (UID: \"b5a5a3d3-2cfe-4853-bdc3-1401e2b055a2\") " pod="openshift-ingress/e2e-ait-d6a47bf7-openshift-default-6bd8b956bc-5lwbr"
Oct 31 00:39:15.140078 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:15.140024    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b5a5a3d3-2cfe-4853-bdc3-1401e2b055a2-workload-socket\") pod \"e2e-ait-d6a47bf7-openshift-default-6bd8b956bc-5lwbr\" (UID: \"b5a5a3d3-2cfe-4853-bdc3-1401e2b055a2\") " pod="openshift-ingress/e2e-ait-d6a47bf7-openshift-default-6bd8b956bc-5lwbr"
Oct 31 00:39:15.140116 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:15.140073    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pcvtg\" (UniqueName: \"kubernetes.io/projected/b5a5a3d3-2cfe-4853-bdc3-1401e2b055a2-kube-api-access-pcvtg\") pod \"e2e-ait-d6a47bf7-openshift-default-6bd8b956bc-5lwbr\" (UID: \"b5a5a3d3-2cfe-4853-bdc3-1401e2b055a2\") " pod="openshift-ingress/e2e-ait-d6a47bf7-openshift-default-6bd8b956bc-5lwbr"
Oct 31 00:39:15.140164 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:15.140123    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b5a5a3d3-2cfe-4853-bdc3-1401e2b055a2-istio-podinfo\") pod \"e2e-ait-d6a47bf7-openshift-default-6bd8b956bc-5lwbr\" (UID: \"b5a5a3d3-2cfe-4853-bdc3-1401e2b055a2\") " pod="openshift-ingress/e2e-ait-d6a47bf7-openshift-default-6bd8b956bc-5lwbr"
Oct 31 00:39:15.140164 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:15.140145    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b5a5a3d3-2cfe-4853-bdc3-1401e2b055a2-istio-envoy\") pod \"e2e-ait-d6a47bf7-openshift-default-6bd8b956bc-5lwbr\" (UID: \"b5a5a3d3-2cfe-4853-bdc3-1401e2b055a2\") " pod="openshift-ingress/e2e-ait-d6a47bf7-openshift-default-6bd8b956bc-5lwbr"
Oct 31 00:39:15.140273 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:15.140168    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b5a5a3d3-2cfe-4853-bdc3-1401e2b055a2-istiod-ca-cert\") pod \"e2e-ait-d6a47bf7-openshift-default-6bd8b956bc-5lwbr\" (UID: \"b5a5a3d3-2cfe-4853-bdc3-1401e2b055a2\") " pod="openshift-ingress/e2e-ait-d6a47bf7-openshift-default-6bd8b956bc-5lwbr"
Oct 31 00:39:15.144324 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:15.144293    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:39:15.147276 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:15.147257    4284 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-d6a47bf7-openshift-default-dockercfg-x2cff\""
Oct 31 00:39:15.241602 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:15.241568    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b5a5a3d3-2cfe-4853-bdc3-1401e2b055a2-workload-socket\") pod \"e2e-ait-d6a47bf7-openshift-default-6bd8b956bc-5lwbr\" (UID: \"b5a5a3d3-2cfe-4853-bdc3-1401e2b055a2\") " pod="openshift-ingress/e2e-ait-d6a47bf7-openshift-default-6bd8b956bc-5lwbr"
Oct 31 00:39:15.241785 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:15.241617    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pcvtg\" (UniqueName: \"kubernetes.io/projected/b5a5a3d3-2cfe-4853-bdc3-1401e2b055a2-kube-api-access-pcvtg\") pod \"e2e-ait-d6a47bf7-openshift-default-6bd8b956bc-5lwbr\" (UID: \"b5a5a3d3-2cfe-4853-bdc3-1401e2b055a2\") " pod="openshift-ingress/e2e-ait-d6a47bf7-openshift-default-6bd8b956bc-5lwbr"
Oct 31 00:39:15.241785 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:15.241639    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b5a5a3d3-2cfe-4853-bdc3-1401e2b055a2-istio-podinfo\") pod \"e2e-ait-d6a47bf7-openshift-default-6bd8b956bc-5lwbr\" (UID: \"b5a5a3d3-2cfe-4853-bdc3-1401e2b055a2\") " pod="openshift-ingress/e2e-ait-d6a47bf7-openshift-default-6bd8b956bc-5lwbr"
Oct 31 00:39:15.241785 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:15.241657    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b5a5a3d3-2cfe-4853-bdc3-1401e2b055a2-istio-envoy\") pod \"e2e-ait-d6a47bf7-openshift-default-6bd8b956bc-5lwbr\" (UID: \"b5a5a3d3-2cfe-4853-bdc3-1401e2b055a2\") " pod="openshift-ingress/e2e-ait-d6a47bf7-openshift-default-6bd8b956bc-5lwbr"
Oct 31 00:39:15.241785 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:15.241676    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b5a5a3d3-2cfe-4853-bdc3-1401e2b055a2-istiod-ca-cert\") pod \"e2e-ait-d6a47bf7-openshift-default-6bd8b956bc-5lwbr\" (UID: \"b5a5a3d3-2cfe-4853-bdc3-1401e2b055a2\") " pod="openshift-ingress/e2e-ait-d6a47bf7-openshift-default-6bd8b956bc-5lwbr"
Oct 31 00:39:15.241785 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:15.241711    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b5a5a3d3-2cfe-4853-bdc3-1401e2b055a2-istio-data\") pod \"e2e-ait-d6a47bf7-openshift-default-6bd8b956bc-5lwbr\" (UID: \"b5a5a3d3-2cfe-4853-bdc3-1401e2b055a2\") " pod="openshift-ingress/e2e-ait-d6a47bf7-openshift-default-6bd8b956bc-5lwbr"
Oct 31 00:39:15.242113 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:15.241856    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b5a5a3d3-2cfe-4853-bdc3-1401e2b055a2-credential-socket\") pod \"e2e-ait-d6a47bf7-openshift-default-6bd8b956bc-5lwbr\" (UID: \"b5a5a3d3-2cfe-4853-bdc3-1401e2b055a2\") " pod="openshift-ingress/e2e-ait-d6a47bf7-openshift-default-6bd8b956bc-5lwbr"
Oct 31 00:39:15.242113 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:15.241903    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b5a5a3d3-2cfe-4853-bdc3-1401e2b055a2-istio-token\") pod \"e2e-ait-d6a47bf7-openshift-default-6bd8b956bc-5lwbr\" (UID: \"b5a5a3d3-2cfe-4853-bdc3-1401e2b055a2\") " pod="openshift-ingress/e2e-ait-d6a47bf7-openshift-default-6bd8b956bc-5lwbr"
Oct 31 00:39:15.242113 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:15.241942    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b5a5a3d3-2cfe-4853-bdc3-1401e2b055a2-workload-certs\") pod \"e2e-ait-d6a47bf7-openshift-default-6bd8b956bc-5lwbr\" (UID: \"b5a5a3d3-2cfe-4853-bdc3-1401e2b055a2\") " pod="openshift-ingress/e2e-ait-d6a47bf7-openshift-default-6bd8b956bc-5lwbr"
Oct 31 00:39:15.242113 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:15.242007    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b5a5a3d3-2cfe-4853-bdc3-1401e2b055a2-workload-socket\") pod \"e2e-ait-d6a47bf7-openshift-default-6bd8b956bc-5lwbr\" (UID: \"b5a5a3d3-2cfe-4853-bdc3-1401e2b055a2\") " pod="openshift-ingress/e2e-ait-d6a47bf7-openshift-default-6bd8b956bc-5lwbr"
Oct 31 00:39:15.242346 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:15.242277    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b5a5a3d3-2cfe-4853-bdc3-1401e2b055a2-credential-socket\") pod \"e2e-ait-d6a47bf7-openshift-default-6bd8b956bc-5lwbr\" (UID: \"b5a5a3d3-2cfe-4853-bdc3-1401e2b055a2\") " pod="openshift-ingress/e2e-ait-d6a47bf7-openshift-default-6bd8b956bc-5lwbr"
Oct 31 00:39:15.242383 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:15.242358    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b5a5a3d3-2cfe-4853-bdc3-1401e2b055a2-workload-certs\") pod \"e2e-ait-d6a47bf7-openshift-default-6bd8b956bc-5lwbr\" (UID: \"b5a5a3d3-2cfe-4853-bdc3-1401e2b055a2\") " pod="openshift-ingress/e2e-ait-d6a47bf7-openshift-default-6bd8b956bc-5lwbr"
Oct 31 00:39:15.242518 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:15.242494    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b5a5a3d3-2cfe-4853-bdc3-1401e2b055a2-istio-data\") pod \"e2e-ait-d6a47bf7-openshift-default-6bd8b956bc-5lwbr\" (UID: \"b5a5a3d3-2cfe-4853-bdc3-1401e2b055a2\") " pod="openshift-ingress/e2e-ait-d6a47bf7-openshift-default-6bd8b956bc-5lwbr"
Oct 31 00:39:15.242637 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:15.242617    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b5a5a3d3-2cfe-4853-bdc3-1401e2b055a2-istiod-ca-cert\") pod \"e2e-ait-d6a47bf7-openshift-default-6bd8b956bc-5lwbr\" (UID: \"b5a5a3d3-2cfe-4853-bdc3-1401e2b055a2\") " pod="openshift-ingress/e2e-ait-d6a47bf7-openshift-default-6bd8b956bc-5lwbr"
Oct 31 00:39:15.244286 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:15.244267    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b5a5a3d3-2cfe-4853-bdc3-1401e2b055a2-istio-podinfo\") pod \"e2e-ait-d6a47bf7-openshift-default-6bd8b956bc-5lwbr\" (UID: \"b5a5a3d3-2cfe-4853-bdc3-1401e2b055a2\") " pod="openshift-ingress/e2e-ait-d6a47bf7-openshift-default-6bd8b956bc-5lwbr"
Oct 31 00:39:15.244473 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:15.244456    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b5a5a3d3-2cfe-4853-bdc3-1401e2b055a2-istio-envoy\") pod \"e2e-ait-d6a47bf7-openshift-default-6bd8b956bc-5lwbr\" (UID: \"b5a5a3d3-2cfe-4853-bdc3-1401e2b055a2\") " pod="openshift-ingress/e2e-ait-d6a47bf7-openshift-default-6bd8b956bc-5lwbr"
Oct 31 00:39:15.300491 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:15.300458    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pcvtg\" (UniqueName: \"kubernetes.io/projected/b5a5a3d3-2cfe-4853-bdc3-1401e2b055a2-kube-api-access-pcvtg\") pod \"e2e-ait-d6a47bf7-openshift-default-6bd8b956bc-5lwbr\" (UID: \"b5a5a3d3-2cfe-4853-bdc3-1401e2b055a2\") " pod="openshift-ingress/e2e-ait-d6a47bf7-openshift-default-6bd8b956bc-5lwbr"
Oct 31 00:39:15.300638 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:15.300510    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b5a5a3d3-2cfe-4853-bdc3-1401e2b055a2-istio-token\") pod \"e2e-ait-d6a47bf7-openshift-default-6bd8b956bc-5lwbr\" (UID: \"b5a5a3d3-2cfe-4853-bdc3-1401e2b055a2\") " pod="openshift-ingress/e2e-ait-d6a47bf7-openshift-default-6bd8b956bc-5lwbr"
Oct 31 00:39:15.352927 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:15.352896    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-d6a47bf7-openshift-default-6bd8b956bc-5lwbr"
Oct 31 00:39:15.564641 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:15.564603    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:39:15.767150 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:15.767112    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:39:15.769537 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:15.769517    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-d6a47bf7-openshift-default-6bd8b956bc-5lwbr"]
Oct 31 00:39:15.770790 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:39:15.770771    4284 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb5a5a3d3_2cfe_4853_bdc3_1401e2b055a2.slice/crio-da8e4e2506ba5a72dc7c1644d1f7841f42d493ca0b16392f71adefe2c9bc13cf WatchSource:0}: Error finding container da8e4e2506ba5a72dc7c1644d1f7841f42d493ca0b16392f71adefe2c9bc13cf: Status 404 returned error can't find the container with id da8e4e2506ba5a72dc7c1644d1f7841f42d493ca0b16392f71adefe2c9bc13cf
Oct 31 00:39:15.772870 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:15.772844    4284 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 00:39:15.772923 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:15.772905    4284 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 00:39:15.772958 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:15.772931    4284 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 00:39:16.029498 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:16.029415    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:39:16.176384 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:16.176347    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:39:16.655268 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:16.655168    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-d6a47bf7-openshift-default-6bd8b956bc-5lwbr" event={"ID":"b5a5a3d3-2cfe-4853-bdc3-1401e2b055a2","Type":"ContainerStarted","Data":"e3d1ad0398c08cac241da7abc1c6d8c913f1ea1c8725dea9128d481196aba8e3"}
Oct 31 00:39:16.655268 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:16.655260    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-d6a47bf7-openshift-default-6bd8b956bc-5lwbr" event={"ID":"b5a5a3d3-2cfe-4853-bdc3-1401e2b055a2","Type":"ContainerStarted","Data":"da8e4e2506ba5a72dc7c1644d1f7841f42d493ca0b16392f71adefe2c9bc13cf"}
Oct 31 00:39:16.819124 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:16.819069    4284 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-d6a47bf7-openshift-default-6bd8b956bc-5lwbr" podStartSLOduration=2.8190550869999997 podStartE2EDuration="2.819055087s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 00:39:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:39:16.818859236 +0000 UTC m=+670.963866152" watchObservedRunningTime="2026-10-31 00:39:16.819055087 +0000 UTC m=+670.964062003"
Oct 31 00:39:17.014122 ip-10-0-132-173 kubenswrapper[4284]: E1031 00:39:17.014034    4284 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fd8sp" containerName="extractor"
Oct 31 00:39:17.140332 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:17.140301    4284 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-95fa4445-openshift-default-c9bd77677-wz7qd"]
Oct 31 00:39:17.143952 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:17.143931    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-95fa4445-openshift-default-c9bd77677-wz7qd"
Oct 31 00:39:17.194355 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:17.194323    4284 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-95fa4445-openshift-default-dockercfg-4bpdg\""
Oct 31 00:39:17.236055 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:17.236022    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:39:17.243344 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:17.243320    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-95fa4445-openshift-default-c9bd77677-wz7qd"]
Oct 31 00:39:17.261688 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:17.261646    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/06987c79-2521-487f-a60d-3050698f9719-istio-data\") pod \"e2e-ait-95fa4445-openshift-default-c9bd77677-wz7qd\" (UID: \"06987c79-2521-487f-a60d-3050698f9719\") " pod="openshift-ingress/e2e-ait-95fa4445-openshift-default-c9bd77677-wz7qd"
Oct 31 00:39:17.261861 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:17.261702    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/06987c79-2521-487f-a60d-3050698f9719-istiod-ca-cert\") pod \"e2e-ait-95fa4445-openshift-default-c9bd77677-wz7qd\" (UID: \"06987c79-2521-487f-a60d-3050698f9719\") " pod="openshift-ingress/e2e-ait-95fa4445-openshift-default-c9bd77677-wz7qd"
Oct 31 00:39:17.261861 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:17.261774    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/06987c79-2521-487f-a60d-3050698f9719-istio-envoy\") pod \"e2e-ait-95fa4445-openshift-default-c9bd77677-wz7qd\" (UID: \"06987c79-2521-487f-a60d-3050698f9719\") " pod="openshift-ingress/e2e-ait-95fa4445-openshift-default-c9bd77677-wz7qd"
Oct 31 00:39:17.261861 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:17.261825    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/06987c79-2521-487f-a60d-3050698f9719-istio-token\") pod \"e2e-ait-95fa4445-openshift-default-c9bd77677-wz7qd\" (UID: \"06987c79-2521-487f-a60d-3050698f9719\") " pod="openshift-ingress/e2e-ait-95fa4445-openshift-default-c9bd77677-wz7qd"
Oct 31 00:39:17.262027 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:17.261903    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/06987c79-2521-487f-a60d-3050698f9719-credential-socket\") pod \"e2e-ait-95fa4445-openshift-default-c9bd77677-wz7qd\" (UID: \"06987c79-2521-487f-a60d-3050698f9719\") " pod="openshift-ingress/e2e-ait-95fa4445-openshift-default-c9bd77677-wz7qd"
Oct 31 00:39:17.262027 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:17.261931    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7vr8w\" (UniqueName: \"kubernetes.io/projected/06987c79-2521-487f-a60d-3050698f9719-kube-api-access-7vr8w\") pod \"e2e-ait-95fa4445-openshift-default-c9bd77677-wz7qd\" (UID: \"06987c79-2521-487f-a60d-3050698f9719\") " pod="openshift-ingress/e2e-ait-95fa4445-openshift-default-c9bd77677-wz7qd"
Oct 31 00:39:17.262027 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:17.262003    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/06987c79-2521-487f-a60d-3050698f9719-workload-socket\") pod \"e2e-ait-95fa4445-openshift-default-c9bd77677-wz7qd\" (UID: \"06987c79-2521-487f-a60d-3050698f9719\") " pod="openshift-ingress/e2e-ait-95fa4445-openshift-default-c9bd77677-wz7qd"
Oct 31 00:39:17.262199 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:17.262035    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/06987c79-2521-487f-a60d-3050698f9719-workload-certs\") pod \"e2e-ait-95fa4445-openshift-default-c9bd77677-wz7qd\" (UID: \"06987c79-2521-487f-a60d-3050698f9719\") " pod="openshift-ingress/e2e-ait-95fa4445-openshift-default-c9bd77677-wz7qd"
Oct 31 00:39:17.262199 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:17.262072    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/06987c79-2521-487f-a60d-3050698f9719-istio-podinfo\") pod \"e2e-ait-95fa4445-openshift-default-c9bd77677-wz7qd\" (UID: \"06987c79-2521-487f-a60d-3050698f9719\") " pod="openshift-ingress/e2e-ait-95fa4445-openshift-default-c9bd77677-wz7qd"
Oct 31 00:39:17.353526 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:17.353430    4284 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-d6a47bf7-openshift-default-6bd8b956bc-5lwbr"
Oct 31 00:39:17.358909 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:17.358885    4284 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-d6a47bf7-openshift-default-6bd8b956bc-5lwbr"
Oct 31 00:39:17.363278 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:17.363253    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/06987c79-2521-487f-a60d-3050698f9719-workload-socket\") pod \"e2e-ait-95fa4445-openshift-default-c9bd77677-wz7qd\" (UID: \"06987c79-2521-487f-a60d-3050698f9719\") " pod="openshift-ingress/e2e-ait-95fa4445-openshift-default-c9bd77677-wz7qd"
Oct 31 00:39:17.363404 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:17.363302    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/06987c79-2521-487f-a60d-3050698f9719-workload-certs\") pod \"e2e-ait-95fa4445-openshift-default-c9bd77677-wz7qd\" (UID: \"06987c79-2521-487f-a60d-3050698f9719\") " pod="openshift-ingress/e2e-ait-95fa4445-openshift-default-c9bd77677-wz7qd"
Oct 31 00:39:17.363404 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:17.363327    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/06987c79-2521-487f-a60d-3050698f9719-istio-podinfo\") pod \"e2e-ait-95fa4445-openshift-default-c9bd77677-wz7qd\" (UID: \"06987c79-2521-487f-a60d-3050698f9719\") " pod="openshift-ingress/e2e-ait-95fa4445-openshift-default-c9bd77677-wz7qd"
Oct 31 00:39:17.363404 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:17.363371    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/06987c79-2521-487f-a60d-3050698f9719-istio-data\") pod \"e2e-ait-95fa4445-openshift-default-c9bd77677-wz7qd\" (UID: \"06987c79-2521-487f-a60d-3050698f9719\") " pod="openshift-ingress/e2e-ait-95fa4445-openshift-default-c9bd77677-wz7qd"
Oct 31 00:39:17.363576 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:17.363413    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/06987c79-2521-487f-a60d-3050698f9719-istiod-ca-cert\") pod \"e2e-ait-95fa4445-openshift-default-c9bd77677-wz7qd\" (UID: \"06987c79-2521-487f-a60d-3050698f9719\") " pod="openshift-ingress/e2e-ait-95fa4445-openshift-default-c9bd77677-wz7qd"
Oct 31 00:39:17.363576 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:17.363438    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/06987c79-2521-487f-a60d-3050698f9719-istio-envoy\") pod \"e2e-ait-95fa4445-openshift-default-c9bd77677-wz7qd\" (UID: \"06987c79-2521-487f-a60d-3050698f9719\") " pod="openshift-ingress/e2e-ait-95fa4445-openshift-default-c9bd77677-wz7qd"
Oct 31 00:39:17.363576 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:17.363466    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/06987c79-2521-487f-a60d-3050698f9719-istio-token\") pod \"e2e-ait-95fa4445-openshift-default-c9bd77677-wz7qd\" (UID: \"06987c79-2521-487f-a60d-3050698f9719\") " pod="openshift-ingress/e2e-ait-95fa4445-openshift-default-c9bd77677-wz7qd"
Oct 31 00:39:17.363576 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:17.363541    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/06987c79-2521-487f-a60d-3050698f9719-credential-socket\") pod \"e2e-ait-95fa4445-openshift-default-c9bd77677-wz7qd\" (UID: \"06987c79-2521-487f-a60d-3050698f9719\") " pod="openshift-ingress/e2e-ait-95fa4445-openshift-default-c9bd77677-wz7qd"
Oct 31 00:39:17.363768 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:17.363581    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7vr8w\" (UniqueName: \"kubernetes.io/projected/06987c79-2521-487f-a60d-3050698f9719-kube-api-access-7vr8w\") pod \"e2e-ait-95fa4445-openshift-default-c9bd77677-wz7qd\" (UID: \"06987c79-2521-487f-a60d-3050698f9719\") " pod="openshift-ingress/e2e-ait-95fa4445-openshift-default-c9bd77677-wz7qd"
Oct 31 00:39:17.363768 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:17.363694    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/06987c79-2521-487f-a60d-3050698f9719-workload-socket\") pod \"e2e-ait-95fa4445-openshift-default-c9bd77677-wz7qd\" (UID: \"06987c79-2521-487f-a60d-3050698f9719\") " pod="openshift-ingress/e2e-ait-95fa4445-openshift-default-c9bd77677-wz7qd"
Oct 31 00:39:17.363869 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:17.363760    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/06987c79-2521-487f-a60d-3050698f9719-istio-data\") pod \"e2e-ait-95fa4445-openshift-default-c9bd77677-wz7qd\" (UID: \"06987c79-2521-487f-a60d-3050698f9719\") " pod="openshift-ingress/e2e-ait-95fa4445-openshift-default-c9bd77677-wz7qd"
Oct 31 00:39:17.363948 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:17.363927    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/06987c79-2521-487f-a60d-3050698f9719-credential-socket\") pod \"e2e-ait-95fa4445-openshift-default-c9bd77677-wz7qd\" (UID: \"06987c79-2521-487f-a60d-3050698f9719\") " pod="openshift-ingress/e2e-ait-95fa4445-openshift-default-c9bd77677-wz7qd"
Oct 31 00:39:17.364051 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:17.364033    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/06987c79-2521-487f-a60d-3050698f9719-workload-certs\") pod \"e2e-ait-95fa4445-openshift-default-c9bd77677-wz7qd\" (UID: \"06987c79-2521-487f-a60d-3050698f9719\") " pod="openshift-ingress/e2e-ait-95fa4445-openshift-default-c9bd77677-wz7qd"
Oct 31 00:39:17.364204 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:17.364185    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/06987c79-2521-487f-a60d-3050698f9719-istiod-ca-cert\") pod \"e2e-ait-95fa4445-openshift-default-c9bd77677-wz7qd\" (UID: \"06987c79-2521-487f-a60d-3050698f9719\") " pod="openshift-ingress/e2e-ait-95fa4445-openshift-default-c9bd77677-wz7qd"
Oct 31 00:39:17.365850 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:17.365833    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/06987c79-2521-487f-a60d-3050698f9719-istio-envoy\") pod \"e2e-ait-95fa4445-openshift-default-c9bd77677-wz7qd\" (UID: \"06987c79-2521-487f-a60d-3050698f9719\") " pod="openshift-ingress/e2e-ait-95fa4445-openshift-default-c9bd77677-wz7qd"
Oct 31 00:39:17.366058 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:17.366042    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/06987c79-2521-487f-a60d-3050698f9719-istio-podinfo\") pod \"e2e-ait-95fa4445-openshift-default-c9bd77677-wz7qd\" (UID: \"06987c79-2521-487f-a60d-3050698f9719\") " pod="openshift-ingress/e2e-ait-95fa4445-openshift-default-c9bd77677-wz7qd"
Oct 31 00:39:17.422574 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:17.422545    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/06987c79-2521-487f-a60d-3050698f9719-istio-token\") pod \"e2e-ait-95fa4445-openshift-default-c9bd77677-wz7qd\" (UID: \"06987c79-2521-487f-a60d-3050698f9719\") " pod="openshift-ingress/e2e-ait-95fa4445-openshift-default-c9bd77677-wz7qd"
Oct 31 00:39:17.422717 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:17.422678    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7vr8w\" (UniqueName: \"kubernetes.io/projected/06987c79-2521-487f-a60d-3050698f9719-kube-api-access-7vr8w\") pod \"e2e-ait-95fa4445-openshift-default-c9bd77677-wz7qd\" (UID: \"06987c79-2521-487f-a60d-3050698f9719\") " pod="openshift-ingress/e2e-ait-95fa4445-openshift-default-c9bd77677-wz7qd"
Oct 31 00:39:17.456928 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:17.456483    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-95fa4445-openshift-default-c9bd77677-wz7qd"
Oct 31 00:39:17.603181 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:17.603142    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:39:17.659280 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:17.659251    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-d6a47bf7-openshift-default-6bd8b956bc-5lwbr"
Oct 31 00:39:17.660374 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:17.660355    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-d6a47bf7-openshift-default-6bd8b956bc-5lwbr"
Oct 31 00:39:17.875201 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:17.875178    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-95fa4445-openshift-default-c9bd77677-wz7qd"]
Oct 31 00:39:17.877103 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:39:17.877076    4284 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod06987c79_2521_487f_a60d_3050698f9719.slice/crio-3f5b227ddc44e4d90770709a5a0fd851c4ea6011b4f9b3370f580ae290dcc3a6 WatchSource:0}: Error finding container 3f5b227ddc44e4d90770709a5a0fd851c4ea6011b4f9b3370f580ae290dcc3a6: Status 404 returned error can't find the container with id 3f5b227ddc44e4d90770709a5a0fd851c4ea6011b4f9b3370f580ae290dcc3a6
Oct 31 00:39:17.879328 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:17.879299    4284 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 00:39:17.879425 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:17.879375    4284 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 00:39:17.879505 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:17.879419    4284 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 00:39:18.517358 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:18.517318    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:39:18.665011 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:18.664970    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-95fa4445-openshift-default-c9bd77677-wz7qd" event={"ID":"06987c79-2521-487f-a60d-3050698f9719","Type":"ContainerStarted","Data":"5078f568d9db91b92e652db9f85003d72d143a1c54dbf308c657860aac539164"}
Oct 31 00:39:18.665228 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:18.665036    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-95fa4445-openshift-default-c9bd77677-wz7qd" event={"ID":"06987c79-2521-487f-a60d-3050698f9719","Type":"ContainerStarted","Data":"3f5b227ddc44e4d90770709a5a0fd851c4ea6011b4f9b3370f580ae290dcc3a6"}
Oct 31 00:39:18.721881 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:18.721852    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:39:18.781174 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:18.781081    4284 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-95fa4445-openshift-default-c9bd77677-wz7qd" podStartSLOduration=1.781067449 podStartE2EDuration="1.781067449s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 00:39:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:39:18.780114876 +0000 UTC m=+672.925121795" watchObservedRunningTime="2026-10-31 00:39:18.781067449 +0000 UTC m=+672.926074434"
Oct 31 00:39:19.115875 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:19.115788    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:39:19.274200 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:19.274160    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:39:19.415826 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:19.415788    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:39:19.457798 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:19.457767    4284 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-95fa4445-openshift-default-c9bd77677-wz7qd"
Oct 31 00:39:19.463088 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:19.463065    4284 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-95fa4445-openshift-default-c9bd77677-wz7qd"
Oct 31 00:39:19.669712 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:19.669633    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-95fa4445-openshift-default-c9bd77677-wz7qd"
Oct 31 00:39:19.670703 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:19.670686    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-95fa4445-openshift-default-c9bd77677-wz7qd"
Oct 31 00:39:20.138513 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:20.138418    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:39:20.339100 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:20.339049    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:39:21.556435 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:21.556401    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:39:21.709071 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:21.709032    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:39:21.866163 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:21.866083    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:39:22.453779 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:22.453733    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:39:22.609761 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:22.609722    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:39:22.757169 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:22.757074    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:39:22.855923 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:22.855890    4284 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-30d0c8aa-7959895f-lrjn5"]
Oct 31 00:39:22.859422 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:22.859406    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-30d0c8aa-7959895f-lrjn5"
Oct 31 00:39:22.928796 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:22.928764    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h9wfn\" (UniqueName: \"kubernetes.io/projected/f2147b56-458f-4d40-bbbe-75e90a4c1093-kube-api-access-h9wfn\") pod \"payload-pre-processing-e2e-migrate-30d0c8aa-7959895f-lrjn5\" (UID: \"f2147b56-458f-4d40-bbbe-75e90a4c1093\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-30d0c8aa-7959895f-lrjn5"
Oct 31 00:39:22.928985 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:22.928826    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f2147b56-458f-4d40-bbbe-75e90a4c1093-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-30d0c8aa-7959895f-lrjn5\" (UID: \"f2147b56-458f-4d40-bbbe-75e90a4c1093\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-30d0c8aa-7959895f-lrjn5"
Oct 31 00:39:22.965134 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:22.965097    4284 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-migrate-30d0c8aa-dockercfg-tv4pj\""
Oct 31 00:39:22.965393 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:22.965135    4284 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-migrate-30d0c8aa\""
Oct 31 00:39:22.967008 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:22.966984    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-30d0c8aa-7959895f-lrjn5"]
Oct 31 00:39:23.014854 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:23.014778    4284 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-30d0c8aa-dfc88b4b5-qfvz7"]
Oct 31 00:39:23.018071 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:23.018056    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-30d0c8aa-dfc88b4b5-qfvz7"
Oct 31 00:39:23.029759 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:23.029528    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f2147b56-458f-4d40-bbbe-75e90a4c1093-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-30d0c8aa-7959895f-lrjn5\" (UID: \"f2147b56-458f-4d40-bbbe-75e90a4c1093\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-30d0c8aa-7959895f-lrjn5"
Oct 31 00:39:23.029759 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:23.029570    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9fb753b3-95a2-41cf-b736-8244616735a1-plugins-config-volume\") pod \"payload-processing-e2e-migrate-30d0c8aa-dfc88b4b5-qfvz7\" (UID: \"9fb753b3-95a2-41cf-b736-8244616735a1\") " pod="openshift-ingress/payload-processing-e2e-migrate-30d0c8aa-dfc88b4b5-qfvz7"
Oct 31 00:39:23.029759 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:23.029669    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z2mf4\" (UniqueName: \"kubernetes.io/projected/9fb753b3-95a2-41cf-b736-8244616735a1-kube-api-access-z2mf4\") pod \"payload-processing-e2e-migrate-30d0c8aa-dfc88b4b5-qfvz7\" (UID: \"9fb753b3-95a2-41cf-b736-8244616735a1\") " pod="openshift-ingress/payload-processing-e2e-migrate-30d0c8aa-dfc88b4b5-qfvz7"
Oct 31 00:39:23.029759 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:23.029703    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h9wfn\" (UniqueName: \"kubernetes.io/projected/f2147b56-458f-4d40-bbbe-75e90a4c1093-kube-api-access-h9wfn\") pod \"payload-pre-processing-e2e-migrate-30d0c8aa-7959895f-lrjn5\" (UID: \"f2147b56-458f-4d40-bbbe-75e90a4c1093\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-30d0c8aa-7959895f-lrjn5"
Oct 31 00:39:23.030184 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:23.030166    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f2147b56-458f-4d40-bbbe-75e90a4c1093-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-30d0c8aa-7959895f-lrjn5\" (UID: \"f2147b56-458f-4d40-bbbe-75e90a4c1093\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-30d0c8aa-7959895f-lrjn5"
Oct 31 00:39:23.064105 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:23.064070    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-30d0c8aa-dfc88b4b5-qfvz7"]
Oct 31 00:39:23.093676 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:23.093640    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h9wfn\" (UniqueName: \"kubernetes.io/projected/f2147b56-458f-4d40-bbbe-75e90a4c1093-kube-api-access-h9wfn\") pod \"payload-pre-processing-e2e-migrate-30d0c8aa-7959895f-lrjn5\" (UID: \"f2147b56-458f-4d40-bbbe-75e90a4c1093\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-30d0c8aa-7959895f-lrjn5"
Oct 31 00:39:23.131351 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:23.131311    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z2mf4\" (UniqueName: \"kubernetes.io/projected/9fb753b3-95a2-41cf-b736-8244616735a1-kube-api-access-z2mf4\") pod \"payload-processing-e2e-migrate-30d0c8aa-dfc88b4b5-qfvz7\" (UID: \"9fb753b3-95a2-41cf-b736-8244616735a1\") " pod="openshift-ingress/payload-processing-e2e-migrate-30d0c8aa-dfc88b4b5-qfvz7"
Oct 31 00:39:23.131526 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:23.131400    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9fb753b3-95a2-41cf-b736-8244616735a1-plugins-config-volume\") pod \"payload-processing-e2e-migrate-30d0c8aa-dfc88b4b5-qfvz7\" (UID: \"9fb753b3-95a2-41cf-b736-8244616735a1\") " pod="openshift-ingress/payload-processing-e2e-migrate-30d0c8aa-dfc88b4b5-qfvz7"
Oct 31 00:39:23.132044 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:23.132027    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9fb753b3-95a2-41cf-b736-8244616735a1-plugins-config-volume\") pod \"payload-processing-e2e-migrate-30d0c8aa-dfc88b4b5-qfvz7\" (UID: \"9fb753b3-95a2-41cf-b736-8244616735a1\") " pod="openshift-ingress/payload-processing-e2e-migrate-30d0c8aa-dfc88b4b5-qfvz7"
Oct 31 00:39:23.172474 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:23.172436    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-30d0c8aa-7959895f-lrjn5"
Oct 31 00:39:23.190967 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:23.190942    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z2mf4\" (UniqueName: \"kubernetes.io/projected/9fb753b3-95a2-41cf-b736-8244616735a1-kube-api-access-z2mf4\") pod \"payload-processing-e2e-migrate-30d0c8aa-dfc88b4b5-qfvz7\" (UID: \"9fb753b3-95a2-41cf-b736-8244616735a1\") " pod="openshift-ingress/payload-processing-e2e-migrate-30d0c8aa-dfc88b4b5-qfvz7"
Oct 31 00:39:23.330795 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:23.330709    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-30d0c8aa-dfc88b4b5-qfvz7"
Oct 31 00:39:23.553450 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:23.553422    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-30d0c8aa-7959895f-lrjn5"]
Oct 31 00:39:23.555265 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:39:23.555236    4284 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf2147b56_458f_4d40_bbbe_75e90a4c1093.slice/crio-ecbd32ba94916788e8284d88123c2a910e9791bb785e29cc2c2916107808b21a WatchSource:0}: Error finding container ecbd32ba94916788e8284d88123c2a910e9791bb785e29cc2c2916107808b21a: Status 404 returned error can't find the container with id ecbd32ba94916788e8284d88123c2a910e9791bb785e29cc2c2916107808b21a
Oct 31 00:39:23.689927 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:23.689892    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-30d0c8aa-7959895f-lrjn5" event={"ID":"f2147b56-458f-4d40-bbbe-75e90a4c1093","Type":"ContainerStarted","Data":"19749f9e2c162e83539f918b4bb1c624d84759e64a11721f78fa83ed23f25ace"}
Oct 31 00:39:23.690329 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:23.689947    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-30d0c8aa-7959895f-lrjn5" event={"ID":"f2147b56-458f-4d40-bbbe-75e90a4c1093","Type":"ContainerStarted","Data":"ecbd32ba94916788e8284d88123c2a910e9791bb785e29cc2c2916107808b21a"}
Oct 31 00:39:23.690329 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:23.689976    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-30d0c8aa-7959895f-lrjn5"
Oct 31 00:39:23.707430 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:23.707408    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-30d0c8aa-dfc88b4b5-qfvz7"]
Oct 31 00:39:23.709053 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:39:23.709028    4284 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9fb753b3_95a2_41cf_b736_8244616735a1.slice/crio-db814b67f160c13b38fe09449b3871fb21a08eef91ff667136898e698fde595a WatchSource:0}: Error finding container db814b67f160c13b38fe09449b3871fb21a08eef91ff667136898e698fde595a: Status 404 returned error can't find the container with id db814b67f160c13b38fe09449b3871fb21a08eef91ff667136898e698fde595a
Oct 31 00:39:23.803402 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:23.803346    4284 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-migrate-30d0c8aa-7959895f-lrjn5" podStartSLOduration=1.803332538 podStartE2EDuration="1.803332538s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 00:39:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:39:23.801607803 +0000 UTC m=+677.946614719" watchObservedRunningTime="2026-10-31 00:39:23.803332538 +0000 UTC m=+677.948339454"
Oct 31 00:39:24.330631 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:24.330599    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:39:24.536062 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:24.536018    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:39:24.566164 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:24.566134    4284 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-d6a47bf7-94654b468-rsmhv"]
Oct 31 00:39:24.569926 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:24.569904    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d6a47bf7-94654b468-rsmhv"
Oct 31 00:39:24.619907 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:24.619840    4284 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-d6a47bf7-serving-cert\""
Oct 31 00:39:24.643861 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:24.643830    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4d6xg\" (UniqueName: \"kubernetes.io/projected/5f7bb018-65c5-4f84-ac0b-d6d303cbbb22-kube-api-access-4d6xg\") pod \"maas-api-e2e-ait-d6a47bf7-94654b468-rsmhv\" (UID: \"5f7bb018-65c5-4f84-ac0b-d6d303cbbb22\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-d6a47bf7-94654b468-rsmhv"
Oct 31 00:39:24.644039 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:24.643876    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5f7bb018-65c5-4f84-ac0b-d6d303cbbb22-maas-api-tls\") pod \"maas-api-e2e-ait-d6a47bf7-94654b468-rsmhv\" (UID: \"5f7bb018-65c5-4f84-ac0b-d6d303cbbb22\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-d6a47bf7-94654b468-rsmhv"
Oct 31 00:39:24.644039 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:24.643967    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5f7bb018-65c5-4f84-ac0b-d6d303cbbb22-metrics-certs\") pod \"maas-api-e2e-ait-d6a47bf7-94654b468-rsmhv\" (UID: \"5f7bb018-65c5-4f84-ac0b-d6d303cbbb22\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-d6a47bf7-94654b468-rsmhv"
Oct 31 00:39:24.644039 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:24.644005    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5f7bb018-65c5-4f84-ac0b-d6d303cbbb22-openshift-service-ca\") pod \"maas-api-e2e-ait-d6a47bf7-94654b468-rsmhv\" (UID: \"5f7bb018-65c5-4f84-ac0b-d6d303cbbb22\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-d6a47bf7-94654b468-rsmhv"
Oct 31 00:39:24.669262 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:24.669234    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-d6a47bf7-94654b468-rsmhv"]
Oct 31 00:39:24.685756 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:24.685723    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:39:24.697495 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:24.697454    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-30d0c8aa-dfc88b4b5-qfvz7" event={"ID":"9fb753b3-95a2-41cf-b736-8244616735a1","Type":"ContainerStarted","Data":"d52b4229634906423e3ca4020c7935f872529478ea4b73746a2b72c7c93d8b00"}
Oct 31 00:39:24.697976 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:24.697523    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-30d0c8aa-dfc88b4b5-qfvz7" event={"ID":"9fb753b3-95a2-41cf-b736-8244616735a1","Type":"ContainerStarted","Data":"db814b67f160c13b38fe09449b3871fb21a08eef91ff667136898e698fde595a"}
Oct 31 00:39:24.744857 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:24.744829    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5f7bb018-65c5-4f84-ac0b-d6d303cbbb22-metrics-certs\") pod \"maas-api-e2e-ait-d6a47bf7-94654b468-rsmhv\" (UID: \"5f7bb018-65c5-4f84-ac0b-d6d303cbbb22\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-d6a47bf7-94654b468-rsmhv"
Oct 31 00:39:24.745025 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:24.744866    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5f7bb018-65c5-4f84-ac0b-d6d303cbbb22-openshift-service-ca\") pod \"maas-api-e2e-ait-d6a47bf7-94654b468-rsmhv\" (UID: \"5f7bb018-65c5-4f84-ac0b-d6d303cbbb22\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-d6a47bf7-94654b468-rsmhv"
Oct 31 00:39:24.745025 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:24.744991    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4d6xg\" (UniqueName: \"kubernetes.io/projected/5f7bb018-65c5-4f84-ac0b-d6d303cbbb22-kube-api-access-4d6xg\") pod \"maas-api-e2e-ait-d6a47bf7-94654b468-rsmhv\" (UID: \"5f7bb018-65c5-4f84-ac0b-d6d303cbbb22\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-d6a47bf7-94654b468-rsmhv"
Oct 31 00:39:24.745136 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:24.745054    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5f7bb018-65c5-4f84-ac0b-d6d303cbbb22-maas-api-tls\") pod \"maas-api-e2e-ait-d6a47bf7-94654b468-rsmhv\" (UID: \"5f7bb018-65c5-4f84-ac0b-d6d303cbbb22\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-d6a47bf7-94654b468-rsmhv"
Oct 31 00:39:24.745188 ip-10-0-132-173 kubenswrapper[4284]: E1031 00:39:24.745172    4284 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-d6a47bf7-serving-cert: secret "maas-api-e2e-ait-d6a47bf7-serving-cert" not found
Oct 31 00:39:24.745268 ip-10-0-132-173 kubenswrapper[4284]: E1031 00:39:24.745250    4284 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5f7bb018-65c5-4f84-ac0b-d6d303cbbb22-maas-api-tls podName:5f7bb018-65c5-4f84-ac0b-d6d303cbbb22 nodeName:}" failed. No retries permitted until 2026-10-31 00:39:25.245228663 +0000 UTC m=+679.390235570 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/5f7bb018-65c5-4f84-ac0b-d6d303cbbb22-maas-api-tls") pod "maas-api-e2e-ait-d6a47bf7-94654b468-rsmhv" (UID: "5f7bb018-65c5-4f84-ac0b-d6d303cbbb22") : secret "maas-api-e2e-ait-d6a47bf7-serving-cert" not found
Oct 31 00:39:24.745646 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:24.745625    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5f7bb018-65c5-4f84-ac0b-d6d303cbbb22-openshift-service-ca\") pod \"maas-api-e2e-ait-d6a47bf7-94654b468-rsmhv\" (UID: \"5f7bb018-65c5-4f84-ac0b-d6d303cbbb22\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-d6a47bf7-94654b468-rsmhv"
Oct 31 00:39:24.747546 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:24.747524    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5f7bb018-65c5-4f84-ac0b-d6d303cbbb22-metrics-certs\") pod \"maas-api-e2e-ait-d6a47bf7-94654b468-rsmhv\" (UID: \"5f7bb018-65c5-4f84-ac0b-d6d303cbbb22\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-d6a47bf7-94654b468-rsmhv"
Oct 31 00:39:24.822338 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:24.822282    4284 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-migrate-30d0c8aa-dfc88b4b5-qfvz7" podStartSLOduration=2.822267484 podStartE2EDuration="2.822267484s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 00:39:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:39:24.818774604 +0000 UTC m=+678.963781531" watchObservedRunningTime="2026-10-31 00:39:24.822267484 +0000 UTC m=+678.967274399"
Oct 31 00:39:24.825711 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:24.825681    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4d6xg\" (UniqueName: \"kubernetes.io/projected/5f7bb018-65c5-4f84-ac0b-d6d303cbbb22-kube-api-access-4d6xg\") pod \"maas-api-e2e-ait-d6a47bf7-94654b468-rsmhv\" (UID: \"5f7bb018-65c5-4f84-ac0b-d6d303cbbb22\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-d6a47bf7-94654b468-rsmhv"
Oct 31 00:39:24.862036 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:24.862002    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:39:25.020692 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:25.020658    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:39:25.173402 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:25.173362    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:39:25.249550 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:25.249512    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5f7bb018-65c5-4f84-ac0b-d6d303cbbb22-maas-api-tls\") pod \"maas-api-e2e-ait-d6a47bf7-94654b468-rsmhv\" (UID: \"5f7bb018-65c5-4f84-ac0b-d6d303cbbb22\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-d6a47bf7-94654b468-rsmhv"
Oct 31 00:39:25.252149 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:25.252123    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5f7bb018-65c5-4f84-ac0b-d6d303cbbb22-maas-api-tls\") pod \"maas-api-e2e-ait-d6a47bf7-94654b468-rsmhv\" (UID: \"5f7bb018-65c5-4f84-ac0b-d6d303cbbb22\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-d6a47bf7-94654b468-rsmhv"
Oct 31 00:39:25.483713 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:25.483677    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d6a47bf7-94654b468-rsmhv"
Oct 31 00:39:25.701920 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:25.701888    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-migrate-30d0c8aa-dfc88b4b5-qfvz7"
Oct 31 00:39:25.858636 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:25.858610    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-d6a47bf7-94654b468-rsmhv"]
Oct 31 00:39:25.860464 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:39:25.860436    4284 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5f7bb018_65c5_4f84_ac0b_d6d303cbbb22.slice/crio-80170210f50e8e781e97d6be378f747a2872d5ffd6533ee32c49c1ba4a844622 WatchSource:0}: Error finding container 80170210f50e8e781e97d6be378f747a2872d5ffd6533ee32c49c1ba4a844622: Status 404 returned error can't find the container with id 80170210f50e8e781e97d6be378f747a2872d5ffd6533ee32c49c1ba4a844622
Oct 31 00:39:26.707251 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:26.707200    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d6a47bf7-94654b468-rsmhv" event={"ID":"5f7bb018-65c5-4f84-ac0b-d6d303cbbb22","Type":"ContainerStarted","Data":"80170210f50e8e781e97d6be378f747a2872d5ffd6533ee32c49c1ba4a844622"}
Oct 31 00:39:27.212038 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:27.212005    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:39:27.406454 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:27.406420    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:39:27.714574 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:27.714486    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d6a47bf7-94654b468-rsmhv" event={"ID":"5f7bb018-65c5-4f84-ac0b-d6d303cbbb22","Type":"ContainerStarted","Data":"f0b6e9cf7a94e9376b79346ab66f447f760e59785820762fa736a0a2d484b7ec"}
Oct 31 00:39:27.715019 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:27.714618    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d6a47bf7-94654b468-rsmhv"
Oct 31 00:39:27.815145 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:27.815106    4284 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-24d78330-867bff44fc-xljll"]
Oct 31 00:39:27.818818 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:27.818799    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-24d78330-867bff44fc-xljll"
Oct 31 00:39:27.862498 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:27.862426    4284 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d6a47bf7-94654b468-rsmhv" podStartSLOduration=2.623635175 podStartE2EDuration="3.862405024s" totalImagesPullingTime="1.238769849s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 00:39:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:39:27.829900552 +0000 UTC m=+681.974907469" watchObservedRunningTime="2026-10-31 00:39:27.862405024 +0000 UTC m=+682.007411944"
Oct 31 00:39:27.866907 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:27.866870    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-24d78330-867bff44fc-xljll"]
Oct 31 00:39:27.868784 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:27.868758    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-24h7w\" (UniqueName: \"kubernetes.io/projected/441dd734-1e3c-40f7-a8e8-c8fbd0863d6e-kube-api-access-24h7w\") pod \"payload-processing-e2e-ait-24d78330-867bff44fc-xljll\" (UID: \"441dd734-1e3c-40f7-a8e8-c8fbd0863d6e\") " pod="openshift-ingress/payload-processing-e2e-ait-24d78330-867bff44fc-xljll"
Oct 31 00:39:27.868923 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:27.868908    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/441dd734-1e3c-40f7-a8e8-c8fbd0863d6e-plugins-config-volume\") pod \"payload-processing-e2e-ait-24d78330-867bff44fc-xljll\" (UID: \"441dd734-1e3c-40f7-a8e8-c8fbd0863d6e\") " pod="openshift-ingress/payload-processing-e2e-ait-24d78330-867bff44fc-xljll"
Oct 31 00:39:27.877337 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:27.877313    4284 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-24d78330\""
Oct 31 00:39:27.877337 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:27.877321    4284 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-24d78330-dockercfg-wrs4h\""
Oct 31 00:39:27.970147 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:27.970060    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/441dd734-1e3c-40f7-a8e8-c8fbd0863d6e-plugins-config-volume\") pod \"payload-processing-e2e-ait-24d78330-867bff44fc-xljll\" (UID: \"441dd734-1e3c-40f7-a8e8-c8fbd0863d6e\") " pod="openshift-ingress/payload-processing-e2e-ait-24d78330-867bff44fc-xljll"
Oct 31 00:39:27.970321 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:27.970183    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-24h7w\" (UniqueName: \"kubernetes.io/projected/441dd734-1e3c-40f7-a8e8-c8fbd0863d6e-kube-api-access-24h7w\") pod \"payload-processing-e2e-ait-24d78330-867bff44fc-xljll\" (UID: \"441dd734-1e3c-40f7-a8e8-c8fbd0863d6e\") " pod="openshift-ingress/payload-processing-e2e-ait-24d78330-867bff44fc-xljll"
Oct 31 00:39:27.970887 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:27.970862    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/441dd734-1e3c-40f7-a8e8-c8fbd0863d6e-plugins-config-volume\") pod \"payload-processing-e2e-ait-24d78330-867bff44fc-xljll\" (UID: \"441dd734-1e3c-40f7-a8e8-c8fbd0863d6e\") " pod="openshift-ingress/payload-processing-e2e-ait-24d78330-867bff44fc-xljll"
Oct 31 00:39:28.030255 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:28.030201    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-24h7w\" (UniqueName: \"kubernetes.io/projected/441dd734-1e3c-40f7-a8e8-c8fbd0863d6e-kube-api-access-24h7w\") pod \"payload-processing-e2e-ait-24d78330-867bff44fc-xljll\" (UID: \"441dd734-1e3c-40f7-a8e8-c8fbd0863d6e\") " pod="openshift-ingress/payload-processing-e2e-ait-24d78330-867bff44fc-xljll"
Oct 31 00:39:28.136425 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:28.136385    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-24d78330-867bff44fc-xljll"
Oct 31 00:39:28.436455 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:28.436412    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:39:28.503014 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:28.502990    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-24d78330-867bff44fc-xljll"]
Oct 31 00:39:28.504856 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:39:28.504815    4284 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod441dd734_1e3c_40f7_a8e8_c8fbd0863d6e.slice/crio-978b385eaa953d886bb4b4b1168b7f1fd6d12f7d8b7b6fc11e3fc2eb62cc1b90 WatchSource:0}: Error finding container 978b385eaa953d886bb4b4b1168b7f1fd6d12f7d8b7b6fc11e3fc2eb62cc1b90: Status 404 returned error can't find the container with id 978b385eaa953d886bb4b4b1168b7f1fd6d12f7d8b7b6fc11e3fc2eb62cc1b90
Oct 31 00:39:28.641391 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:28.641355    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:39:28.721570 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:28.721471    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-24d78330-867bff44fc-xljll" event={"ID":"441dd734-1e3c-40f7-a8e8-c8fbd0863d6e","Type":"ContainerStarted","Data":"ba8601b583dab5e868eb80143133a817ace81e681a35ba0120d4370c5d8c9a11"}
Oct 31 00:39:28.721570 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:28.721534    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-24d78330-867bff44fc-xljll" event={"ID":"441dd734-1e3c-40f7-a8e8-c8fbd0863d6e","Type":"ContainerStarted","Data":"978b385eaa953d886bb4b4b1168b7f1fd6d12f7d8b7b6fc11e3fc2eb62cc1b90"}
Oct 31 00:39:28.855617 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:28.855549    4284 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-24d78330-867bff44fc-xljll" podStartSLOduration=1.855533968 podStartE2EDuration="1.855533968s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 00:39:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:39:28.854033934 +0000 UTC m=+682.999040849" watchObservedRunningTime="2026-10-31 00:39:28.855533968 +0000 UTC m=+683.000540883"
Oct 31 00:39:28.868815 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:28.868774    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:39:29.615974 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:29.615932    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:39:29.731924 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:29.731894    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-24d78330-867bff44fc-xljll"
Oct 31 00:39:29.786560 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:29.786527    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:39:30.042004 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:30.041970    4284 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-30d0c8aa-6888fdf64d-j7gv6"]
Oct 31 00:39:30.045436 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:30.045420    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-30d0c8aa-6888fdf64d-j7gv6"
Oct 31 00:39:30.091320 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:30.091286    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wrxqf\" (UniqueName: \"kubernetes.io/projected/768ad3f4-6a42-468a-ad3a-7dc92fb234c4-kube-api-access-wrxqf\") pod \"maas-api-e2e-migrate-30d0c8aa-6888fdf64d-j7gv6\" (UID: \"768ad3f4-6a42-468a-ad3a-7dc92fb234c4\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-30d0c8aa-6888fdf64d-j7gv6"
Oct 31 00:39:30.091575 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:30.091554    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/768ad3f4-6a42-468a-ad3a-7dc92fb234c4-maas-api-tls\") pod \"maas-api-e2e-migrate-30d0c8aa-6888fdf64d-j7gv6\" (UID: \"768ad3f4-6a42-468a-ad3a-7dc92fb234c4\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-30d0c8aa-6888fdf64d-j7gv6"
Oct 31 00:39:30.091765 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:30.091739    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/768ad3f4-6a42-468a-ad3a-7dc92fb234c4-openshift-service-ca\") pod \"maas-api-e2e-migrate-30d0c8aa-6888fdf64d-j7gv6\" (UID: \"768ad3f4-6a42-468a-ad3a-7dc92fb234c4\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-30d0c8aa-6888fdf64d-j7gv6"
Oct 31 00:39:30.091880 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:30.091821    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/768ad3f4-6a42-468a-ad3a-7dc92fb234c4-metrics-certs\") pod \"maas-api-e2e-migrate-30d0c8aa-6888fdf64d-j7gv6\" (UID: \"768ad3f4-6a42-468a-ad3a-7dc92fb234c4\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-30d0c8aa-6888fdf64d-j7gv6"
Oct 31 00:39:30.095178 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:30.095148    4284 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-migrate-30d0c8aa-serving-cert\""
Oct 31 00:39:30.144991 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:30.144961    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-30d0c8aa-6888fdf64d-j7gv6"]
Oct 31 00:39:30.192989 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:30.192955    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wrxqf\" (UniqueName: \"kubernetes.io/projected/768ad3f4-6a42-468a-ad3a-7dc92fb234c4-kube-api-access-wrxqf\") pod \"maas-api-e2e-migrate-30d0c8aa-6888fdf64d-j7gv6\" (UID: \"768ad3f4-6a42-468a-ad3a-7dc92fb234c4\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-30d0c8aa-6888fdf64d-j7gv6"
Oct 31 00:39:30.192989 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:30.192993    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/768ad3f4-6a42-468a-ad3a-7dc92fb234c4-maas-api-tls\") pod \"maas-api-e2e-migrate-30d0c8aa-6888fdf64d-j7gv6\" (UID: \"768ad3f4-6a42-468a-ad3a-7dc92fb234c4\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-30d0c8aa-6888fdf64d-j7gv6"
Oct 31 00:39:30.193253 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:30.193041    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/768ad3f4-6a42-468a-ad3a-7dc92fb234c4-openshift-service-ca\") pod \"maas-api-e2e-migrate-30d0c8aa-6888fdf64d-j7gv6\" (UID: \"768ad3f4-6a42-468a-ad3a-7dc92fb234c4\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-30d0c8aa-6888fdf64d-j7gv6"
Oct 31 00:39:30.193253 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:30.193099    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/768ad3f4-6a42-468a-ad3a-7dc92fb234c4-metrics-certs\") pod \"maas-api-e2e-migrate-30d0c8aa-6888fdf64d-j7gv6\" (UID: \"768ad3f4-6a42-468a-ad3a-7dc92fb234c4\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-30d0c8aa-6888fdf64d-j7gv6"
Oct 31 00:39:30.193253 ip-10-0-132-173 kubenswrapper[4284]: E1031 00:39:30.193189    4284 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-migrate-30d0c8aa-serving-cert: secret "maas-api-e2e-migrate-30d0c8aa-serving-cert" not found
Oct 31 00:39:30.193409 ip-10-0-132-173 kubenswrapper[4284]: E1031 00:39:30.193279    4284 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/768ad3f4-6a42-468a-ad3a-7dc92fb234c4-maas-api-tls podName:768ad3f4-6a42-468a-ad3a-7dc92fb234c4 nodeName:}" failed. No retries permitted until 2026-10-31 00:39:30.693263894 +0000 UTC m=+684.838270800 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/768ad3f4-6a42-468a-ad3a-7dc92fb234c4-maas-api-tls") pod "maas-api-e2e-migrate-30d0c8aa-6888fdf64d-j7gv6" (UID: "768ad3f4-6a42-468a-ad3a-7dc92fb234c4") : secret "maas-api-e2e-migrate-30d0c8aa-serving-cert" not found
Oct 31 00:39:30.193732 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:30.193713    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/768ad3f4-6a42-468a-ad3a-7dc92fb234c4-openshift-service-ca\") pod \"maas-api-e2e-migrate-30d0c8aa-6888fdf64d-j7gv6\" (UID: \"768ad3f4-6a42-468a-ad3a-7dc92fb234c4\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-30d0c8aa-6888fdf64d-j7gv6"
Oct 31 00:39:30.195650 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:30.195628    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/768ad3f4-6a42-468a-ad3a-7dc92fb234c4-metrics-certs\") pod \"maas-api-e2e-migrate-30d0c8aa-6888fdf64d-j7gv6\" (UID: \"768ad3f4-6a42-468a-ad3a-7dc92fb234c4\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-30d0c8aa-6888fdf64d-j7gv6"
Oct 31 00:39:30.255502 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:30.255470    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wrxqf\" (UniqueName: \"kubernetes.io/projected/768ad3f4-6a42-468a-ad3a-7dc92fb234c4-kube-api-access-wrxqf\") pod \"maas-api-e2e-migrate-30d0c8aa-6888fdf64d-j7gv6\" (UID: \"768ad3f4-6a42-468a-ad3a-7dc92fb234c4\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-30d0c8aa-6888fdf64d-j7gv6"
Oct 31 00:39:30.304406 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:30.304319    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:39:30.474067 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:30.474033    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:39:30.625997 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:30.625907    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:39:30.696371 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:30.696337    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/768ad3f4-6a42-468a-ad3a-7dc92fb234c4-maas-api-tls\") pod \"maas-api-e2e-migrate-30d0c8aa-6888fdf64d-j7gv6\" (UID: \"768ad3f4-6a42-468a-ad3a-7dc92fb234c4\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-30d0c8aa-6888fdf64d-j7gv6"
Oct 31 00:39:30.698929 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:30.698908    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/768ad3f4-6a42-468a-ad3a-7dc92fb234c4-maas-api-tls\") pod \"maas-api-e2e-migrate-30d0c8aa-6888fdf64d-j7gv6\" (UID: \"768ad3f4-6a42-468a-ad3a-7dc92fb234c4\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-30d0c8aa-6888fdf64d-j7gv6"
Oct 31 00:39:30.958250 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:30.958192    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-30d0c8aa-6888fdf64d-j7gv6"
Oct 31 00:39:31.328259 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:31.328231    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-30d0c8aa-6888fdf64d-j7gv6"]
Oct 31 00:39:31.329821 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:39:31.329787    4284 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod768ad3f4_6a42_468a_ad3a_7dc92fb234c4.slice/crio-937d43cadd28f733ace7907dfc3ce91a8f2502f90c2aff6591b143c51e37fc10 WatchSource:0}: Error finding container 937d43cadd28f733ace7907dfc3ce91a8f2502f90c2aff6591b143c51e37fc10: Status 404 returned error can't find the container with id 937d43cadd28f733ace7907dfc3ce91a8f2502f90c2aff6591b143c51e37fc10
Oct 31 00:39:31.741158 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:31.741123    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-30d0c8aa-6888fdf64d-j7gv6" event={"ID":"768ad3f4-6a42-468a-ad3a-7dc92fb234c4","Type":"ContainerStarted","Data":"937d43cadd28f733ace7907dfc3ce91a8f2502f90c2aff6591b143c51e37fc10"}
Oct 31 00:39:31.994293 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:31.994180    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:39:32.260127 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:32.260094    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:39:32.571797 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:32.571721    4284 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-95fa4445-f8bcbf959-8qrhg"]
Oct 31 00:39:32.576056 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:32.576029    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-95fa4445-f8bcbf959-8qrhg"
Oct 31 00:39:32.612842 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:32.612800    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8585003b-9127-409c-a2a6-e8cce1e5241f-plugins-config-volume\") pod \"payload-processing-e2e-ait-95fa4445-f8bcbf959-8qrhg\" (UID: \"8585003b-9127-409c-a2a6-e8cce1e5241f\") " pod="openshift-ingress/payload-processing-e2e-ait-95fa4445-f8bcbf959-8qrhg"
Oct 31 00:39:32.613014 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:32.612849    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-brrl5\" (UniqueName: \"kubernetes.io/projected/8585003b-9127-409c-a2a6-e8cce1e5241f-kube-api-access-brrl5\") pod \"payload-processing-e2e-ait-95fa4445-f8bcbf959-8qrhg\" (UID: \"8585003b-9127-409c-a2a6-e8cce1e5241f\") " pod="openshift-ingress/payload-processing-e2e-ait-95fa4445-f8bcbf959-8qrhg"
Oct 31 00:39:32.629729 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:32.629692    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-95fa4445-f8bcbf959-8qrhg"]
Oct 31 00:39:32.674819 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:32.674790    4284 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-95fa4445\""
Oct 31 00:39:32.674974 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:32.674892    4284 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-95fa4445-dockercfg-vm5b8\""
Oct 31 00:39:32.713577 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:32.713542    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8585003b-9127-409c-a2a6-e8cce1e5241f-plugins-config-volume\") pod \"payload-processing-e2e-ait-95fa4445-f8bcbf959-8qrhg\" (UID: \"8585003b-9127-409c-a2a6-e8cce1e5241f\") " pod="openshift-ingress/payload-processing-e2e-ait-95fa4445-f8bcbf959-8qrhg"
Oct 31 00:39:32.713777 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:32.713589    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-brrl5\" (UniqueName: \"kubernetes.io/projected/8585003b-9127-409c-a2a6-e8cce1e5241f-kube-api-access-brrl5\") pod \"payload-processing-e2e-ait-95fa4445-f8bcbf959-8qrhg\" (UID: \"8585003b-9127-409c-a2a6-e8cce1e5241f\") " pod="openshift-ingress/payload-processing-e2e-ait-95fa4445-f8bcbf959-8qrhg"
Oct 31 00:39:32.714362 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:32.714344    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8585003b-9127-409c-a2a6-e8cce1e5241f-plugins-config-volume\") pod \"payload-processing-e2e-ait-95fa4445-f8bcbf959-8qrhg\" (UID: \"8585003b-9127-409c-a2a6-e8cce1e5241f\") " pod="openshift-ingress/payload-processing-e2e-ait-95fa4445-f8bcbf959-8qrhg"
Oct 31 00:39:32.747494 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:32.747461    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-30d0c8aa-6888fdf64d-j7gv6" event={"ID":"768ad3f4-6a42-468a-ad3a-7dc92fb234c4","Type":"ContainerStarted","Data":"2bc67ce86ebb30152dc36d5b5e9fc06c4d697b0e9d41cc2b4615c6b0721f3224"}
Oct 31 00:39:32.747678 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:32.747535    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-30d0c8aa-6888fdf64d-j7gv6"
Oct 31 00:39:32.782878 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:32.782847    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-brrl5\" (UniqueName: \"kubernetes.io/projected/8585003b-9127-409c-a2a6-e8cce1e5241f-kube-api-access-brrl5\") pod \"payload-processing-e2e-ait-95fa4445-f8bcbf959-8qrhg\" (UID: \"8585003b-9127-409c-a2a6-e8cce1e5241f\") " pod="openshift-ingress/payload-processing-e2e-ait-95fa4445-f8bcbf959-8qrhg"
Oct 31 00:39:32.866616 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:32.866504    4284 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-30d0c8aa-6888fdf64d-j7gv6" podStartSLOduration=3.006565161 podStartE2EDuration="3.866490421s" totalImagesPullingTime="859.92526ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 00:39:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:39:32.86455233 +0000 UTC m=+687.009559245" watchObservedRunningTime="2026-10-31 00:39:32.866490421 +0000 UTC m=+687.011497337"
Oct 31 00:39:32.895293 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:32.895254    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-95fa4445-f8bcbf959-8qrhg"
Oct 31 00:39:33.264598 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:33.264574    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-95fa4445-f8bcbf959-8qrhg"]
Oct 31 00:39:33.266619 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:39:33.266591    4284 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8585003b_9127_409c_a2a6_e8cce1e5241f.slice/crio-1d1660f8de0a587754a9d84817e4232ce7a960fb159662c59714f1a779cbe64e WatchSource:0}: Error finding container 1d1660f8de0a587754a9d84817e4232ce7a960fb159662c59714f1a779cbe64e: Status 404 returned error can't find the container with id 1d1660f8de0a587754a9d84817e4232ce7a960fb159662c59714f1a779cbe64e
Oct 31 00:39:33.727566 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:33.727538    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d6a47bf7-94654b468-rsmhv"
Oct 31 00:39:33.753771 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:33.753733    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-95fa4445-f8bcbf959-8qrhg" event={"ID":"8585003b-9127-409c-a2a6-e8cce1e5241f","Type":"ContainerStarted","Data":"fa47acb0a086194cae4486470ed1f7e17326b33fe37a3387a57aa0661aae0089"}
Oct 31 00:39:33.753975 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:33.753793    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-95fa4445-f8bcbf959-8qrhg" event={"ID":"8585003b-9127-409c-a2a6-e8cce1e5241f","Type":"ContainerStarted","Data":"1d1660f8de0a587754a9d84817e4232ce7a960fb159662c59714f1a779cbe64e"}
Oct 31 00:39:33.753975 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:33.753893    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-95fa4445-f8bcbf959-8qrhg"
Oct 31 00:39:33.953207 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:33.953152    4284 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-95fa4445-f8bcbf959-8qrhg" podStartSLOduration=1.953138954 podStartE2EDuration="1.953138954s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 00:39:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:39:33.951721849 +0000 UTC m=+688.096728766" watchObservedRunningTime="2026-10-31 00:39:33.953138954 +0000 UTC m=+688.098145888"
Oct 31 00:39:34.710268 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:34.710235    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-30d0c8aa-7959895f-lrjn5"
Oct 31 00:39:35.443879 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:35.443833    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:39:36.712078 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:36.712044    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-migrate-30d0c8aa-dfc88b4b5-qfvz7"
Oct 31 00:39:38.760310 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:38.760284    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-30d0c8aa-6888fdf64d-j7gv6"
Oct 31 00:39:40.628426 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:40.628389    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:39:40.739425 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:40.739395    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-24d78330-867bff44fc-xljll"
Oct 31 00:39:40.776541 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:40.776490    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:39:40.932525 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:40.932487    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:39:41.145377 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:41.145336    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:39:41.318784 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:41.318700    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:39:43.805658 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:43.805621    4284 generic.go:356] "Generic (PLEG): container finished" podID="b5a5a3d3-2cfe-4853-bdc3-1401e2b055a2" containerID="e3d1ad0398c08cac241da7abc1c6d8c913f1ea1c8725dea9128d481196aba8e3" exitCode=0
Oct 31 00:39:43.806149 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:43.805684    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-d6a47bf7-openshift-default-6bd8b956bc-5lwbr" event={"ID":"b5a5a3d3-2cfe-4853-bdc3-1401e2b055a2","Type":"ContainerDied","Data":"e3d1ad0398c08cac241da7abc1c6d8c913f1ea1c8725dea9128d481196aba8e3"}
Oct 31 00:39:43.806149 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:43.806039    4284 scope.go:122] "RemoveContainer" containerID="e3d1ad0398c08cac241da7abc1c6d8c913f1ea1c8725dea9128d481196aba8e3"
Oct 31 00:39:43.807285 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:43.807259    4284 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 00:39:43.807343 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:43.807326    4284 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 00:39:43.807378 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:43.807357    4284 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 00:39:44.761664 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:44.761638    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-95fa4445-f8bcbf959-8qrhg"
Oct 31 00:39:44.814757 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:44.814729    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-d6a47bf7-openshift-default-6bd8b956bc-5lwbr" event={"ID":"b5a5a3d3-2cfe-4853-bdc3-1401e2b055a2","Type":"ContainerStarted","Data":"787a620bbc3c73ceeb5594626b5e6cb50f53e35d2dde188524d1aa93b00817be"}
Oct 31 00:39:45.353280 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:45.353246    4284 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-d6a47bf7-openshift-default-6bd8b956bc-5lwbr"
Oct 31 00:39:45.354719 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:45.354695    4284 patch_prober.go:29] interesting pod/e2e-ait-d6a47bf7-openshift-default-6bd8b956bc-5lwbr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" start-of-body=
Oct 31 00:39:45.354858 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:45.354737    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-d6a47bf7-openshift-default-6bd8b956bc-5lwbr" podUID="b5a5a3d3-2cfe-4853-bdc3-1401e2b055a2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused"
Oct 31 00:39:45.465895 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:45.465863    4284 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-2b9202f4-84bdc6fd79-jbltf"]
Oct 31 00:39:45.469584 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:45.469565    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-2b9202f4-84bdc6fd79-jbltf"
Oct 31 00:39:45.519577 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:45.519541    4284 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-2b9202f4-serving-cert\""
Oct 31 00:39:45.525683 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:45.525657    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-2b9202f4-84bdc6fd79-jbltf"]
Oct 31 00:39:45.533543 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:45.533517    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cf5cae5a-b883-47db-8160-af723f17e17a-maas-api-tls\") pod \"maas-api-e2e-derive-2b9202f4-84bdc6fd79-jbltf\" (UID: \"cf5cae5a-b883-47db-8160-af723f17e17a\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-2b9202f4-84bdc6fd79-jbltf"
Oct 31 00:39:45.533677 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:45.533573    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cf5cae5a-b883-47db-8160-af723f17e17a-openshift-service-ca\") pod \"maas-api-e2e-derive-2b9202f4-84bdc6fd79-jbltf\" (UID: \"cf5cae5a-b883-47db-8160-af723f17e17a\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-2b9202f4-84bdc6fd79-jbltf"
Oct 31 00:39:45.533720 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:45.533703    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cf5cae5a-b883-47db-8160-af723f17e17a-metrics-certs\") pod \"maas-api-e2e-derive-2b9202f4-84bdc6fd79-jbltf\" (UID: \"cf5cae5a-b883-47db-8160-af723f17e17a\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-2b9202f4-84bdc6fd79-jbltf"
Oct 31 00:39:45.533778 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:45.533758    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5srw2\" (UniqueName: \"kubernetes.io/projected/cf5cae5a-b883-47db-8160-af723f17e17a-kube-api-access-5srw2\") pod \"maas-api-e2e-derive-2b9202f4-84bdc6fd79-jbltf\" (UID: \"cf5cae5a-b883-47db-8160-af723f17e17a\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-2b9202f4-84bdc6fd79-jbltf"
Oct 31 00:39:45.635071 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:45.634979    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cf5cae5a-b883-47db-8160-af723f17e17a-metrics-certs\") pod \"maas-api-e2e-derive-2b9202f4-84bdc6fd79-jbltf\" (UID: \"cf5cae5a-b883-47db-8160-af723f17e17a\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-2b9202f4-84bdc6fd79-jbltf"
Oct 31 00:39:45.635071 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:45.635032    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5srw2\" (UniqueName: \"kubernetes.io/projected/cf5cae5a-b883-47db-8160-af723f17e17a-kube-api-access-5srw2\") pod \"maas-api-e2e-derive-2b9202f4-84bdc6fd79-jbltf\" (UID: \"cf5cae5a-b883-47db-8160-af723f17e17a\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-2b9202f4-84bdc6fd79-jbltf"
Oct 31 00:39:45.635347 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:45.635079    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cf5cae5a-b883-47db-8160-af723f17e17a-maas-api-tls\") pod \"maas-api-e2e-derive-2b9202f4-84bdc6fd79-jbltf\" (UID: \"cf5cae5a-b883-47db-8160-af723f17e17a\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-2b9202f4-84bdc6fd79-jbltf"
Oct 31 00:39:45.635347 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:45.635117    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cf5cae5a-b883-47db-8160-af723f17e17a-openshift-service-ca\") pod \"maas-api-e2e-derive-2b9202f4-84bdc6fd79-jbltf\" (UID: \"cf5cae5a-b883-47db-8160-af723f17e17a\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-2b9202f4-84bdc6fd79-jbltf"
Oct 31 00:39:45.635347 ip-10-0-132-173 kubenswrapper[4284]: E1031 00:39:45.635205    4284 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-derive-2b9202f4-serving-cert: secret "maas-api-e2e-derive-2b9202f4-serving-cert" not found
Oct 31 00:39:45.635347 ip-10-0-132-173 kubenswrapper[4284]: E1031 00:39:45.635287    4284 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cf5cae5a-b883-47db-8160-af723f17e17a-maas-api-tls podName:cf5cae5a-b883-47db-8160-af723f17e17a nodeName:}" failed. No retries permitted until 2026-10-31 00:39:46.135269697 +0000 UTC m=+700.280276591 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/cf5cae5a-b883-47db-8160-af723f17e17a-maas-api-tls") pod "maas-api-e2e-derive-2b9202f4-84bdc6fd79-jbltf" (UID: "cf5cae5a-b883-47db-8160-af723f17e17a") : secret "maas-api-e2e-derive-2b9202f4-serving-cert" not found
Oct 31 00:39:45.635811 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:45.635794    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cf5cae5a-b883-47db-8160-af723f17e17a-openshift-service-ca\") pod \"maas-api-e2e-derive-2b9202f4-84bdc6fd79-jbltf\" (UID: \"cf5cae5a-b883-47db-8160-af723f17e17a\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-2b9202f4-84bdc6fd79-jbltf"
Oct 31 00:39:45.637596 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:45.637575    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cf5cae5a-b883-47db-8160-af723f17e17a-metrics-certs\") pod \"maas-api-e2e-derive-2b9202f4-84bdc6fd79-jbltf\" (UID: \"cf5cae5a-b883-47db-8160-af723f17e17a\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-2b9202f4-84bdc6fd79-jbltf"
Oct 31 00:39:45.662137 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:45.662080    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:39:45.733228 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:45.730014    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5srw2\" (UniqueName: \"kubernetes.io/projected/cf5cae5a-b883-47db-8160-af723f17e17a-kube-api-access-5srw2\") pod \"maas-api-e2e-derive-2b9202f4-84bdc6fd79-jbltf\" (UID: \"cf5cae5a-b883-47db-8160-af723f17e17a\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-2b9202f4-84bdc6fd79-jbltf"
Oct 31 00:39:45.856922 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:45.856883    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:39:46.010921 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:46.010879    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:39:46.140109 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:46.140074    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cf5cae5a-b883-47db-8160-af723f17e17a-maas-api-tls\") pod \"maas-api-e2e-derive-2b9202f4-84bdc6fd79-jbltf\" (UID: \"cf5cae5a-b883-47db-8160-af723f17e17a\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-2b9202f4-84bdc6fd79-jbltf"
Oct 31 00:39:46.142706 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:46.142682    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cf5cae5a-b883-47db-8160-af723f17e17a-maas-api-tls\") pod \"maas-api-e2e-derive-2b9202f4-84bdc6fd79-jbltf\" (UID: \"cf5cae5a-b883-47db-8160-af723f17e17a\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-2b9202f4-84bdc6fd79-jbltf"
Oct 31 00:39:46.353865 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:46.353774    4284 patch_prober.go:29] interesting pod/e2e-ait-d6a47bf7-openshift-default-6bd8b956bc-5lwbr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" start-of-body=
Oct 31 00:39:46.353865 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:46.353832    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-d6a47bf7-openshift-default-6bd8b956bc-5lwbr" podUID="b5a5a3d3-2cfe-4853-bdc3-1401e2b055a2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused"
Oct 31 00:39:46.381807 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:46.381777    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-2b9202f4-84bdc6fd79-jbltf"
Oct 31 00:39:46.782240 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:46.782195    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-2b9202f4-84bdc6fd79-jbltf"]
Oct 31 00:39:46.783261 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:39:46.783233    4284 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcf5cae5a_b883_47db_8160_af723f17e17a.slice/crio-9fcdb2f5ca73c04df90900d45533c0f2b17c4f80da433ebf6400ee2b439e7f21 WatchSource:0}: Error finding container 9fcdb2f5ca73c04df90900d45533c0f2b17c4f80da433ebf6400ee2b439e7f21: Status 404 returned error can't find the container with id 9fcdb2f5ca73c04df90900d45533c0f2b17c4f80da433ebf6400ee2b439e7f21
Oct 31 00:39:46.825328 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:46.825292    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-2b9202f4-84bdc6fd79-jbltf" event={"ID":"cf5cae5a-b883-47db-8160-af723f17e17a","Type":"ContainerStarted","Data":"9fcdb2f5ca73c04df90900d45533c0f2b17c4f80da433ebf6400ee2b439e7f21"}
Oct 31 00:39:47.036167 ip-10-0-132-173 kubenswrapper[4284]: E1031 00:39:47.036074    4284 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fd8sp" containerName="extractor"
Oct 31 00:39:47.354251 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:47.354149    4284 patch_prober.go:29] interesting pod/e2e-ait-d6a47bf7-openshift-default-6bd8b956bc-5lwbr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" start-of-body=
Oct 31 00:39:47.354251 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:47.354204    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-d6a47bf7-openshift-default-6bd8b956bc-5lwbr" podUID="b5a5a3d3-2cfe-4853-bdc3-1401e2b055a2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused"
Oct 31 00:39:47.660101 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:47.660066    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-d6a47bf7-openshift-default-6bd8b956bc-5lwbr"
Oct 31 00:39:48.353709 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:48.353673    4284 patch_prober.go:29] interesting pod/e2e-ait-d6a47bf7-openshift-default-6bd8b956bc-5lwbr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" start-of-body=
Oct 31 00:39:48.354115 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:48.353731    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-d6a47bf7-openshift-default-6bd8b956bc-5lwbr" podUID="b5a5a3d3-2cfe-4853-bdc3-1401e2b055a2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused"
Oct 31 00:39:48.836431 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:48.836389    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-2b9202f4-84bdc6fd79-jbltf" event={"ID":"cf5cae5a-b883-47db-8160-af723f17e17a","Type":"ContainerStarted","Data":"56b8580f89bfdb3bef0dd57a7bd952028ad448383db3230f48f0bce26d12eb0c"}
Oct 31 00:39:48.836588 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:48.836455    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-2b9202f4-84bdc6fd79-jbltf"
Oct 31 00:39:48.965790 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:48.965726    4284 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-2b9202f4-84bdc6fd79-jbltf" podStartSLOduration=3.024949805 podStartE2EDuration="3.965711353s" totalImagesPullingTime="940.761548ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 00:39:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:39:48.964838068 +0000 UTC m=+703.109844997" watchObservedRunningTime="2026-10-31 00:39:48.965711353 +0000 UTC m=+703.110718268"
Oct 31 00:39:49.353146 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:49.353112    4284 patch_prober.go:29] interesting pod/e2e-ait-d6a47bf7-openshift-default-6bd8b956bc-5lwbr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" start-of-body=
Oct 31 00:39:49.353336 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:49.353177    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-d6a47bf7-openshift-default-6bd8b956bc-5lwbr" podUID="b5a5a3d3-2cfe-4853-bdc3-1401e2b055a2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused"
Oct 31 00:39:50.353730 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:50.353689    4284 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-95fa4445-7c699f7c77-pnr6h"]
Oct 31 00:39:50.357488 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:50.357466    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-95fa4445-7c699f7c77-pnr6h"
Oct 31 00:39:50.358501 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:50.358481    4284 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-d6a47bf7-openshift-default-6bd8b956bc-5lwbr"
Oct 31 00:39:50.359687 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:50.359670    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-d6a47bf7-openshift-default-6bd8b956bc-5lwbr"
Oct 31 00:39:50.409101 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:50.409068    4284 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-95fa4445-serving-cert\""
Oct 31 00:39:50.458951 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:50.458915    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-95fa4445-7c699f7c77-pnr6h"]
Oct 31 00:39:50.480239 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:50.480179    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f41cd753-2c38-4a4a-a1a4-ec1bbacf75bb-metrics-certs\") pod \"maas-api-e2e-ait-95fa4445-7c699f7c77-pnr6h\" (UID: \"f41cd753-2c38-4a4a-a1a4-ec1bbacf75bb\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-95fa4445-7c699f7c77-pnr6h"
Oct 31 00:39:50.480425 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:50.480370    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p225q\" (UniqueName: \"kubernetes.io/projected/f41cd753-2c38-4a4a-a1a4-ec1bbacf75bb-kube-api-access-p225q\") pod \"maas-api-e2e-ait-95fa4445-7c699f7c77-pnr6h\" (UID: \"f41cd753-2c38-4a4a-a1a4-ec1bbacf75bb\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-95fa4445-7c699f7c77-pnr6h"
Oct 31 00:39:50.480477 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:50.480445    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f41cd753-2c38-4a4a-a1a4-ec1bbacf75bb-openshift-service-ca\") pod \"maas-api-e2e-ait-95fa4445-7c699f7c77-pnr6h\" (UID: \"f41cd753-2c38-4a4a-a1a4-ec1bbacf75bb\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-95fa4445-7c699f7c77-pnr6h"
Oct 31 00:39:50.480610 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:50.480581    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f41cd753-2c38-4a4a-a1a4-ec1bbacf75bb-maas-api-tls\") pod \"maas-api-e2e-ait-95fa4445-7c699f7c77-pnr6h\" (UID: \"f41cd753-2c38-4a4a-a1a4-ec1bbacf75bb\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-95fa4445-7c699f7c77-pnr6h"
Oct 31 00:39:50.564756 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:50.564721    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:39:50.582016 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:50.581984    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p225q\" (UniqueName: \"kubernetes.io/projected/f41cd753-2c38-4a4a-a1a4-ec1bbacf75bb-kube-api-access-p225q\") pod \"maas-api-e2e-ait-95fa4445-7c699f7c77-pnr6h\" (UID: \"f41cd753-2c38-4a4a-a1a4-ec1bbacf75bb\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-95fa4445-7c699f7c77-pnr6h"
Oct 31 00:39:50.582170 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:50.582036    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f41cd753-2c38-4a4a-a1a4-ec1bbacf75bb-openshift-service-ca\") pod \"maas-api-e2e-ait-95fa4445-7c699f7c77-pnr6h\" (UID: \"f41cd753-2c38-4a4a-a1a4-ec1bbacf75bb\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-95fa4445-7c699f7c77-pnr6h"
Oct 31 00:39:50.582170 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:50.582073    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f41cd753-2c38-4a4a-a1a4-ec1bbacf75bb-maas-api-tls\") pod \"maas-api-e2e-ait-95fa4445-7c699f7c77-pnr6h\" (UID: \"f41cd753-2c38-4a4a-a1a4-ec1bbacf75bb\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-95fa4445-7c699f7c77-pnr6h"
Oct 31 00:39:50.582170 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:50.582123    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f41cd753-2c38-4a4a-a1a4-ec1bbacf75bb-metrics-certs\") pod \"maas-api-e2e-ait-95fa4445-7c699f7c77-pnr6h\" (UID: \"f41cd753-2c38-4a4a-a1a4-ec1bbacf75bb\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-95fa4445-7c699f7c77-pnr6h"
Oct 31 00:39:50.582346 ip-10-0-132-173 kubenswrapper[4284]: E1031 00:39:50.582274    4284 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-95fa4445-serving-cert: secret "maas-api-e2e-ait-95fa4445-serving-cert" not found
Oct 31 00:39:50.582346 ip-10-0-132-173 kubenswrapper[4284]: E1031 00:39:50.582333    4284 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f41cd753-2c38-4a4a-a1a4-ec1bbacf75bb-maas-api-tls podName:f41cd753-2c38-4a4a-a1a4-ec1bbacf75bb nodeName:}" failed. No retries permitted until 2026-10-31 00:39:51.082315257 +0000 UTC m=+705.227322159 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/f41cd753-2c38-4a4a-a1a4-ec1bbacf75bb-maas-api-tls") pod "maas-api-e2e-ait-95fa4445-7c699f7c77-pnr6h" (UID: "f41cd753-2c38-4a4a-a1a4-ec1bbacf75bb") : secret "maas-api-e2e-ait-95fa4445-serving-cert" not found
Oct 31 00:39:50.582757 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:50.582734    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f41cd753-2c38-4a4a-a1a4-ec1bbacf75bb-openshift-service-ca\") pod \"maas-api-e2e-ait-95fa4445-7c699f7c77-pnr6h\" (UID: \"f41cd753-2c38-4a4a-a1a4-ec1bbacf75bb\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-95fa4445-7c699f7c77-pnr6h"
Oct 31 00:39:50.584737 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:50.584712    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f41cd753-2c38-4a4a-a1a4-ec1bbacf75bb-metrics-certs\") pod \"maas-api-e2e-ait-95fa4445-7c699f7c77-pnr6h\" (UID: \"f41cd753-2c38-4a4a-a1a4-ec1bbacf75bb\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-95fa4445-7c699f7c77-pnr6h"
Oct 31 00:39:50.641157 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:50.641072    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p225q\" (UniqueName: \"kubernetes.io/projected/f41cd753-2c38-4a4a-a1a4-ec1bbacf75bb-kube-api-access-p225q\") pod \"maas-api-e2e-ait-95fa4445-7c699f7c77-pnr6h\" (UID: \"f41cd753-2c38-4a4a-a1a4-ec1bbacf75bb\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-95fa4445-7c699f7c77-pnr6h"
Oct 31 00:39:50.770683 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:50.770645    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:39:50.925870 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:50.925829    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:39:51.086925 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:51.086887    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f41cd753-2c38-4a4a-a1a4-ec1bbacf75bb-maas-api-tls\") pod \"maas-api-e2e-ait-95fa4445-7c699f7c77-pnr6h\" (UID: \"f41cd753-2c38-4a4a-a1a4-ec1bbacf75bb\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-95fa4445-7c699f7c77-pnr6h"
Oct 31 00:39:51.089562 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:51.089537    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f41cd753-2c38-4a4a-a1a4-ec1bbacf75bb-maas-api-tls\") pod \"maas-api-e2e-ait-95fa4445-7c699f7c77-pnr6h\" (UID: \"f41cd753-2c38-4a4a-a1a4-ec1bbacf75bb\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-95fa4445-7c699f7c77-pnr6h"
Oct 31 00:39:51.270630 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:51.270539    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-95fa4445-7c699f7c77-pnr6h"
Oct 31 00:39:51.664442 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:51.664413    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-95fa4445-7c699f7c77-pnr6h"]
Oct 31 00:39:51.666513 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:39:51.666484    4284 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf41cd753_2c38_4a4a_a1a4_ec1bbacf75bb.slice/crio-5b3540ead3c98e07dae7e6343885f18f3fe20b954a3bebf2f539f2127df99b4e WatchSource:0}: Error finding container 5b3540ead3c98e07dae7e6343885f18f3fe20b954a3bebf2f539f2127df99b4e: Status 404 returned error can't find the container with id 5b3540ead3c98e07dae7e6343885f18f3fe20b954a3bebf2f539f2127df99b4e
Oct 31 00:39:51.853510 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:51.853480    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-95fa4445-7c699f7c77-pnr6h" event={"ID":"f41cd753-2c38-4a4a-a1a4-ec1bbacf75bb","Type":"ContainerStarted","Data":"5b3540ead3c98e07dae7e6343885f18f3fe20b954a3bebf2f539f2127df99b4e"}
Oct 31 00:39:52.656078 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:52.656036    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:39:52.860558 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:52.860476    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-95fa4445-7c699f7c77-pnr6h" event={"ID":"f41cd753-2c38-4a4a-a1a4-ec1bbacf75bb","Type":"ContainerStarted","Data":"df019eab9a2fd3932e754754cd70bdfcff01d264b8eeec7a60cdcf9dec1105be"}
Oct 31 00:39:52.860953 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:52.860587    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-95fa4445-7c699f7c77-pnr6h"
Oct 31 00:39:52.997950 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:52.997899    4284 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-95fa4445-7c699f7c77-pnr6h" podStartSLOduration=2.093853684 podStartE2EDuration="2.997882296s" totalImagesPullingTime="904.028612ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 00:39:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:39:52.996205253 +0000 UTC m=+707.141212168" watchObservedRunningTime="2026-10-31 00:39:52.997882296 +0000 UTC m=+707.142889257"
Oct 31 00:39:54.045988 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:54.045951    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:39:54.241115 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:54.241084    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:39:54.848266 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:54.848231    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-2b9202f4-84bdc6fd79-jbltf"
Oct 31 00:39:55.596582 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:55.596544    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:39:58.872714 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:39:58.872688    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-95fa4445-7c699f7c77-pnr6h"
Oct 31 00:40:00.474883 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:00.474844    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:40:05.370697 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:05.370662    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:40:09.962343 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:09.962257    4284 generic.go:356] "Generic (PLEG): container finished" podID="06987c79-2521-487f-a60d-3050698f9719" containerID="5078f568d9db91b92e652db9f85003d72d143a1c54dbf308c657860aac539164" exitCode=0
Oct 31 00:40:09.963560 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:09.963536    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-95fa4445-openshift-default-c9bd77677-wz7qd" event={"ID":"06987c79-2521-487f-a60d-3050698f9719","Type":"ContainerDied","Data":"5078f568d9db91b92e652db9f85003d72d143a1c54dbf308c657860aac539164"}
Oct 31 00:40:09.963925 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:09.963912    4284 scope.go:122] "RemoveContainer" containerID="5078f568d9db91b92e652db9f85003d72d143a1c54dbf308c657860aac539164"
Oct 31 00:40:09.965248 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:09.965201    4284 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 00:40:09.965328 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:09.965289    4284 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 00:40:09.965328 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:09.965320    4284 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 00:40:10.698376 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:10.698306    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:40:10.968914 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:10.968824    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-95fa4445-openshift-default-c9bd77677-wz7qd" event={"ID":"06987c79-2521-487f-a60d-3050698f9719","Type":"ContainerStarted","Data":"f5a24fe6446e45bf1f8d243f02e2759b02f8b8316400142e14a7c27c8c3f686c"}
Oct 31 00:40:11.457911 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:11.457878    4284 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-95fa4445-openshift-default-c9bd77677-wz7qd"
Oct 31 00:40:11.458114 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:11.458077    4284 patch_prober.go:29] interesting pod/e2e-ait-95fa4445-openshift-default-c9bd77677-wz7qd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" start-of-body=
Oct 31 00:40:11.458188 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:11.458127    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-95fa4445-openshift-default-c9bd77677-wz7qd" podUID="06987c79-2521-487f-a60d-3050698f9719" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused"
Oct 31 00:40:12.457892 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:12.457852    4284 patch_prober.go:29] interesting pod/e2e-ait-95fa4445-openshift-default-c9bd77677-wz7qd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" start-of-body=
Oct 31 00:40:12.458355 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:12.457910    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-95fa4445-openshift-default-c9bd77677-wz7qd" podUID="06987c79-2521-487f-a60d-3050698f9719" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused"
Oct 31 00:40:13.457977 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:13.457935    4284 patch_prober.go:29] interesting pod/e2e-ait-95fa4445-openshift-default-c9bd77677-wz7qd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" start-of-body=
Oct 31 00:40:13.458397 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:13.458003    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-95fa4445-openshift-default-c9bd77677-wz7qd" podUID="06987c79-2521-487f-a60d-3050698f9719" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused"
Oct 31 00:40:14.458314 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:14.458280    4284 patch_prober.go:29] interesting pod/e2e-ait-95fa4445-openshift-default-c9bd77677-wz7qd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" start-of-body=
Oct 31 00:40:14.458714 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:14.458339    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-95fa4445-openshift-default-c9bd77677-wz7qd" podUID="06987c79-2521-487f-a60d-3050698f9719" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused"
Oct 31 00:40:15.457273 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:15.457239    4284 patch_prober.go:29] interesting pod/e2e-ait-95fa4445-openshift-default-c9bd77677-wz7qd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" start-of-body=
Oct 31 00:40:15.457476 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:15.457299    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-95fa4445-openshift-default-c9bd77677-wz7qd" podUID="06987c79-2521-487f-a60d-3050698f9719" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused"
Oct 31 00:40:15.962815 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:15.962778    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:40:16.461670 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:16.461639    4284 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-95fa4445-openshift-default-c9bd77677-wz7qd"
Oct 31 00:40:16.461950 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:16.461933    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-95fa4445-openshift-default-c9bd77677-wz7qd"
Oct 31 00:40:16.462686 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:16.462671    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-95fa4445-openshift-default-c9bd77677-wz7qd"
Oct 31 00:40:17.013294 ip-10-0-132-173 kubenswrapper[4284]: E1031 00:40:17.013262    4284 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fd8sp" containerName="extractor"
Oct 31 00:40:17.742774 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:17.742741    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:40:17.893019 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:17.892980    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:40:21.058867 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:21.058827    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:40:25.555433 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:25.555394    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-d6a47bf7-94654b468-rsmhv"]
Oct 31 00:40:25.556036 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:25.555677    4284 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d6a47bf7-94654b468-rsmhv" podUID="5f7bb018-65c5-4f84-ac0b-d6d303cbbb22" containerName="maas-api" containerID="cri-o://f0b6e9cf7a94e9376b79346ab66f447f760e59785820762fa736a0a2d484b7ec" gracePeriod=30
Oct 31 00:40:25.671969 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:25.671926    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:40:25.828299 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:25.828275    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d6a47bf7-94654b468-rsmhv"
Oct 31 00:40:25.955835 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:25.955800    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/5f7bb018-65c5-4f84-ac0b-d6d303cbbb22-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5f7bb018-65c5-4f84-ac0b-d6d303cbbb22-maas-api-tls\") pod \"5f7bb018-65c5-4f84-ac0b-d6d303cbbb22\" (UID: \"5f7bb018-65c5-4f84-ac0b-d6d303cbbb22\") "
Oct 31 00:40:25.956040 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:25.955917    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/5f7bb018-65c5-4f84-ac0b-d6d303cbbb22-metrics-certs\" (UniqueName: \"kubernetes.io/secret/5f7bb018-65c5-4f84-ac0b-d6d303cbbb22-metrics-certs\") pod \"5f7bb018-65c5-4f84-ac0b-d6d303cbbb22\" (UID: \"5f7bb018-65c5-4f84-ac0b-d6d303cbbb22\") "
Oct 31 00:40:25.956040 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:25.956034    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/5f7bb018-65c5-4f84-ac0b-d6d303cbbb22-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5f7bb018-65c5-4f84-ac0b-d6d303cbbb22-openshift-service-ca\") pod \"5f7bb018-65c5-4f84-ac0b-d6d303cbbb22\" (UID: \"5f7bb018-65c5-4f84-ac0b-d6d303cbbb22\") "
Oct 31 00:40:25.956162 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:25.956077    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5f7bb018-65c5-4f84-ac0b-d6d303cbbb22-kube-api-access-4d6xg\" (UniqueName: \"kubernetes.io/projected/5f7bb018-65c5-4f84-ac0b-d6d303cbbb22-kube-api-access-4d6xg\") pod \"5f7bb018-65c5-4f84-ac0b-d6d303cbbb22\" (UID: \"5f7bb018-65c5-4f84-ac0b-d6d303cbbb22\") "
Oct 31 00:40:25.956536 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:25.956502    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5f7bb018-65c5-4f84-ac0b-d6d303cbbb22-openshift-service-ca" pod "5f7bb018-65c5-4f84-ac0b-d6d303cbbb22" (UID: "5f7bb018-65c5-4f84-ac0b-d6d303cbbb22"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 00:40:25.958753 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:25.958727    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5f7bb018-65c5-4f84-ac0b-d6d303cbbb22-kube-api-access-4d6xg" pod "5f7bb018-65c5-4f84-ac0b-d6d303cbbb22" (UID: "5f7bb018-65c5-4f84-ac0b-d6d303cbbb22"). InnerVolumeSpecName "kube-api-access-4d6xg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 00:40:25.958882 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:25.958864    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5f7bb018-65c5-4f84-ac0b-d6d303cbbb22-metrics-certs" pod "5f7bb018-65c5-4f84-ac0b-d6d303cbbb22" (UID: "5f7bb018-65c5-4f84-ac0b-d6d303cbbb22"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 00:40:25.959230 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:25.959149    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5f7bb018-65c5-4f84-ac0b-d6d303cbbb22-maas-api-tls" pod "5f7bb018-65c5-4f84-ac0b-d6d303cbbb22" (UID: "5f7bb018-65c5-4f84-ac0b-d6d303cbbb22"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 00:40:26.068379 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:26.066502    4284 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5f7bb018-65c5-4f84-ac0b-d6d303cbbb22-openshift-service-ca\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:40:26.068379 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:26.066542    4284 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4d6xg\" (UniqueName: \"kubernetes.io/projected/5f7bb018-65c5-4f84-ac0b-d6d303cbbb22-kube-api-access-4d6xg\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:40:26.068379 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:26.066558    4284 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5f7bb018-65c5-4f84-ac0b-d6d303cbbb22-maas-api-tls\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:40:26.068379 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:26.066571    4284 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5f7bb018-65c5-4f84-ac0b-d6d303cbbb22-metrics-certs\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:40:26.075137 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:26.075108    4284 generic.go:356] "Generic (PLEG): container finished" podID="5f7bb018-65c5-4f84-ac0b-d6d303cbbb22" containerID="f0b6e9cf7a94e9376b79346ab66f447f760e59785820762fa736a0a2d484b7ec" exitCode=0
Oct 31 00:40:26.075355 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:26.075166    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d6a47bf7-94654b468-rsmhv"
Oct 31 00:40:26.075355 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:26.075185    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d6a47bf7-94654b468-rsmhv" event={"ID":"5f7bb018-65c5-4f84-ac0b-d6d303cbbb22","Type":"ContainerDied","Data":"f0b6e9cf7a94e9376b79346ab66f447f760e59785820762fa736a0a2d484b7ec"}
Oct 31 00:40:26.075355 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:26.075276    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d6a47bf7-94654b468-rsmhv" event={"ID":"5f7bb018-65c5-4f84-ac0b-d6d303cbbb22","Type":"ContainerDied","Data":"80170210f50e8e781e97d6be378f747a2872d5ffd6533ee32c49c1ba4a844622"}
Oct 31 00:40:26.075355 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:26.075306    4284 scope.go:122] "RemoveContainer" containerID="f0b6e9cf7a94e9376b79346ab66f447f760e59785820762fa736a0a2d484b7ec"
Oct 31 00:40:26.084053 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:26.084031    4284 scope.go:122] "RemoveContainer" containerID="f0b6e9cf7a94e9376b79346ab66f447f760e59785820762fa736a0a2d484b7ec"
Oct 31 00:40:26.084457 ip-10-0-132-173 kubenswrapper[4284]: E1031 00:40:26.084436    4284 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f0b6e9cf7a94e9376b79346ab66f447f760e59785820762fa736a0a2d484b7ec\": container with ID starting with f0b6e9cf7a94e9376b79346ab66f447f760e59785820762fa736a0a2d484b7ec not found: ID does not exist" containerID="f0b6e9cf7a94e9376b79346ab66f447f760e59785820762fa736a0a2d484b7ec"
Oct 31 00:40:26.084544 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:26.084465    4284 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f0b6e9cf7a94e9376b79346ab66f447f760e59785820762fa736a0a2d484b7ec"} err="failed to get container status \"f0b6e9cf7a94e9376b79346ab66f447f760e59785820762fa736a0a2d484b7ec\": rpc error: code = NotFound desc = could not find container \"f0b6e9cf7a94e9376b79346ab66f447f760e59785820762fa736a0a2d484b7ec\": container with ID starting with f0b6e9cf7a94e9376b79346ab66f447f760e59785820762fa736a0a2d484b7ec not found: ID does not exist"
Oct 31 00:40:26.233419 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:26.233390    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-d6a47bf7-94654b468-rsmhv"]
Oct 31 00:40:26.237625 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:26.237594    4284 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-d6a47bf7-94654b468-rsmhv"]
Oct 31 00:40:27.285424 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:27.285367    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:40:27.433858 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:27.433821    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:40:27.598800 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:27.598708    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:40:27.934419 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:27.934388    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-d6a47bf7-openshift-default-6bd8b956bc-5lwbr"]
Oct 31 00:40:27.934598 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:27.934577    4284 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-d6a47bf7-openshift-default-6bd8b956bc-5lwbr" podUID="b5a5a3d3-2cfe-4853-bdc3-1401e2b055a2" containerName="istio-proxy" containerID="cri-o://787a620bbc3c73ceeb5594626b5e6cb50f53e35d2dde188524d1aa93b00817be" gracePeriod=30
Oct 31 00:40:27.959193 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:27.959165    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:40:27.962793 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:27.962763    4284 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5f7bb018-65c5-4f84-ac0b-d6d303cbbb22" path="/var/lib/kubelet/pods/5f7bb018-65c5-4f84-ac0b-d6d303cbbb22/volumes"
Oct 31 00:40:28.847546 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:28.847512    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:40:29.683629 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:29.683567    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:40:29.878073 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:29.878033    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:40:32.197081 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:32.197038    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:40:32.241095 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:32.241065    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-24d78330-867bff44fc-xljll"]
Oct 31 00:40:32.591657 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:32.591545    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:40:32.790700 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:32.790664    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:40:33.117681 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:33.117655    4284 generic.go:356] "Generic (PLEG): container finished" podID="b5a5a3d3-2cfe-4853-bdc3-1401e2b055a2" containerID="787a620bbc3c73ceeb5594626b5e6cb50f53e35d2dde188524d1aa93b00817be" exitCode=0
Oct 31 00:40:33.117848 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:33.117698    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-d6a47bf7-openshift-default-6bd8b956bc-5lwbr" event={"ID":"b5a5a3d3-2cfe-4853-bdc3-1401e2b055a2","Type":"ContainerDied","Data":"787a620bbc3c73ceeb5594626b5e6cb50f53e35d2dde188524d1aa93b00817be"}
Oct 31 00:40:33.117848 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:33.117761    4284 scope.go:122] "RemoveContainer" containerID="e3d1ad0398c08cac241da7abc1c6d8c913f1ea1c8725dea9128d481196aba8e3"
Oct 31 00:40:33.196032 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:33.196006    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-d6a47bf7-openshift-default-6bd8b956bc-5lwbr"
Oct 31 00:40:33.346368 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:33.346290    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b5a5a3d3-2cfe-4853-bdc3-1401e2b055a2-istio-token\" (UniqueName: \"kubernetes.io/projected/b5a5a3d3-2cfe-4853-bdc3-1401e2b055a2-istio-token\") pod \"b5a5a3d3-2cfe-4853-bdc3-1401e2b055a2\" (UID: \"b5a5a3d3-2cfe-4853-bdc3-1401e2b055a2\") "
Oct 31 00:40:33.346368 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:33.346346    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b5a5a3d3-2cfe-4853-bdc3-1401e2b055a2-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b5a5a3d3-2cfe-4853-bdc3-1401e2b055a2-istiod-ca-cert\") pod \"b5a5a3d3-2cfe-4853-bdc3-1401e2b055a2\" (UID: \"b5a5a3d3-2cfe-4853-bdc3-1401e2b055a2\") "
Oct 31 00:40:33.346883 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:33.346384    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b5a5a3d3-2cfe-4853-bdc3-1401e2b055a2-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b5a5a3d3-2cfe-4853-bdc3-1401e2b055a2-workload-socket\") pod \"b5a5a3d3-2cfe-4853-bdc3-1401e2b055a2\" (UID: \"b5a5a3d3-2cfe-4853-bdc3-1401e2b055a2\") "
Oct 31 00:40:33.346883 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:33.346434    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b5a5a3d3-2cfe-4853-bdc3-1401e2b055a2-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b5a5a3d3-2cfe-4853-bdc3-1401e2b055a2-workload-certs\") pod \"b5a5a3d3-2cfe-4853-bdc3-1401e2b055a2\" (UID: \"b5a5a3d3-2cfe-4853-bdc3-1401e2b055a2\") "
Oct 31 00:40:33.346883 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:33.346465    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/b5a5a3d3-2cfe-4853-bdc3-1401e2b055a2-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b5a5a3d3-2cfe-4853-bdc3-1401e2b055a2-istio-podinfo\") pod \"b5a5a3d3-2cfe-4853-bdc3-1401e2b055a2\" (UID: \"b5a5a3d3-2cfe-4853-bdc3-1401e2b055a2\") "
Oct 31 00:40:33.346883 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:33.346503    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b5a5a3d3-2cfe-4853-bdc3-1401e2b055a2-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b5a5a3d3-2cfe-4853-bdc3-1401e2b055a2-credential-socket\") pod \"b5a5a3d3-2cfe-4853-bdc3-1401e2b055a2\" (UID: \"b5a5a3d3-2cfe-4853-bdc3-1401e2b055a2\") "
Oct 31 00:40:33.346883 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:33.346667    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b5a5a3d3-2cfe-4853-bdc3-1401e2b055a2-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b5a5a3d3-2cfe-4853-bdc3-1401e2b055a2-istio-data\") pod \"b5a5a3d3-2cfe-4853-bdc3-1401e2b055a2\" (UID: \"b5a5a3d3-2cfe-4853-bdc3-1401e2b055a2\") "
Oct 31 00:40:33.346883 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:33.346727    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b5a5a3d3-2cfe-4853-bdc3-1401e2b055a2-kube-api-access-pcvtg\" (UniqueName: \"kubernetes.io/projected/b5a5a3d3-2cfe-4853-bdc3-1401e2b055a2-kube-api-access-pcvtg\") pod \"b5a5a3d3-2cfe-4853-bdc3-1401e2b055a2\" (UID: \"b5a5a3d3-2cfe-4853-bdc3-1401e2b055a2\") "
Oct 31 00:40:33.346883 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:33.346769    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b5a5a3d3-2cfe-4853-bdc3-1401e2b055a2-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b5a5a3d3-2cfe-4853-bdc3-1401e2b055a2-istio-envoy\") pod \"b5a5a3d3-2cfe-4853-bdc3-1401e2b055a2\" (UID: \"b5a5a3d3-2cfe-4853-bdc3-1401e2b055a2\") "
Oct 31 00:40:33.347239 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:33.346726    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b5a5a3d3-2cfe-4853-bdc3-1401e2b055a2-workload-certs" pod "b5a5a3d3-2cfe-4853-bdc3-1401e2b055a2" (UID: "b5a5a3d3-2cfe-4853-bdc3-1401e2b055a2"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:40:33.347239 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:33.346760    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b5a5a3d3-2cfe-4853-bdc3-1401e2b055a2-credential-socket" pod "b5a5a3d3-2cfe-4853-bdc3-1401e2b055a2" (UID: "b5a5a3d3-2cfe-4853-bdc3-1401e2b055a2"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:40:33.347239 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:33.346797    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b5a5a3d3-2cfe-4853-bdc3-1401e2b055a2-istiod-ca-cert" pod "b5a5a3d3-2cfe-4853-bdc3-1401e2b055a2" (UID: "b5a5a3d3-2cfe-4853-bdc3-1401e2b055a2"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 00:40:33.347239 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:33.346927    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b5a5a3d3-2cfe-4853-bdc3-1401e2b055a2-istio-data" pod "b5a5a3d3-2cfe-4853-bdc3-1401e2b055a2" (UID: "b5a5a3d3-2cfe-4853-bdc3-1401e2b055a2"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:40:33.347438 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:33.347299    4284 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b5a5a3d3-2cfe-4853-bdc3-1401e2b055a2-istio-data\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:40:33.347438 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:33.347424    4284 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b5a5a3d3-2cfe-4853-bdc3-1401e2b055a2-istiod-ca-cert\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:40:33.347541 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:33.347439    4284 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b5a5a3d3-2cfe-4853-bdc3-1401e2b055a2-workload-certs\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:40:33.347541 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:33.347454    4284 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b5a5a3d3-2cfe-4853-bdc3-1401e2b055a2-credential-socket\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:40:33.347674 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:33.347653    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b5a5a3d3-2cfe-4853-bdc3-1401e2b055a2-workload-socket" pod "b5a5a3d3-2cfe-4853-bdc3-1401e2b055a2" (UID: "b5a5a3d3-2cfe-4853-bdc3-1401e2b055a2"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:40:33.348953 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:33.348926    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b5a5a3d3-2cfe-4853-bdc3-1401e2b055a2-istio-podinfo" pod "b5a5a3d3-2cfe-4853-bdc3-1401e2b055a2" (UID: "b5a5a3d3-2cfe-4853-bdc3-1401e2b055a2"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 00:40:33.349086 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:33.348961    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b5a5a3d3-2cfe-4853-bdc3-1401e2b055a2-istio-envoy" pod "b5a5a3d3-2cfe-4853-bdc3-1401e2b055a2" (UID: "b5a5a3d3-2cfe-4853-bdc3-1401e2b055a2"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:40:33.349393 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:33.349374    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b5a5a3d3-2cfe-4853-bdc3-1401e2b055a2-kube-api-access-pcvtg" pod "b5a5a3d3-2cfe-4853-bdc3-1401e2b055a2" (UID: "b5a5a3d3-2cfe-4853-bdc3-1401e2b055a2"). InnerVolumeSpecName "kube-api-access-pcvtg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 00:40:33.349489 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:33.349403    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b5a5a3d3-2cfe-4853-bdc3-1401e2b055a2-istio-token" pod "b5a5a3d3-2cfe-4853-bdc3-1401e2b055a2" (UID: "b5a5a3d3-2cfe-4853-bdc3-1401e2b055a2"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 00:40:33.448116 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:33.448078    4284 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b5a5a3d3-2cfe-4853-bdc3-1401e2b055a2-istio-token\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:40:33.448116 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:33.448107    4284 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b5a5a3d3-2cfe-4853-bdc3-1401e2b055a2-workload-socket\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:40:33.448116 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:33.448117    4284 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b5a5a3d3-2cfe-4853-bdc3-1401e2b055a2-istio-podinfo\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:40:33.448116 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:33.448125    4284 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pcvtg\" (UniqueName: \"kubernetes.io/projected/b5a5a3d3-2cfe-4853-bdc3-1401e2b055a2-kube-api-access-pcvtg\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:40:33.448471 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:33.448135    4284 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b5a5a3d3-2cfe-4853-bdc3-1401e2b055a2-istio-envoy\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:40:33.937937 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:33.937902    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:40:34.094767 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:34.094737    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:40:34.126870 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:34.126723    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-d6a47bf7-openshift-default-6bd8b956bc-5lwbr" event={"ID":"b5a5a3d3-2cfe-4853-bdc3-1401e2b055a2","Type":"ContainerDied","Data":"da8e4e2506ba5a72dc7c1644d1f7841f42d493ca0b16392f71adefe2c9bc13cf"}
Oct 31 00:40:34.129434 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:34.127160    4284 scope.go:122] "RemoveContainer" containerID="787a620bbc3c73ceeb5594626b5e6cb50f53e35d2dde188524d1aa93b00817be"
Oct 31 00:40:34.129434 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:34.127386    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-d6a47bf7-openshift-default-6bd8b956bc-5lwbr"
Oct 31 00:40:34.285594 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:34.285503    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-d6a47bf7-openshift-default-6bd8b956bc-5lwbr"]
Oct 31 00:40:34.301119 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:34.301084    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:40:34.333719 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:34.333690    4284 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-d6a47bf7-openshift-default-6bd8b956bc-5lwbr"]
Oct 31 00:40:34.619993 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:34.619913    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-30d0c8aa-dfc88b4b5-qfvz7"]
Oct 31 00:40:34.723111 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:34.723072    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-30d0c8aa-7959895f-lrjn5"]
Oct 31 00:40:34.966394 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:34.966352    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:40:35.167402 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:35.167363    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:40:35.961688 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:35.961656    4284 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b5a5a3d3-2cfe-4853-bdc3-1401e2b055a2" path="/var/lib/kubelet/pods/b5a5a3d3-2cfe-4853-bdc3-1401e2b055a2/volumes"
Oct 31 00:40:36.866816 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:36.866777    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:40:37.241815 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:37.241751    4284 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-24d78330-867bff44fc-xljll" podUID="441dd734-1e3c-40f7-a8e8-c8fbd0863d6e" containerName="payload-processing" containerID="cri-o://ba8601b583dab5e868eb80143133a817ace81e681a35ba0120d4370c5d8c9a11" gracePeriod=25
Oct 31 00:40:37.497583 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:37.497517    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-24d78330-867bff44fc-xljll"
Oct 31 00:40:37.587373 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:37.587336    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/441dd734-1e3c-40f7-a8e8-c8fbd0863d6e-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/441dd734-1e3c-40f7-a8e8-c8fbd0863d6e-plugins-config-volume\") pod \"441dd734-1e3c-40f7-a8e8-c8fbd0863d6e\" (UID: \"441dd734-1e3c-40f7-a8e8-c8fbd0863d6e\") "
Oct 31 00:40:37.587537 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:37.587406    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/441dd734-1e3c-40f7-a8e8-c8fbd0863d6e-kube-api-access-24h7w\" (UniqueName: \"kubernetes.io/projected/441dd734-1e3c-40f7-a8e8-c8fbd0863d6e-kube-api-access-24h7w\") pod \"441dd734-1e3c-40f7-a8e8-c8fbd0863d6e\" (UID: \"441dd734-1e3c-40f7-a8e8-c8fbd0863d6e\") "
Oct 31 00:40:37.587732 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:37.587706    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/441dd734-1e3c-40f7-a8e8-c8fbd0863d6e-plugins-config-volume" pod "441dd734-1e3c-40f7-a8e8-c8fbd0863d6e" (UID: "441dd734-1e3c-40f7-a8e8-c8fbd0863d6e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 00:40:37.589654 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:37.589625    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/441dd734-1e3c-40f7-a8e8-c8fbd0863d6e-kube-api-access-24h7w" pod "441dd734-1e3c-40f7-a8e8-c8fbd0863d6e" (UID: "441dd734-1e3c-40f7-a8e8-c8fbd0863d6e"). InnerVolumeSpecName "kube-api-access-24h7w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 00:40:37.688889 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:37.688848    4284 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-24h7w\" (UniqueName: \"kubernetes.io/projected/441dd734-1e3c-40f7-a8e8-c8fbd0863d6e-kube-api-access-24h7w\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:40:37.688889 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:37.688890    4284 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/441dd734-1e3c-40f7-a8e8-c8fbd0863d6e-plugins-config-volume\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:40:37.885343 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:37.885249    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-2b9202f4-84bdc6fd79-jbltf"]
Oct 31 00:40:37.885523 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:37.885465    4284 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-derive-2b9202f4-84bdc6fd79-jbltf" podUID="cf5cae5a-b883-47db-8160-af723f17e17a" containerName="maas-api" containerID="cri-o://56b8580f89bfdb3bef0dd57a7bd952028ad448383db3230f48f0bce26d12eb0c" gracePeriod=30
Oct 31 00:40:37.990593 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:37.990561    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:40:38.139029 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:38.138965    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-2b9202f4-84bdc6fd79-jbltf"
Oct 31 00:40:38.151095 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:38.151066    4284 generic.go:356] "Generic (PLEG): container finished" podID="441dd734-1e3c-40f7-a8e8-c8fbd0863d6e" containerID="ba8601b583dab5e868eb80143133a817ace81e681a35ba0120d4370c5d8c9a11" exitCode=0
Oct 31 00:40:38.151296 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:38.151131    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-24d78330-867bff44fc-xljll"
Oct 31 00:40:38.151296 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:38.151144    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-24d78330-867bff44fc-xljll" event={"ID":"441dd734-1e3c-40f7-a8e8-c8fbd0863d6e","Type":"ContainerDied","Data":"ba8601b583dab5e868eb80143133a817ace81e681a35ba0120d4370c5d8c9a11"}
Oct 31 00:40:38.151296 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:38.151253    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-24d78330-867bff44fc-xljll" event={"ID":"441dd734-1e3c-40f7-a8e8-c8fbd0863d6e","Type":"ContainerDied","Data":"978b385eaa953d886bb4b4b1168b7f1fd6d12f7d8b7b6fc11e3fc2eb62cc1b90"}
Oct 31 00:40:38.151296 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:38.151282    4284 scope.go:122] "RemoveContainer" containerID="ba8601b583dab5e868eb80143133a817ace81e681a35ba0120d4370c5d8c9a11"
Oct 31 00:40:38.152429 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:38.152352    4284 generic.go:356] "Generic (PLEG): container finished" podID="cf5cae5a-b883-47db-8160-af723f17e17a" containerID="56b8580f89bfdb3bef0dd57a7bd952028ad448383db3230f48f0bce26d12eb0c" exitCode=0
Oct 31 00:40:38.152429 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:38.152403    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-2b9202f4-84bdc6fd79-jbltf"
Oct 31 00:40:38.152540 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:38.152430    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-2b9202f4-84bdc6fd79-jbltf" event={"ID":"cf5cae5a-b883-47db-8160-af723f17e17a","Type":"ContainerDied","Data":"56b8580f89bfdb3bef0dd57a7bd952028ad448383db3230f48f0bce26d12eb0c"}
Oct 31 00:40:38.152540 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:38.152484    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-2b9202f4-84bdc6fd79-jbltf" event={"ID":"cf5cae5a-b883-47db-8160-af723f17e17a","Type":"ContainerDied","Data":"9fcdb2f5ca73c04df90900d45533c0f2b17c4f80da433ebf6400ee2b439e7f21"}
Oct 31 00:40:38.160111 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:38.160088    4284 scope.go:122] "RemoveContainer" containerID="ba8601b583dab5e868eb80143133a817ace81e681a35ba0120d4370c5d8c9a11"
Oct 31 00:40:38.160407 ip-10-0-132-173 kubenswrapper[4284]: E1031 00:40:38.160386    4284 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ba8601b583dab5e868eb80143133a817ace81e681a35ba0120d4370c5d8c9a11\": container with ID starting with ba8601b583dab5e868eb80143133a817ace81e681a35ba0120d4370c5d8c9a11 not found: ID does not exist" containerID="ba8601b583dab5e868eb80143133a817ace81e681a35ba0120d4370c5d8c9a11"
Oct 31 00:40:38.160493 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:38.160418    4284 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ba8601b583dab5e868eb80143133a817ace81e681a35ba0120d4370c5d8c9a11"} err="failed to get container status \"ba8601b583dab5e868eb80143133a817ace81e681a35ba0120d4370c5d8c9a11\": rpc error: code = NotFound desc = could not find container \"ba8601b583dab5e868eb80143133a817ace81e681a35ba0120d4370c5d8c9a11\": container with ID starting with ba8601b583dab5e868eb80143133a817ace81e681a35ba0120d4370c5d8c9a11 not found: ID does not exist"
Oct 31 00:40:38.160493 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:38.160443    4284 scope.go:122] "RemoveContainer" containerID="56b8580f89bfdb3bef0dd57a7bd952028ad448383db3230f48f0bce26d12eb0c"
Oct 31 00:40:38.169673 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:38.169654    4284 scope.go:122] "RemoveContainer" containerID="56b8580f89bfdb3bef0dd57a7bd952028ad448383db3230f48f0bce26d12eb0c"
Oct 31 00:40:38.169964 ip-10-0-132-173 kubenswrapper[4284]: E1031 00:40:38.169947    4284 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"56b8580f89bfdb3bef0dd57a7bd952028ad448383db3230f48f0bce26d12eb0c\": container with ID starting with 56b8580f89bfdb3bef0dd57a7bd952028ad448383db3230f48f0bce26d12eb0c not found: ID does not exist" containerID="56b8580f89bfdb3bef0dd57a7bd952028ad448383db3230f48f0bce26d12eb0c"
Oct 31 00:40:38.170025 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:38.169976    4284 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"56b8580f89bfdb3bef0dd57a7bd952028ad448383db3230f48f0bce26d12eb0c"} err="failed to get container status \"56b8580f89bfdb3bef0dd57a7bd952028ad448383db3230f48f0bce26d12eb0c\": rpc error: code = NotFound desc = could not find container \"56b8580f89bfdb3bef0dd57a7bd952028ad448383db3230f48f0bce26d12eb0c\": container with ID starting with 56b8580f89bfdb3bef0dd57a7bd952028ad448383db3230f48f0bce26d12eb0c not found: ID does not exist"
Oct 31 00:40:38.287639 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:38.287601    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:40:38.301042 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:38.301008    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/cf5cae5a-b883-47db-8160-af723f17e17a-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cf5cae5a-b883-47db-8160-af723f17e17a-maas-api-tls\") pod \"cf5cae5a-b883-47db-8160-af723f17e17a\" (UID: \"cf5cae5a-b883-47db-8160-af723f17e17a\") "
Oct 31 00:40:38.301189 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:38.301076    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/cf5cae5a-b883-47db-8160-af723f17e17a-metrics-certs\" (UniqueName: \"kubernetes.io/secret/cf5cae5a-b883-47db-8160-af723f17e17a-metrics-certs\") pod \"cf5cae5a-b883-47db-8160-af723f17e17a\" (UID: \"cf5cae5a-b883-47db-8160-af723f17e17a\") "
Oct 31 00:40:38.301189 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:38.301118    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/cf5cae5a-b883-47db-8160-af723f17e17a-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cf5cae5a-b883-47db-8160-af723f17e17a-openshift-service-ca\") pod \"cf5cae5a-b883-47db-8160-af723f17e17a\" (UID: \"cf5cae5a-b883-47db-8160-af723f17e17a\") "
Oct 31 00:40:38.301294 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:38.301234    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/cf5cae5a-b883-47db-8160-af723f17e17a-kube-api-access-5srw2\" (UniqueName: \"kubernetes.io/projected/cf5cae5a-b883-47db-8160-af723f17e17a-kube-api-access-5srw2\") pod \"cf5cae5a-b883-47db-8160-af723f17e17a\" (UID: \"cf5cae5a-b883-47db-8160-af723f17e17a\") "
Oct 31 00:40:38.301565 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:38.301523    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cf5cae5a-b883-47db-8160-af723f17e17a-openshift-service-ca" pod "cf5cae5a-b883-47db-8160-af723f17e17a" (UID: "cf5cae5a-b883-47db-8160-af723f17e17a"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 00:40:38.303396 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:38.303374    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cf5cae5a-b883-47db-8160-af723f17e17a-kube-api-access-5srw2" pod "cf5cae5a-b883-47db-8160-af723f17e17a" (UID: "cf5cae5a-b883-47db-8160-af723f17e17a"). InnerVolumeSpecName "kube-api-access-5srw2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 00:40:38.303904 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:38.303883    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cf5cae5a-b883-47db-8160-af723f17e17a-metrics-certs" pod "cf5cae5a-b883-47db-8160-af723f17e17a" (UID: "cf5cae5a-b883-47db-8160-af723f17e17a"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 00:40:38.303993 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:38.303942    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cf5cae5a-b883-47db-8160-af723f17e17a-maas-api-tls" pod "cf5cae5a-b883-47db-8160-af723f17e17a" (UID: "cf5cae5a-b883-47db-8160-af723f17e17a"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 00:40:38.360698 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:38.360659    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-24d78330-867bff44fc-xljll"]
Oct 31 00:40:38.369943 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:38.369916    4284 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-24d78330-867bff44fc-xljll"]
Oct 31 00:40:38.403028 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:38.402949    4284 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cf5cae5a-b883-47db-8160-af723f17e17a-metrics-certs\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:40:38.403028 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:38.402977    4284 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cf5cae5a-b883-47db-8160-af723f17e17a-openshift-service-ca\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:40:38.403028 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:38.402986    4284 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5srw2\" (UniqueName: \"kubernetes.io/projected/cf5cae5a-b883-47db-8160-af723f17e17a-kube-api-access-5srw2\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:40:38.403028 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:38.402995    4284 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cf5cae5a-b883-47db-8160-af723f17e17a-maas-api-tls\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:40:38.459350 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:38.459314    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:40:38.571117 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:38.571084    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-2b9202f4-84bdc6fd79-jbltf"]
Oct 31 00:40:38.575441 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:38.575409    4284 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-2b9202f4-84bdc6fd79-jbltf"]
Oct 31 00:40:38.684270 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:38.684203    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:40:38.856468 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:38.856434    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:40:39.049827 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:39.049743    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:40:39.620447 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:39.620396    4284 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-migrate-30d0c8aa-dfc88b4b5-qfvz7" podUID="9fb753b3-95a2-41cf-b736-8244616735a1" containerName="payload-processing" containerID="cri-o://d52b4229634906423e3ca4020c7935f872529478ea4b73746a2b72c7c93d8b00" gracePeriod=25
Oct 31 00:40:39.724093 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:39.724016    4284 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-migrate-30d0c8aa-7959895f-lrjn5" podUID="f2147b56-458f-4d40-bbbe-75e90a4c1093" containerName="payload-pre-processing" containerID="cri-o://19749f9e2c162e83539f918b4bb1c624d84759e64a11721f78fa83ed23f25ace" gracePeriod=25
Oct 31 00:40:39.882004 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:39.881930    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-30d0c8aa-dfc88b4b5-qfvz7"
Oct 31 00:40:39.944055 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:39.944020    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:40:39.961662 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:39.961633    4284 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="441dd734-1e3c-40f7-a8e8-c8fbd0863d6e" path="/var/lib/kubelet/pods/441dd734-1e3c-40f7-a8e8-c8fbd0863d6e/volumes"
Oct 31 00:40:39.962027 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:39.962015    4284 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="cf5cae5a-b883-47db-8160-af723f17e17a" path="/var/lib/kubelet/pods/cf5cae5a-b883-47db-8160-af723f17e17a/volumes"
Oct 31 00:40:39.988800 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:39.988766    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-30d0c8aa-7959895f-lrjn5"
Oct 31 00:40:40.019350 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:40.019304    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/9fb753b3-95a2-41cf-b736-8244616735a1-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9fb753b3-95a2-41cf-b736-8244616735a1-plugins-config-volume\") pod \"9fb753b3-95a2-41cf-b736-8244616735a1\" (UID: \"9fb753b3-95a2-41cf-b736-8244616735a1\") "
Oct 31 00:40:40.019536 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:40.019398    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9fb753b3-95a2-41cf-b736-8244616735a1-kube-api-access-z2mf4\" (UniqueName: \"kubernetes.io/projected/9fb753b3-95a2-41cf-b736-8244616735a1-kube-api-access-z2mf4\") pod \"9fb753b3-95a2-41cf-b736-8244616735a1\" (UID: \"9fb753b3-95a2-41cf-b736-8244616735a1\") "
Oct 31 00:40:40.019715 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:40.019691    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9fb753b3-95a2-41cf-b736-8244616735a1-plugins-config-volume" pod "9fb753b3-95a2-41cf-b736-8244616735a1" (UID: "9fb753b3-95a2-41cf-b736-8244616735a1"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 00:40:40.022358 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:40.022330    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9fb753b3-95a2-41cf-b736-8244616735a1-kube-api-access-z2mf4" pod "9fb753b3-95a2-41cf-b736-8244616735a1" (UID: "9fb753b3-95a2-41cf-b736-8244616735a1"). InnerVolumeSpecName "kube-api-access-z2mf4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 00:40:40.120078 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:40.120048    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f2147b56-458f-4d40-bbbe-75e90a4c1093-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f2147b56-458f-4d40-bbbe-75e90a4c1093-plugins-config-volume\") pod \"f2147b56-458f-4d40-bbbe-75e90a4c1093\" (UID: \"f2147b56-458f-4d40-bbbe-75e90a4c1093\") "
Oct 31 00:40:40.120249 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:40.120112    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f2147b56-458f-4d40-bbbe-75e90a4c1093-kube-api-access-h9wfn\" (UniqueName: \"kubernetes.io/projected/f2147b56-458f-4d40-bbbe-75e90a4c1093-kube-api-access-h9wfn\") pod \"f2147b56-458f-4d40-bbbe-75e90a4c1093\" (UID: \"f2147b56-458f-4d40-bbbe-75e90a4c1093\") "
Oct 31 00:40:40.120465 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:40.120443    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f2147b56-458f-4d40-bbbe-75e90a4c1093-plugins-config-volume" pod "f2147b56-458f-4d40-bbbe-75e90a4c1093" (UID: "f2147b56-458f-4d40-bbbe-75e90a4c1093"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 00:40:40.120552 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:40.120538    4284 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f2147b56-458f-4d40-bbbe-75e90a4c1093-plugins-config-volume\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:40:40.120593 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:40.120557    4284 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9fb753b3-95a2-41cf-b736-8244616735a1-plugins-config-volume\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:40:40.120593 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:40.120568    4284 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z2mf4\" (UniqueName: \"kubernetes.io/projected/9fb753b3-95a2-41cf-b736-8244616735a1-kube-api-access-z2mf4\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:40:40.122348 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:40.122331    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f2147b56-458f-4d40-bbbe-75e90a4c1093-kube-api-access-h9wfn" pod "f2147b56-458f-4d40-bbbe-75e90a4c1093" (UID: "f2147b56-458f-4d40-bbbe-75e90a4c1093"). InnerVolumeSpecName "kube-api-access-h9wfn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 00:40:40.164718 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:40.164691    4284 generic.go:356] "Generic (PLEG): container finished" podID="9fb753b3-95a2-41cf-b736-8244616735a1" containerID="d52b4229634906423e3ca4020c7935f872529478ea4b73746a2b72c7c93d8b00" exitCode=0
Oct 31 00:40:40.164875 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:40.164738    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-30d0c8aa-dfc88b4b5-qfvz7"
Oct 31 00:40:40.164875 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:40.164767    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-30d0c8aa-dfc88b4b5-qfvz7" event={"ID":"9fb753b3-95a2-41cf-b736-8244616735a1","Type":"ContainerDied","Data":"d52b4229634906423e3ca4020c7935f872529478ea4b73746a2b72c7c93d8b00"}
Oct 31 00:40:40.164875 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:40.164824    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-30d0c8aa-dfc88b4b5-qfvz7" event={"ID":"9fb753b3-95a2-41cf-b736-8244616735a1","Type":"ContainerDied","Data":"db814b67f160c13b38fe09449b3871fb21a08eef91ff667136898e698fde595a"}
Oct 31 00:40:40.164875 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:40.164841    4284 scope.go:122] "RemoveContainer" containerID="d52b4229634906423e3ca4020c7935f872529478ea4b73746a2b72c7c93d8b00"
Oct 31 00:40:40.166092 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:40.166074    4284 generic.go:356] "Generic (PLEG): container finished" podID="f2147b56-458f-4d40-bbbe-75e90a4c1093" containerID="19749f9e2c162e83539f918b4bb1c624d84759e64a11721f78fa83ed23f25ace" exitCode=0
Oct 31 00:40:40.166202 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:40.166123    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-30d0c8aa-7959895f-lrjn5"
Oct 31 00:40:40.166202 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:40.166126    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-30d0c8aa-7959895f-lrjn5" event={"ID":"f2147b56-458f-4d40-bbbe-75e90a4c1093","Type":"ContainerDied","Data":"19749f9e2c162e83539f918b4bb1c624d84759e64a11721f78fa83ed23f25ace"}
Oct 31 00:40:40.166335 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:40.166248    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-30d0c8aa-7959895f-lrjn5" event={"ID":"f2147b56-458f-4d40-bbbe-75e90a4c1093","Type":"ContainerDied","Data":"ecbd32ba94916788e8284d88123c2a910e9791bb785e29cc2c2916107808b21a"}
Oct 31 00:40:40.174859 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:40.174841    4284 scope.go:122] "RemoveContainer" containerID="d52b4229634906423e3ca4020c7935f872529478ea4b73746a2b72c7c93d8b00"
Oct 31 00:40:40.175141 ip-10-0-132-173 kubenswrapper[4284]: E1031 00:40:40.175123    4284 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d52b4229634906423e3ca4020c7935f872529478ea4b73746a2b72c7c93d8b00\": container with ID starting with d52b4229634906423e3ca4020c7935f872529478ea4b73746a2b72c7c93d8b00 not found: ID does not exist" containerID="d52b4229634906423e3ca4020c7935f872529478ea4b73746a2b72c7c93d8b00"
Oct 31 00:40:40.175265 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:40.175148    4284 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d52b4229634906423e3ca4020c7935f872529478ea4b73746a2b72c7c93d8b00"} err="failed to get container status \"d52b4229634906423e3ca4020c7935f872529478ea4b73746a2b72c7c93d8b00\": rpc error: code = NotFound desc = could not find container \"d52b4229634906423e3ca4020c7935f872529478ea4b73746a2b72c7c93d8b00\": container with ID starting with d52b4229634906423e3ca4020c7935f872529478ea4b73746a2b72c7c93d8b00 not found: ID does not exist"
Oct 31 00:40:40.175265 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:40.175164    4284 scope.go:122] "RemoveContainer" containerID="19749f9e2c162e83539f918b4bb1c624d84759e64a11721f78fa83ed23f25ace"
Oct 31 00:40:40.183768 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:40.183752    4284 scope.go:122] "RemoveContainer" containerID="19749f9e2c162e83539f918b4bb1c624d84759e64a11721f78fa83ed23f25ace"
Oct 31 00:40:40.183982 ip-10-0-132-173 kubenswrapper[4284]: E1031 00:40:40.183965    4284 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"19749f9e2c162e83539f918b4bb1c624d84759e64a11721f78fa83ed23f25ace\": container with ID starting with 19749f9e2c162e83539f918b4bb1c624d84759e64a11721f78fa83ed23f25ace not found: ID does not exist" containerID="19749f9e2c162e83539f918b4bb1c624d84759e64a11721f78fa83ed23f25ace"
Oct 31 00:40:40.184020 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:40.183990    4284 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"19749f9e2c162e83539f918b4bb1c624d84759e64a11721f78fa83ed23f25ace"} err="failed to get container status \"19749f9e2c162e83539f918b4bb1c624d84759e64a11721f78fa83ed23f25ace\": rpc error: code = NotFound desc = could not find container \"19749f9e2c162e83539f918b4bb1c624d84759e64a11721f78fa83ed23f25ace\": container with ID starting with 19749f9e2c162e83539f918b4bb1c624d84759e64a11721f78fa83ed23f25ace not found: ID does not exist"
Oct 31 00:40:40.221549 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:40.221517    4284 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h9wfn\" (UniqueName: \"kubernetes.io/projected/f2147b56-458f-4d40-bbbe-75e90a4c1093-kube-api-access-h9wfn\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:40:40.323526 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:40.323413    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-30d0c8aa-7959895f-lrjn5"]
Oct 31 00:40:40.326343 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:40.326317    4284 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-30d0c8aa-7959895f-lrjn5"]
Oct 31 00:40:40.430860 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:40.430779    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-30d0c8aa-dfc88b4b5-qfvz7"]
Oct 31 00:40:40.432384 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:40.432365    4284 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-30d0c8aa-dfc88b4b5-qfvz7"]
Oct 31 00:40:40.814145 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:40.814062    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:40:41.201949 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:41.201915    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-95fa4445-f8bcbf959-8qrhg"]
Oct 31 00:40:41.557552 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:41.557467    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:40:41.708929 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:41.708887    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:40:41.961734 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:41.961705    4284 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="9fb753b3-95a2-41cf-b736-8244616735a1" path="/var/lib/kubelet/pods/9fb753b3-95a2-41cf-b736-8244616735a1/volumes"
Oct 31 00:40:41.962114 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:41.962041    4284 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f2147b56-458f-4d40-bbbe-75e90a4c1093" path="/var/lib/kubelet/pods/f2147b56-458f-4d40-bbbe-75e90a4c1093/volumes"
Oct 31 00:40:41.965191 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:41.965166    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:40:42.231174 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:42.231098    4284 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-knm5g"]
Oct 31 00:40:42.231496 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:42.231483    4284 cpu_manager.go:414] "containerMap: removing container" podUID="9fb753b3-95a2-41cf-b736-8244616735a1" containerName="payload-processing"
Oct 31 00:40:42.231537 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:42.231497    4284 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9fb753b3-95a2-41cf-b736-8244616735a1" containerName="payload-processing"
Oct 31 00:40:42.231537 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:42.231507    4284 cpu_manager.go:414] "containerMap: removing container" podUID="5f7bb018-65c5-4f84-ac0b-d6d303cbbb22" containerName="maas-api"
Oct 31 00:40:42.231537 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:42.231512    4284 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5f7bb018-65c5-4f84-ac0b-d6d303cbbb22" containerName="maas-api"
Oct 31 00:40:42.231537 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:42.231529    4284 cpu_manager.go:414] "containerMap: removing container" podUID="f2147b56-458f-4d40-bbbe-75e90a4c1093" containerName="payload-pre-processing"
Oct 31 00:40:42.231537 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:42.231534    4284 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f2147b56-458f-4d40-bbbe-75e90a4c1093" containerName="payload-pre-processing"
Oct 31 00:40:42.231688 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:42.231553    4284 cpu_manager.go:414] "containerMap: removing container" podUID="b5a5a3d3-2cfe-4853-bdc3-1401e2b055a2" containerName="istio-proxy"
Oct 31 00:40:42.231688 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:42.231558    4284 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b5a5a3d3-2cfe-4853-bdc3-1401e2b055a2" containerName="istio-proxy"
Oct 31 00:40:42.231688 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:42.231564    4284 cpu_manager.go:414] "containerMap: removing container" podUID="441dd734-1e3c-40f7-a8e8-c8fbd0863d6e" containerName="payload-processing"
Oct 31 00:40:42.231688 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:42.231569    4284 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="441dd734-1e3c-40f7-a8e8-c8fbd0863d6e" containerName="payload-processing"
Oct 31 00:40:42.231688 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:42.231574    4284 cpu_manager.go:414] "containerMap: removing container" podUID="b5a5a3d3-2cfe-4853-bdc3-1401e2b055a2" containerName="istio-proxy"
Oct 31 00:40:42.231688 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:42.231578    4284 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b5a5a3d3-2cfe-4853-bdc3-1401e2b055a2" containerName="istio-proxy"
Oct 31 00:40:42.231688 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:42.231587    4284 cpu_manager.go:414] "containerMap: removing container" podUID="cf5cae5a-b883-47db-8160-af723f17e17a" containerName="maas-api"
Oct 31 00:40:42.231688 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:42.231591    4284 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="cf5cae5a-b883-47db-8160-af723f17e17a" containerName="maas-api"
Oct 31 00:40:42.231688 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:42.231652    4284 memory_manager.go:365] "RemoveStaleState removing state" podUID="b5a5a3d3-2cfe-4853-bdc3-1401e2b055a2" containerName="istio-proxy"
Oct 31 00:40:42.231688 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:42.231659    4284 memory_manager.go:365] "RemoveStaleState removing state" podUID="441dd734-1e3c-40f7-a8e8-c8fbd0863d6e" containerName="payload-processing"
Oct 31 00:40:42.231688 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:42.231665    4284 memory_manager.go:365] "RemoveStaleState removing state" podUID="b5a5a3d3-2cfe-4853-bdc3-1401e2b055a2" containerName="istio-proxy"
Oct 31 00:40:42.231688 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:42.231671    4284 memory_manager.go:365] "RemoveStaleState removing state" podUID="cf5cae5a-b883-47db-8160-af723f17e17a" containerName="maas-api"
Oct 31 00:40:42.231688 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:42.231677    4284 memory_manager.go:365] "RemoveStaleState removing state" podUID="9fb753b3-95a2-41cf-b736-8244616735a1" containerName="payload-processing"
Oct 31 00:40:42.231688 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:42.231682    4284 memory_manager.go:365] "RemoveStaleState removing state" podUID="5f7bb018-65c5-4f84-ac0b-d6d303cbbb22" containerName="maas-api"
Oct 31 00:40:42.231688 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:42.231689    4284 memory_manager.go:365] "RemoveStaleState removing state" podUID="f2147b56-458f-4d40-bbbe-75e90a4c1093" containerName="payload-pre-processing"
Oct 31 00:40:42.237329 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:42.237304    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-knm5g"
Oct 31 00:40:42.296793 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:42.296762    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-knm5g"]
Oct 31 00:40:42.308178 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:42.308149    4284 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w4-ef2572-openshift-default-dockercfg-fvb72\""
Oct 31 00:40:42.314807 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:42.314763    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:40:42.340565 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:42.340540    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a75a6e60-7634-4ae3-ac95-f5cdd33c1af8-istio-token\") pod \"e2e-worker-w4-ef2572-openshift-default-7649dd87-knm5g\" (UID: \"a75a6e60-7634-4ae3-ac95-f5cdd33c1af8\") " pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-knm5g"
Oct 31 00:40:42.340728 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:42.340576    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a75a6e60-7634-4ae3-ac95-f5cdd33c1af8-istio-data\") pod \"e2e-worker-w4-ef2572-openshift-default-7649dd87-knm5g\" (UID: \"a75a6e60-7634-4ae3-ac95-f5cdd33c1af8\") " pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-knm5g"
Oct 31 00:40:42.340728 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:42.340601    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a75a6e60-7634-4ae3-ac95-f5cdd33c1af8-workload-socket\") pod \"e2e-worker-w4-ef2572-openshift-default-7649dd87-knm5g\" (UID: \"a75a6e60-7634-4ae3-ac95-f5cdd33c1af8\") " pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-knm5g"
Oct 31 00:40:42.340856 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:42.340765    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a75a6e60-7634-4ae3-ac95-f5cdd33c1af8-istio-envoy\") pod \"e2e-worker-w4-ef2572-openshift-default-7649dd87-knm5g\" (UID: \"a75a6e60-7634-4ae3-ac95-f5cdd33c1af8\") " pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-knm5g"
Oct 31 00:40:42.340856 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:42.340841    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a75a6e60-7634-4ae3-ac95-f5cdd33c1af8-workload-certs\") pod \"e2e-worker-w4-ef2572-openshift-default-7649dd87-knm5g\" (UID: \"a75a6e60-7634-4ae3-ac95-f5cdd33c1af8\") " pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-knm5g"
Oct 31 00:40:42.340950 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:42.340896    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a75a6e60-7634-4ae3-ac95-f5cdd33c1af8-credential-socket\") pod \"e2e-worker-w4-ef2572-openshift-default-7649dd87-knm5g\" (UID: \"a75a6e60-7634-4ae3-ac95-f5cdd33c1af8\") " pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-knm5g"
Oct 31 00:40:42.340988 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:42.340947    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a75a6e60-7634-4ae3-ac95-f5cdd33c1af8-istiod-ca-cert\") pod \"e2e-worker-w4-ef2572-openshift-default-7649dd87-knm5g\" (UID: \"a75a6e60-7634-4ae3-ac95-f5cdd33c1af8\") " pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-knm5g"
Oct 31 00:40:42.340988 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:42.340974    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a75a6e60-7634-4ae3-ac95-f5cdd33c1af8-istio-podinfo\") pod \"e2e-worker-w4-ef2572-openshift-default-7649dd87-knm5g\" (UID: \"a75a6e60-7634-4ae3-ac95-f5cdd33c1af8\") " pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-knm5g"
Oct 31 00:40:42.341068 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:42.341005    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zdm5g\" (UniqueName: \"kubernetes.io/projected/a75a6e60-7634-4ae3-ac95-f5cdd33c1af8-kube-api-access-zdm5g\") pod \"e2e-worker-w4-ef2572-openshift-default-7649dd87-knm5g\" (UID: \"a75a6e60-7634-4ae3-ac95-f5cdd33c1af8\") " pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-knm5g"
Oct 31 00:40:42.441759 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:42.441719    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zdm5g\" (UniqueName: \"kubernetes.io/projected/a75a6e60-7634-4ae3-ac95-f5cdd33c1af8-kube-api-access-zdm5g\") pod \"e2e-worker-w4-ef2572-openshift-default-7649dd87-knm5g\" (UID: \"a75a6e60-7634-4ae3-ac95-f5cdd33c1af8\") " pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-knm5g"
Oct 31 00:40:42.441759 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:42.441764    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a75a6e60-7634-4ae3-ac95-f5cdd33c1af8-istio-token\") pod \"e2e-worker-w4-ef2572-openshift-default-7649dd87-knm5g\" (UID: \"a75a6e60-7634-4ae3-ac95-f5cdd33c1af8\") " pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-knm5g"
Oct 31 00:40:42.441973 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:42.441872    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a75a6e60-7634-4ae3-ac95-f5cdd33c1af8-istio-data\") pod \"e2e-worker-w4-ef2572-openshift-default-7649dd87-knm5g\" (UID: \"a75a6e60-7634-4ae3-ac95-f5cdd33c1af8\") " pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-knm5g"
Oct 31 00:40:42.441973 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:42.441913    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a75a6e60-7634-4ae3-ac95-f5cdd33c1af8-workload-socket\") pod \"e2e-worker-w4-ef2572-openshift-default-7649dd87-knm5g\" (UID: \"a75a6e60-7634-4ae3-ac95-f5cdd33c1af8\") " pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-knm5g"
Oct 31 00:40:42.441973 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:42.441967    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a75a6e60-7634-4ae3-ac95-f5cdd33c1af8-istio-envoy\") pod \"e2e-worker-w4-ef2572-openshift-default-7649dd87-knm5g\" (UID: \"a75a6e60-7634-4ae3-ac95-f5cdd33c1af8\") " pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-knm5g"
Oct 31 00:40:42.442093 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:42.442009    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a75a6e60-7634-4ae3-ac95-f5cdd33c1af8-workload-certs\") pod \"e2e-worker-w4-ef2572-openshift-default-7649dd87-knm5g\" (UID: \"a75a6e60-7634-4ae3-ac95-f5cdd33c1af8\") " pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-knm5g"
Oct 31 00:40:42.442093 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:42.442073    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a75a6e60-7634-4ae3-ac95-f5cdd33c1af8-credential-socket\") pod \"e2e-worker-w4-ef2572-openshift-default-7649dd87-knm5g\" (UID: \"a75a6e60-7634-4ae3-ac95-f5cdd33c1af8\") " pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-knm5g"
Oct 31 00:40:42.442240 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:42.442109    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a75a6e60-7634-4ae3-ac95-f5cdd33c1af8-istiod-ca-cert\") pod \"e2e-worker-w4-ef2572-openshift-default-7649dd87-knm5g\" (UID: \"a75a6e60-7634-4ae3-ac95-f5cdd33c1af8\") " pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-knm5g"
Oct 31 00:40:42.442240 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:42.442136    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a75a6e60-7634-4ae3-ac95-f5cdd33c1af8-istio-podinfo\") pod \"e2e-worker-w4-ef2572-openshift-default-7649dd87-knm5g\" (UID: \"a75a6e60-7634-4ae3-ac95-f5cdd33c1af8\") " pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-knm5g"
Oct 31 00:40:42.442345 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:42.442327    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a75a6e60-7634-4ae3-ac95-f5cdd33c1af8-istio-data\") pod \"e2e-worker-w4-ef2572-openshift-default-7649dd87-knm5g\" (UID: \"a75a6e60-7634-4ae3-ac95-f5cdd33c1af8\") " pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-knm5g"
Oct 31 00:40:42.442418 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:42.442399    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a75a6e60-7634-4ae3-ac95-f5cdd33c1af8-workload-socket\") pod \"e2e-worker-w4-ef2572-openshift-default-7649dd87-knm5g\" (UID: \"a75a6e60-7634-4ae3-ac95-f5cdd33c1af8\") " pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-knm5g"
Oct 31 00:40:42.442529 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:42.442507    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a75a6e60-7634-4ae3-ac95-f5cdd33c1af8-workload-certs\") pod \"e2e-worker-w4-ef2572-openshift-default-7649dd87-knm5g\" (UID: \"a75a6e60-7634-4ae3-ac95-f5cdd33c1af8\") " pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-knm5g"
Oct 31 00:40:42.442585 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:42.442559    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a75a6e60-7634-4ae3-ac95-f5cdd33c1af8-credential-socket\") pod \"e2e-worker-w4-ef2572-openshift-default-7649dd87-knm5g\" (UID: \"a75a6e60-7634-4ae3-ac95-f5cdd33c1af8\") " pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-knm5g"
Oct 31 00:40:42.442817 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:42.442800    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a75a6e60-7634-4ae3-ac95-f5cdd33c1af8-istiod-ca-cert\") pod \"e2e-worker-w4-ef2572-openshift-default-7649dd87-knm5g\" (UID: \"a75a6e60-7634-4ae3-ac95-f5cdd33c1af8\") " pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-knm5g"
Oct 31 00:40:42.444291 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:42.444274    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a75a6e60-7634-4ae3-ac95-f5cdd33c1af8-istio-envoy\") pod \"e2e-worker-w4-ef2572-openshift-default-7649dd87-knm5g\" (UID: \"a75a6e60-7634-4ae3-ac95-f5cdd33c1af8\") " pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-knm5g"
Oct 31 00:40:42.444588 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:42.444573    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a75a6e60-7634-4ae3-ac95-f5cdd33c1af8-istio-podinfo\") pod \"e2e-worker-w4-ef2572-openshift-default-7649dd87-knm5g\" (UID: \"a75a6e60-7634-4ae3-ac95-f5cdd33c1af8\") " pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-knm5g"
Oct 31 00:40:42.500456 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:42.500394    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a75a6e60-7634-4ae3-ac95-f5cdd33c1af8-istio-token\") pod \"e2e-worker-w4-ef2572-openshift-default-7649dd87-knm5g\" (UID: \"a75a6e60-7634-4ae3-ac95-f5cdd33c1af8\") " pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-knm5g"
Oct 31 00:40:42.501029 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:42.501003    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zdm5g\" (UniqueName: \"kubernetes.io/projected/a75a6e60-7634-4ae3-ac95-f5cdd33c1af8-kube-api-access-zdm5g\") pod \"e2e-worker-w4-ef2572-openshift-default-7649dd87-knm5g\" (UID: \"a75a6e60-7634-4ae3-ac95-f5cdd33c1af8\") " pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-knm5g"
Oct 31 00:40:42.510385 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:42.510359    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:40:42.559410 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:42.559378    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-knm5g"
Oct 31 00:40:42.660142 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:42.660104    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:40:42.989337 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:42.989315    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-knm5g"]
Oct 31 00:40:42.990176 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:40:42.990152    4284 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda75a6e60_7634_4ae3_ac95_f5cdd33c1af8.slice/crio-88653e6d71a2ea9159d842eee29c3371d9b07c8fc7d46b87a229e56ccb7d898a WatchSource:0}: Error finding container 88653e6d71a2ea9159d842eee29c3371d9b07c8fc7d46b87a229e56ccb7d898a: Status 404 returned error can't find the container with id 88653e6d71a2ea9159d842eee29c3371d9b07c8fc7d46b87a229e56ccb7d898a
Oct 31 00:40:42.992452 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:42.992422    4284 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 00:40:42.992526 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:42.992488    4284 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 00:40:42.992526 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:42.992516    4284 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 00:40:43.166186 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:43.166153    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:40:43.186940 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:43.186896    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-knm5g" event={"ID":"a75a6e60-7634-4ae3-ac95-f5cdd33c1af8","Type":"ContainerStarted","Data":"c285f65ea39e06cf577b1d0d137fe831a23be8ad8243cd2b79bb8e8b9a1b7c23"}
Oct 31 00:40:43.187110 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:43.186962    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-knm5g" event={"ID":"a75a6e60-7634-4ae3-ac95-f5cdd33c1af8","Type":"ContainerStarted","Data":"88653e6d71a2ea9159d842eee29c3371d9b07c8fc7d46b87a229e56ccb7d898a"}
Oct 31 00:40:43.559654 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:43.559618    4284 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-knm5g"
Oct 31 00:40:43.697515 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:43.697464    4284 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-knm5g" podStartSLOduration=1.697445745 podStartE2EDuration="1.697445745s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 00:40:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:40:43.303261008 +0000 UTC m=+757.448267954" watchObservedRunningTime="2026-10-31 00:40:43.697445745 +0000 UTC m=+757.842452663"
Oct 31 00:40:43.698131 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:43.698116    4284 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w1-b63c24-openshift-default-d8c64855f-6mgf7"]
Oct 31 00:40:43.701946 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:43.701921    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-b63c24-openshift-default-d8c64855f-6mgf7"
Oct 31 00:40:43.751972 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:43.751942    4284 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w1-b63c24-openshift-default-dockercfg-58m59\""
Oct 31 00:40:43.760864 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:43.760836    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-b63c24-openshift-default-d8c64855f-6mgf7"]
Oct 31 00:40:43.789003 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:43.788964    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:40:43.855774 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:43.855683    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ccf1138e-2e92-4ab6-bfdd-9979f0b69893-workload-certs\") pod \"e2e-worker-w1-b63c24-openshift-default-d8c64855f-6mgf7\" (UID: \"ccf1138e-2e92-4ab6-bfdd-9979f0b69893\") " pod="openshift-ingress/e2e-worker-w1-b63c24-openshift-default-d8c64855f-6mgf7"
Oct 31 00:40:43.855774 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:43.855745    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ccf1138e-2e92-4ab6-bfdd-9979f0b69893-credential-socket\") pod \"e2e-worker-w1-b63c24-openshift-default-d8c64855f-6mgf7\" (UID: \"ccf1138e-2e92-4ab6-bfdd-9979f0b69893\") " pod="openshift-ingress/e2e-worker-w1-b63c24-openshift-default-d8c64855f-6mgf7"
Oct 31 00:40:43.855992 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:43.855906    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ccf1138e-2e92-4ab6-bfdd-9979f0b69893-istio-data\") pod \"e2e-worker-w1-b63c24-openshift-default-d8c64855f-6mgf7\" (UID: \"ccf1138e-2e92-4ab6-bfdd-9979f0b69893\") " pod="openshift-ingress/e2e-worker-w1-b63c24-openshift-default-d8c64855f-6mgf7"
Oct 31 00:40:43.856049 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:43.856002    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ccf1138e-2e92-4ab6-bfdd-9979f0b69893-istio-envoy\") pod \"e2e-worker-w1-b63c24-openshift-default-d8c64855f-6mgf7\" (UID: \"ccf1138e-2e92-4ab6-bfdd-9979f0b69893\") " pod="openshift-ingress/e2e-worker-w1-b63c24-openshift-default-d8c64855f-6mgf7"
Oct 31 00:40:43.856092 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:43.856050    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-47zsm\" (UniqueName: \"kubernetes.io/projected/ccf1138e-2e92-4ab6-bfdd-9979f0b69893-kube-api-access-47zsm\") pod \"e2e-worker-w1-b63c24-openshift-default-d8c64855f-6mgf7\" (UID: \"ccf1138e-2e92-4ab6-bfdd-9979f0b69893\") " pod="openshift-ingress/e2e-worker-w1-b63c24-openshift-default-d8c64855f-6mgf7"
Oct 31 00:40:43.856138 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:43.856110    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ccf1138e-2e92-4ab6-bfdd-9979f0b69893-istio-podinfo\") pod \"e2e-worker-w1-b63c24-openshift-default-d8c64855f-6mgf7\" (UID: \"ccf1138e-2e92-4ab6-bfdd-9979f0b69893\") " pod="openshift-ingress/e2e-worker-w1-b63c24-openshift-default-d8c64855f-6mgf7"
Oct 31 00:40:43.856189 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:43.856152    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ccf1138e-2e92-4ab6-bfdd-9979f0b69893-workload-socket\") pod \"e2e-worker-w1-b63c24-openshift-default-d8c64855f-6mgf7\" (UID: \"ccf1138e-2e92-4ab6-bfdd-9979f0b69893\") " pod="openshift-ingress/e2e-worker-w1-b63c24-openshift-default-d8c64855f-6mgf7"
Oct 31 00:40:43.856260 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:43.856201    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ccf1138e-2e92-4ab6-bfdd-9979f0b69893-istio-token\") pod \"e2e-worker-w1-b63c24-openshift-default-d8c64855f-6mgf7\" (UID: \"ccf1138e-2e92-4ab6-bfdd-9979f0b69893\") " pod="openshift-ingress/e2e-worker-w1-b63c24-openshift-default-d8c64855f-6mgf7"
Oct 31 00:40:43.856314 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:43.856274    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ccf1138e-2e92-4ab6-bfdd-9979f0b69893-istiod-ca-cert\") pod \"e2e-worker-w1-b63c24-openshift-default-d8c64855f-6mgf7\" (UID: \"ccf1138e-2e92-4ab6-bfdd-9979f0b69893\") " pod="openshift-ingress/e2e-worker-w1-b63c24-openshift-default-d8c64855f-6mgf7"
Oct 31 00:40:43.957733 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:43.957697    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ccf1138e-2e92-4ab6-bfdd-9979f0b69893-istio-data\") pod \"e2e-worker-w1-b63c24-openshift-default-d8c64855f-6mgf7\" (UID: \"ccf1138e-2e92-4ab6-bfdd-9979f0b69893\") " pod="openshift-ingress/e2e-worker-w1-b63c24-openshift-default-d8c64855f-6mgf7"
Oct 31 00:40:43.957923 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:43.957739    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ccf1138e-2e92-4ab6-bfdd-9979f0b69893-istio-envoy\") pod \"e2e-worker-w1-b63c24-openshift-default-d8c64855f-6mgf7\" (UID: \"ccf1138e-2e92-4ab6-bfdd-9979f0b69893\") " pod="openshift-ingress/e2e-worker-w1-b63c24-openshift-default-d8c64855f-6mgf7"
Oct 31 00:40:43.957923 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:43.957766    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-47zsm\" (UniqueName: \"kubernetes.io/projected/ccf1138e-2e92-4ab6-bfdd-9979f0b69893-kube-api-access-47zsm\") pod \"e2e-worker-w1-b63c24-openshift-default-d8c64855f-6mgf7\" (UID: \"ccf1138e-2e92-4ab6-bfdd-9979f0b69893\") " pod="openshift-ingress/e2e-worker-w1-b63c24-openshift-default-d8c64855f-6mgf7"
Oct 31 00:40:43.957923 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:43.957814    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ccf1138e-2e92-4ab6-bfdd-9979f0b69893-istio-podinfo\") pod \"e2e-worker-w1-b63c24-openshift-default-d8c64855f-6mgf7\" (UID: \"ccf1138e-2e92-4ab6-bfdd-9979f0b69893\") " pod="openshift-ingress/e2e-worker-w1-b63c24-openshift-default-d8c64855f-6mgf7"
Oct 31 00:40:43.957923 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:43.957852    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ccf1138e-2e92-4ab6-bfdd-9979f0b69893-workload-socket\") pod \"e2e-worker-w1-b63c24-openshift-default-d8c64855f-6mgf7\" (UID: \"ccf1138e-2e92-4ab6-bfdd-9979f0b69893\") " pod="openshift-ingress/e2e-worker-w1-b63c24-openshift-default-d8c64855f-6mgf7"
Oct 31 00:40:43.957923 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:43.957895    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ccf1138e-2e92-4ab6-bfdd-9979f0b69893-istio-token\") pod \"e2e-worker-w1-b63c24-openshift-default-d8c64855f-6mgf7\" (UID: \"ccf1138e-2e92-4ab6-bfdd-9979f0b69893\") " pod="openshift-ingress/e2e-worker-w1-b63c24-openshift-default-d8c64855f-6mgf7"
Oct 31 00:40:43.958207 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:43.957927    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ccf1138e-2e92-4ab6-bfdd-9979f0b69893-istiod-ca-cert\") pod \"e2e-worker-w1-b63c24-openshift-default-d8c64855f-6mgf7\" (UID: \"ccf1138e-2e92-4ab6-bfdd-9979f0b69893\") " pod="openshift-ingress/e2e-worker-w1-b63c24-openshift-default-d8c64855f-6mgf7"
Oct 31 00:40:43.958207 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:43.957963    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ccf1138e-2e92-4ab6-bfdd-9979f0b69893-workload-certs\") pod \"e2e-worker-w1-b63c24-openshift-default-d8c64855f-6mgf7\" (UID: \"ccf1138e-2e92-4ab6-bfdd-9979f0b69893\") " pod="openshift-ingress/e2e-worker-w1-b63c24-openshift-default-d8c64855f-6mgf7"
Oct 31 00:40:43.958207 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:43.957996    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ccf1138e-2e92-4ab6-bfdd-9979f0b69893-credential-socket\") pod \"e2e-worker-w1-b63c24-openshift-default-d8c64855f-6mgf7\" (UID: \"ccf1138e-2e92-4ab6-bfdd-9979f0b69893\") " pod="openshift-ingress/e2e-worker-w1-b63c24-openshift-default-d8c64855f-6mgf7"
Oct 31 00:40:43.958392 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:43.958234    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ccf1138e-2e92-4ab6-bfdd-9979f0b69893-workload-socket\") pod \"e2e-worker-w1-b63c24-openshift-default-d8c64855f-6mgf7\" (UID: \"ccf1138e-2e92-4ab6-bfdd-9979f0b69893\") " pod="openshift-ingress/e2e-worker-w1-b63c24-openshift-default-d8c64855f-6mgf7"
Oct 31 00:40:43.958392 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:43.958338    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ccf1138e-2e92-4ab6-bfdd-9979f0b69893-credential-socket\") pod \"e2e-worker-w1-b63c24-openshift-default-d8c64855f-6mgf7\" (UID: \"ccf1138e-2e92-4ab6-bfdd-9979f0b69893\") " pod="openshift-ingress/e2e-worker-w1-b63c24-openshift-default-d8c64855f-6mgf7"
Oct 31 00:40:43.958589 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:43.958562    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ccf1138e-2e92-4ab6-bfdd-9979f0b69893-workload-certs\") pod \"e2e-worker-w1-b63c24-openshift-default-d8c64855f-6mgf7\" (UID: \"ccf1138e-2e92-4ab6-bfdd-9979f0b69893\") " pod="openshift-ingress/e2e-worker-w1-b63c24-openshift-default-d8c64855f-6mgf7"
Oct 31 00:40:43.958766 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:43.958746    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ccf1138e-2e92-4ab6-bfdd-9979f0b69893-istio-data\") pod \"e2e-worker-w1-b63c24-openshift-default-d8c64855f-6mgf7\" (UID: \"ccf1138e-2e92-4ab6-bfdd-9979f0b69893\") " pod="openshift-ingress/e2e-worker-w1-b63c24-openshift-default-d8c64855f-6mgf7"
Oct 31 00:40:43.958873 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:43.958850    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ccf1138e-2e92-4ab6-bfdd-9979f0b69893-istiod-ca-cert\") pod \"e2e-worker-w1-b63c24-openshift-default-d8c64855f-6mgf7\" (UID: \"ccf1138e-2e92-4ab6-bfdd-9979f0b69893\") " pod="openshift-ingress/e2e-worker-w1-b63c24-openshift-default-d8c64855f-6mgf7"
Oct 31 00:40:43.961117 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:43.961093    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ccf1138e-2e92-4ab6-bfdd-9979f0b69893-istio-envoy\") pod \"e2e-worker-w1-b63c24-openshift-default-d8c64855f-6mgf7\" (UID: \"ccf1138e-2e92-4ab6-bfdd-9979f0b69893\") " pod="openshift-ingress/e2e-worker-w1-b63c24-openshift-default-d8c64855f-6mgf7"
Oct 31 00:40:43.961348 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:43.961325    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ccf1138e-2e92-4ab6-bfdd-9979f0b69893-istio-podinfo\") pod \"e2e-worker-w1-b63c24-openshift-default-d8c64855f-6mgf7\" (UID: \"ccf1138e-2e92-4ab6-bfdd-9979f0b69893\") " pod="openshift-ingress/e2e-worker-w1-b63c24-openshift-default-d8c64855f-6mgf7"
Oct 31 00:40:44.036643 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:44.036615    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ccf1138e-2e92-4ab6-bfdd-9979f0b69893-istio-token\") pod \"e2e-worker-w1-b63c24-openshift-default-d8c64855f-6mgf7\" (UID: \"ccf1138e-2e92-4ab6-bfdd-9979f0b69893\") " pod="openshift-ingress/e2e-worker-w1-b63c24-openshift-default-d8c64855f-6mgf7"
Oct 31 00:40:44.037006 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:44.036745    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-47zsm\" (UniqueName: \"kubernetes.io/projected/ccf1138e-2e92-4ab6-bfdd-9979f0b69893-kube-api-access-47zsm\") pod \"e2e-worker-w1-b63c24-openshift-default-d8c64855f-6mgf7\" (UID: \"ccf1138e-2e92-4ab6-bfdd-9979f0b69893\") " pod="openshift-ingress/e2e-worker-w1-b63c24-openshift-default-d8c64855f-6mgf7"
Oct 31 00:40:44.111520 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:44.111438    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:40:44.315178 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:44.315149    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-b63c24-openshift-default-d8c64855f-6mgf7"
Oct 31 00:40:44.550098 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:44.550064    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-30d0c8aa-6888fdf64d-j7gv6"]
Oct 31 00:40:44.550299 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:44.550276    4284 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-30d0c8aa-6888fdf64d-j7gv6" podUID="768ad3f4-6a42-468a-ad3a-7dc92fb234c4" containerName="maas-api" containerID="cri-o://2bc67ce86ebb30152dc36d5b5e9fc06c4d697b0e9d41cc2b4615c6b0721f3224" gracePeriod=30
Oct 31 00:40:44.566486 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:44.566461    4284 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-knm5g"
Oct 31 00:40:44.657445 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:44.657415    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:40:44.736029 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:44.736000    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-b63c24-openshift-default-d8c64855f-6mgf7"]
Oct 31 00:40:44.740531 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:40:44.740499    4284 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podccf1138e_2e92_4ab6_bfdd_9979f0b69893.slice/crio-c2aef73553e57cfe4e5f206d1c110cbd918a746ac604fac8c08d49fba6c6f819 WatchSource:0}: Error finding container c2aef73553e57cfe4e5f206d1c110cbd918a746ac604fac8c08d49fba6c6f819: Status 404 returned error can't find the container with id c2aef73553e57cfe4e5f206d1c110cbd918a746ac604fac8c08d49fba6c6f819
Oct 31 00:40:44.743079 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:44.743046    4284 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 00:40:44.743184 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:44.743129    4284 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 00:40:44.743184 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:44.743174    4284 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 00:40:44.851999 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:44.851972    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-30d0c8aa-6888fdf64d-j7gv6"
Oct 31 00:40:44.976777 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:44.976683    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/768ad3f4-6a42-468a-ad3a-7dc92fb234c4-metrics-certs\" (UniqueName: \"kubernetes.io/secret/768ad3f4-6a42-468a-ad3a-7dc92fb234c4-metrics-certs\") pod \"768ad3f4-6a42-468a-ad3a-7dc92fb234c4\" (UID: \"768ad3f4-6a42-468a-ad3a-7dc92fb234c4\") "
Oct 31 00:40:44.976777 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:44.976730    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/768ad3f4-6a42-468a-ad3a-7dc92fb234c4-kube-api-access-wrxqf\" (UniqueName: \"kubernetes.io/projected/768ad3f4-6a42-468a-ad3a-7dc92fb234c4-kube-api-access-wrxqf\") pod \"768ad3f4-6a42-468a-ad3a-7dc92fb234c4\" (UID: \"768ad3f4-6a42-468a-ad3a-7dc92fb234c4\") "
Oct 31 00:40:44.977008 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:44.976843    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/768ad3f4-6a42-468a-ad3a-7dc92fb234c4-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/768ad3f4-6a42-468a-ad3a-7dc92fb234c4-maas-api-tls\") pod \"768ad3f4-6a42-468a-ad3a-7dc92fb234c4\" (UID: \"768ad3f4-6a42-468a-ad3a-7dc92fb234c4\") "
Oct 31 00:40:44.977008 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:44.976884    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/768ad3f4-6a42-468a-ad3a-7dc92fb234c4-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/768ad3f4-6a42-468a-ad3a-7dc92fb234c4-openshift-service-ca\") pod \"768ad3f4-6a42-468a-ad3a-7dc92fb234c4\" (UID: \"768ad3f4-6a42-468a-ad3a-7dc92fb234c4\") "
Oct 31 00:40:44.981203 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:44.977536    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/768ad3f4-6a42-468a-ad3a-7dc92fb234c4-openshift-service-ca" pod "768ad3f4-6a42-468a-ad3a-7dc92fb234c4" (UID: "768ad3f4-6a42-468a-ad3a-7dc92fb234c4"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 00:40:44.981203 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:44.980340    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/768ad3f4-6a42-468a-ad3a-7dc92fb234c4-metrics-certs" pod "768ad3f4-6a42-468a-ad3a-7dc92fb234c4" (UID: "768ad3f4-6a42-468a-ad3a-7dc92fb234c4"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 00:40:44.981203 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:44.980485    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/768ad3f4-6a42-468a-ad3a-7dc92fb234c4-maas-api-tls" pod "768ad3f4-6a42-468a-ad3a-7dc92fb234c4" (UID: "768ad3f4-6a42-468a-ad3a-7dc92fb234c4"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 00:40:44.981203 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:44.980624    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/768ad3f4-6a42-468a-ad3a-7dc92fb234c4-kube-api-access-wrxqf" pod "768ad3f4-6a42-468a-ad3a-7dc92fb234c4" (UID: "768ad3f4-6a42-468a-ad3a-7dc92fb234c4"). InnerVolumeSpecName "kube-api-access-wrxqf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 00:40:45.078613 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:45.078395    4284 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/768ad3f4-6a42-468a-ad3a-7dc92fb234c4-metrics-certs\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:40:45.078613 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:45.078430    4284 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wrxqf\" (UniqueName: \"kubernetes.io/projected/768ad3f4-6a42-468a-ad3a-7dc92fb234c4-kube-api-access-wrxqf\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:40:45.078613 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:45.078446    4284 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/768ad3f4-6a42-468a-ad3a-7dc92fb234c4-maas-api-tls\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:40:45.078613 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:45.078460    4284 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/768ad3f4-6a42-468a-ad3a-7dc92fb234c4-openshift-service-ca\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:40:45.198972 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:45.198937    4284 generic.go:356] "Generic (PLEG): container finished" podID="768ad3f4-6a42-468a-ad3a-7dc92fb234c4" containerID="2bc67ce86ebb30152dc36d5b5e9fc06c4d697b0e9d41cc2b4615c6b0721f3224" exitCode=0
Oct 31 00:40:45.199153 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:45.198989    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-30d0c8aa-6888fdf64d-j7gv6"
Oct 31 00:40:45.199153 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:45.199020    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-30d0c8aa-6888fdf64d-j7gv6" event={"ID":"768ad3f4-6a42-468a-ad3a-7dc92fb234c4","Type":"ContainerDied","Data":"2bc67ce86ebb30152dc36d5b5e9fc06c4d697b0e9d41cc2b4615c6b0721f3224"}
Oct 31 00:40:45.199153 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:45.199077    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-30d0c8aa-6888fdf64d-j7gv6" event={"ID":"768ad3f4-6a42-468a-ad3a-7dc92fb234c4","Type":"ContainerDied","Data":"937d43cadd28f733ace7907dfc3ce91a8f2502f90c2aff6591b143c51e37fc10"}
Oct 31 00:40:45.199153 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:45.199107    4284 scope.go:122] "RemoveContainer" containerID="2bc67ce86ebb30152dc36d5b5e9fc06c4d697b0e9d41cc2b4615c6b0721f3224"
Oct 31 00:40:45.200869 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:45.200843    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-b63c24-openshift-default-d8c64855f-6mgf7" event={"ID":"ccf1138e-2e92-4ab6-bfdd-9979f0b69893","Type":"ContainerStarted","Data":"352c1902de2c418962324852d5b808029015706578d88f6a05bb07b1fe519795"}
Oct 31 00:40:45.200988 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:45.200888    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-b63c24-openshift-default-d8c64855f-6mgf7" event={"ID":"ccf1138e-2e92-4ab6-bfdd-9979f0b69893","Type":"ContainerStarted","Data":"c2aef73553e57cfe4e5f206d1c110cbd918a746ac604fac8c08d49fba6c6f819"}
Oct 31 00:40:45.200988 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:45.200920    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-knm5g"
Oct 31 00:40:45.202155 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:45.202134    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-knm5g"
Oct 31 00:40:45.208276 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:45.208256    4284 scope.go:122] "RemoveContainer" containerID="2bc67ce86ebb30152dc36d5b5e9fc06c4d697b0e9d41cc2b4615c6b0721f3224"
Oct 31 00:40:45.208528 ip-10-0-132-173 kubenswrapper[4284]: E1031 00:40:45.208511    4284 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2bc67ce86ebb30152dc36d5b5e9fc06c4d697b0e9d41cc2b4615c6b0721f3224\": container with ID starting with 2bc67ce86ebb30152dc36d5b5e9fc06c4d697b0e9d41cc2b4615c6b0721f3224 not found: ID does not exist" containerID="2bc67ce86ebb30152dc36d5b5e9fc06c4d697b0e9d41cc2b4615c6b0721f3224"
Oct 31 00:40:45.208576 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:45.208535    4284 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2bc67ce86ebb30152dc36d5b5e9fc06c4d697b0e9d41cc2b4615c6b0721f3224"} err="failed to get container status \"2bc67ce86ebb30152dc36d5b5e9fc06c4d697b0e9d41cc2b4615c6b0721f3224\": rpc error: code = NotFound desc = could not find container \"2bc67ce86ebb30152dc36d5b5e9fc06c4d697b0e9d41cc2b4615c6b0721f3224\": container with ID starting with 2bc67ce86ebb30152dc36d5b5e9fc06c4d697b0e9d41cc2b4615c6b0721f3224 not found: ID does not exist"
Oct 31 00:40:45.315666 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:45.315579    4284 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w1-b63c24-openshift-default-d8c64855f-6mgf7"
Oct 31 00:40:45.320824 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:45.320795    4284 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w1-b63c24-openshift-default-d8c64855f-6mgf7"
Oct 31 00:40:45.356813 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:45.356784    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-30d0c8aa-6888fdf64d-j7gv6"]
Oct 31 00:40:45.360659 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:45.360634    4284 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-30d0c8aa-6888fdf64d-j7gv6"]
Oct 31 00:40:45.475313 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:45.475263    4284 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w1-b63c24-openshift-default-d8c64855f-6mgf7" podStartSLOduration=2.475248839 podStartE2EDuration="2.475248839s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 00:40:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:40:45.474408125 +0000 UTC m=+759.619415040" watchObservedRunningTime="2026-10-31 00:40:45.475248839 +0000 UTC m=+759.620255752"
Oct 31 00:40:45.962404 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:45.962368    4284 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="768ad3f4-6a42-468a-ad3a-7dc92fb234c4" path="/var/lib/kubelet/pods/768ad3f4-6a42-468a-ad3a-7dc92fb234c4/volumes"
Oct 31 00:40:46.086042 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:46.086008    4284 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w2-29ece5-6489c5b684-f79l9"]
Oct 31 00:40:46.086485 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:46.086415    4284 cpu_manager.go:414] "containerMap: removing container" podUID="768ad3f4-6a42-468a-ad3a-7dc92fb234c4" containerName="maas-api"
Oct 31 00:40:46.086485 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:46.086426    4284 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="768ad3f4-6a42-468a-ad3a-7dc92fb234c4" containerName="maas-api"
Oct 31 00:40:46.086563 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:46.086502    4284 memory_manager.go:365] "RemoveStaleState removing state" podUID="768ad3f4-6a42-468a-ad3a-7dc92fb234c4" containerName="maas-api"
Oct 31 00:40:46.091360 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:46.091338    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-29ece5-6489c5b684-f79l9"
Oct 31 00:40:46.143146 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:46.143116    4284 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w2-29ece5-serving-cert\""
Oct 31 00:40:46.145701 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:46.145677    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w2-29ece5-6489c5b684-f79l9"]
Oct 31 00:40:46.170074 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:46.170036    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:40:46.189415 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:46.189379    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fxktz\" (UniqueName: \"kubernetes.io/projected/b56df4cb-7bf2-4e02-ab77-f33e078aba3c-kube-api-access-fxktz\") pod \"maas-api-e2e-worker-w2-29ece5-6489c5b684-f79l9\" (UID: \"b56df4cb-7bf2-4e02-ab77-f33e078aba3c\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-29ece5-6489c5b684-f79l9"
Oct 31 00:40:46.189599 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:46.189511    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b56df4cb-7bf2-4e02-ab77-f33e078aba3c-openshift-service-ca\") pod \"maas-api-e2e-worker-w2-29ece5-6489c5b684-f79l9\" (UID: \"b56df4cb-7bf2-4e02-ab77-f33e078aba3c\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-29ece5-6489c5b684-f79l9"
Oct 31 00:40:46.189599 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:46.189544    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b56df4cb-7bf2-4e02-ab77-f33e078aba3c-metrics-certs\") pod \"maas-api-e2e-worker-w2-29ece5-6489c5b684-f79l9\" (UID: \"b56df4cb-7bf2-4e02-ab77-f33e078aba3c\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-29ece5-6489c5b684-f79l9"
Oct 31 00:40:46.189719 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:46.189681    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b56df4cb-7bf2-4e02-ab77-f33e078aba3c-maas-api-tls\") pod \"maas-api-e2e-worker-w2-29ece5-6489c5b684-f79l9\" (UID: \"b56df4cb-7bf2-4e02-ab77-f33e078aba3c\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-29ece5-6489c5b684-f79l9"
Oct 31 00:40:46.202770 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:46.202716    4284 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-95fa4445-f8bcbf959-8qrhg" podUID="8585003b-9127-409c-a2a6-e8cce1e5241f" containerName="payload-processing" containerID="cri-o://fa47acb0a086194cae4486470ed1f7e17326b33fe37a3387a57aa0661aae0089" gracePeriod=25
Oct 31 00:40:46.208307 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:46.208258    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-b63c24-openshift-default-d8c64855f-6mgf7"
Oct 31 00:40:46.209420 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:46.209398    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w1-b63c24-openshift-default-d8c64855f-6mgf7"
Oct 31 00:40:46.291082 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:46.291044    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b56df4cb-7bf2-4e02-ab77-f33e078aba3c-openshift-service-ca\") pod \"maas-api-e2e-worker-w2-29ece5-6489c5b684-f79l9\" (UID: \"b56df4cb-7bf2-4e02-ab77-f33e078aba3c\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-29ece5-6489c5b684-f79l9"
Oct 31 00:40:46.291082 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:46.291087    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b56df4cb-7bf2-4e02-ab77-f33e078aba3c-metrics-certs\") pod \"maas-api-e2e-worker-w2-29ece5-6489c5b684-f79l9\" (UID: \"b56df4cb-7bf2-4e02-ab77-f33e078aba3c\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-29ece5-6489c5b684-f79l9"
Oct 31 00:40:46.291388 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:46.291129    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b56df4cb-7bf2-4e02-ab77-f33e078aba3c-maas-api-tls\") pod \"maas-api-e2e-worker-w2-29ece5-6489c5b684-f79l9\" (UID: \"b56df4cb-7bf2-4e02-ab77-f33e078aba3c\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-29ece5-6489c5b684-f79l9"
Oct 31 00:40:46.291388 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:46.291184    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fxktz\" (UniqueName: \"kubernetes.io/projected/b56df4cb-7bf2-4e02-ab77-f33e078aba3c-kube-api-access-fxktz\") pod \"maas-api-e2e-worker-w2-29ece5-6489c5b684-f79l9\" (UID: \"b56df4cb-7bf2-4e02-ab77-f33e078aba3c\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-29ece5-6489c5b684-f79l9"
Oct 31 00:40:46.291388 ip-10-0-132-173 kubenswrapper[4284]: E1031 00:40:46.291317    4284 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-worker-w2-29ece5-serving-cert: secret "maas-api-e2e-worker-w2-29ece5-serving-cert" not found
Oct 31 00:40:46.291558 ip-10-0-132-173 kubenswrapper[4284]: E1031 00:40:46.291412    4284 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b56df4cb-7bf2-4e02-ab77-f33e078aba3c-maas-api-tls podName:b56df4cb-7bf2-4e02-ab77-f33e078aba3c nodeName:}" failed. No retries permitted until 2026-10-31 00:40:46.791388243 +0000 UTC m=+760.936395137 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/b56df4cb-7bf2-4e02-ab77-f33e078aba3c-maas-api-tls") pod "maas-api-e2e-worker-w2-29ece5-6489c5b684-f79l9" (UID: "b56df4cb-7bf2-4e02-ab77-f33e078aba3c") : secret "maas-api-e2e-worker-w2-29ece5-serving-cert" not found
Oct 31 00:40:46.291874 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:46.291853    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b56df4cb-7bf2-4e02-ab77-f33e078aba3c-openshift-service-ca\") pod \"maas-api-e2e-worker-w2-29ece5-6489c5b684-f79l9\" (UID: \"b56df4cb-7bf2-4e02-ab77-f33e078aba3c\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-29ece5-6489c5b684-f79l9"
Oct 31 00:40:46.293754 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:46.293737    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b56df4cb-7bf2-4e02-ab77-f33e078aba3c-metrics-certs\") pod \"maas-api-e2e-worker-w2-29ece5-6489c5b684-f79l9\" (UID: \"b56df4cb-7bf2-4e02-ab77-f33e078aba3c\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-29ece5-6489c5b684-f79l9"
Oct 31 00:40:46.316160 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:46.316127    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:40:46.359920 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:46.359885    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fxktz\" (UniqueName: \"kubernetes.io/projected/b56df4cb-7bf2-4e02-ab77-f33e078aba3c-kube-api-access-fxktz\") pod \"maas-api-e2e-worker-w2-29ece5-6489c5b684-f79l9\" (UID: \"b56df4cb-7bf2-4e02-ab77-f33e078aba3c\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-29ece5-6489c5b684-f79l9"
Oct 31 00:40:46.465376 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:46.465283    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:40:46.512055 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:46.512030    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-95fa4445-f8bcbf959-8qrhg"
Oct 31 00:40:46.703848 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:46.703815    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8585003b-9127-409c-a2a6-e8cce1e5241f-kube-api-access-brrl5\" (UniqueName: \"kubernetes.io/projected/8585003b-9127-409c-a2a6-e8cce1e5241f-kube-api-access-brrl5\") pod \"8585003b-9127-409c-a2a6-e8cce1e5241f\" (UID: \"8585003b-9127-409c-a2a6-e8cce1e5241f\") "
Oct 31 00:40:46.704039 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:46.703902    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/8585003b-9127-409c-a2a6-e8cce1e5241f-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8585003b-9127-409c-a2a6-e8cce1e5241f-plugins-config-volume\") pod \"8585003b-9127-409c-a2a6-e8cce1e5241f\" (UID: \"8585003b-9127-409c-a2a6-e8cce1e5241f\") "
Oct 31 00:40:46.704733 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:46.704704    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8585003b-9127-409c-a2a6-e8cce1e5241f-plugins-config-volume" pod "8585003b-9127-409c-a2a6-e8cce1e5241f" (UID: "8585003b-9127-409c-a2a6-e8cce1e5241f"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 00:40:46.704901 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:46.704853    4284 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8585003b-9127-409c-a2a6-e8cce1e5241f-plugins-config-volume\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:40:46.708967 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:46.708939    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8585003b-9127-409c-a2a6-e8cce1e5241f-kube-api-access-brrl5" pod "8585003b-9127-409c-a2a6-e8cce1e5241f" (UID: "8585003b-9127-409c-a2a6-e8cce1e5241f"). InnerVolumeSpecName "kube-api-access-brrl5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 00:40:46.777916 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:46.777817    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:40:46.805918 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:46.805878    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b56df4cb-7bf2-4e02-ab77-f33e078aba3c-maas-api-tls\") pod \"maas-api-e2e-worker-w2-29ece5-6489c5b684-f79l9\" (UID: \"b56df4cb-7bf2-4e02-ab77-f33e078aba3c\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-29ece5-6489c5b684-f79l9"
Oct 31 00:40:46.806096 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:46.806020    4284 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-brrl5\" (UniqueName: \"kubernetes.io/projected/8585003b-9127-409c-a2a6-e8cce1e5241f-kube-api-access-brrl5\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:40:46.808492 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:46.808468    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b56df4cb-7bf2-4e02-ab77-f33e078aba3c-maas-api-tls\") pod \"maas-api-e2e-worker-w2-29ece5-6489c5b684-f79l9\" (UID: \"b56df4cb-7bf2-4e02-ab77-f33e078aba3c\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-29ece5-6489c5b684-f79l9"
Oct 31 00:40:47.004549 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:47.004509    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-29ece5-6489c5b684-f79l9"
Oct 31 00:40:47.016732 ip-10-0-132-173 kubenswrapper[4284]: E1031 00:40:47.016694    4284 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fd8sp" containerName="extractor"
Oct 31 00:40:47.213812 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:47.213781    4284 generic.go:356] "Generic (PLEG): container finished" podID="8585003b-9127-409c-a2a6-e8cce1e5241f" containerID="fa47acb0a086194cae4486470ed1f7e17326b33fe37a3387a57aa0661aae0089" exitCode=0
Oct 31 00:40:47.214257 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:47.213830    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-95fa4445-f8bcbf959-8qrhg"
Oct 31 00:40:47.214257 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:47.213873    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-95fa4445-f8bcbf959-8qrhg" event={"ID":"8585003b-9127-409c-a2a6-e8cce1e5241f","Type":"ContainerDied","Data":"fa47acb0a086194cae4486470ed1f7e17326b33fe37a3387a57aa0661aae0089"}
Oct 31 00:40:47.214257 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:47.213927    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-95fa4445-f8bcbf959-8qrhg" event={"ID":"8585003b-9127-409c-a2a6-e8cce1e5241f","Type":"ContainerDied","Data":"1d1660f8de0a587754a9d84817e4232ce7a960fb159662c59714f1a779cbe64e"}
Oct 31 00:40:47.214257 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:47.213960    4284 scope.go:122] "RemoveContainer" containerID="fa47acb0a086194cae4486470ed1f7e17326b33fe37a3387a57aa0661aae0089"
Oct 31 00:40:47.222942 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:47.222919    4284 scope.go:122] "RemoveContainer" containerID="fa47acb0a086194cae4486470ed1f7e17326b33fe37a3387a57aa0661aae0089"
Oct 31 00:40:47.223200 ip-10-0-132-173 kubenswrapper[4284]: E1031 00:40:47.223185    4284 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fa47acb0a086194cae4486470ed1f7e17326b33fe37a3387a57aa0661aae0089\": container with ID starting with fa47acb0a086194cae4486470ed1f7e17326b33fe37a3387a57aa0661aae0089 not found: ID does not exist" containerID="fa47acb0a086194cae4486470ed1f7e17326b33fe37a3387a57aa0661aae0089"
Oct 31 00:40:47.223283 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:47.223232    4284 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fa47acb0a086194cae4486470ed1f7e17326b33fe37a3387a57aa0661aae0089"} err="failed to get container status \"fa47acb0a086194cae4486470ed1f7e17326b33fe37a3387a57aa0661aae0089\": rpc error: code = NotFound desc = could not find container \"fa47acb0a086194cae4486470ed1f7e17326b33fe37a3387a57aa0661aae0089\": container with ID starting with fa47acb0a086194cae4486470ed1f7e17326b33fe37a3387a57aa0661aae0089 not found: ID does not exist"
Oct 31 00:40:47.370947 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:47.370861    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-95fa4445-f8bcbf959-8qrhg"]
Oct 31 00:40:47.412139 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:40:47.412110    4284 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb56df4cb_7bf2_4e02_ab77_f33e078aba3c.slice/crio-66012d9e15a9a9eafa75da06ffe370cb071fc95e668434468eef7d45d1a0356f WatchSource:0}: Error finding container 66012d9e15a9a9eafa75da06ffe370cb071fc95e668434468eef7d45d1a0356f: Status 404 returned error can't find the container with id 66012d9e15a9a9eafa75da06ffe370cb071fc95e668434468eef7d45d1a0356f
Oct 31 00:40:47.423556 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:47.423476    4284 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-95fa4445-f8bcbf959-8qrhg"]
Oct 31 00:40:47.424617 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:47.424595    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w2-29ece5-6489c5b684-f79l9"]
Oct 31 00:40:47.576524 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:47.576493    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:40:47.769956 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:47.769915    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:40:47.961766 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:47.961735    4284 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="8585003b-9127-409c-a2a6-e8cce1e5241f" path="/var/lib/kubelet/pods/8585003b-9127-409c-a2a6-e8cce1e5241f/volumes"
Oct 31 00:40:48.003240 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:47.999518    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:40:48.221030 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:48.220995    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-29ece5-6489c5b684-f79l9" event={"ID":"b56df4cb-7bf2-4e02-ab77-f33e078aba3c","Type":"ContainerStarted","Data":"66012d9e15a9a9eafa75da06ffe370cb071fc95e668434468eef7d45d1a0356f"}
Oct 31 00:40:48.233675 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:48.233648    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:40:48.386142 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:48.386074    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:40:49.227197 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:49.227157    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-29ece5-6489c5b684-f79l9" event={"ID":"b56df4cb-7bf2-4e02-ab77-f33e078aba3c","Type":"ContainerStarted","Data":"8356bb6bc1994c667b1e13a295044df1029916b59c67ce92cd60fde76552456b"}
Oct 31 00:40:49.227596 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:49.227276    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-29ece5-6489c5b684-f79l9"
Oct 31 00:40:49.342390 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:49.342338    4284 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-29ece5-6489c5b684-f79l9" podStartSLOduration=2.4005718160000002 podStartE2EDuration="3.342325703s" totalImagesPullingTime="941.753887ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 00:40:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:40:49.340472907 +0000 UTC m=+763.485479825" watchObservedRunningTime="2026-10-31 00:40:49.342325703 +0000 UTC m=+763.487332619"
Oct 31 00:40:50.413478 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:50.413440    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:40:50.612832 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:50.612799    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:40:50.976666 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:50.976630    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:40:51.192519 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:51.192485    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:40:51.719123 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:51.719085    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:40:52.978021 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:52.977982    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:40:54.376593 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:54.376552    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:40:54.594333 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:54.594299    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:40:54.747084 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:54.747046    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:40:55.237430 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:55.237404    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-29ece5-6489c5b684-f79l9"
Oct 31 00:40:56.203334 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:56.203303    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-95fa4445-7c699f7c77-pnr6h"]
Oct 31 00:40:56.203795 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:56.203499    4284 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-95fa4445-7c699f7c77-pnr6h" podUID="f41cd753-2c38-4a4a-a1a4-ec1bbacf75bb" containerName="maas-api" containerID="cri-o://df019eab9a2fd3932e754754cd70bdfcff01d264b8eeec7a60cdcf9dec1105be" gracePeriod=30
Oct 31 00:40:56.295127 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:56.295076    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:40:56.470515 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:56.470490    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-95fa4445-7c699f7c77-pnr6h"
Oct 31 00:40:56.523957 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:56.523926    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/f41cd753-2c38-4a4a-a1a4-ec1bbacf75bb-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f41cd753-2c38-4a4a-a1a4-ec1bbacf75bb-maas-api-tls\") pod \"f41cd753-2c38-4a4a-a1a4-ec1bbacf75bb\" (UID: \"f41cd753-2c38-4a4a-a1a4-ec1bbacf75bb\") "
Oct 31 00:40:56.524137 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:56.523987    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f41cd753-2c38-4a4a-a1a4-ec1bbacf75bb-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f41cd753-2c38-4a4a-a1a4-ec1bbacf75bb-openshift-service-ca\") pod \"f41cd753-2c38-4a4a-a1a4-ec1bbacf75bb\" (UID: \"f41cd753-2c38-4a4a-a1a4-ec1bbacf75bb\") "
Oct 31 00:40:56.524137 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:56.524014    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/f41cd753-2c38-4a4a-a1a4-ec1bbacf75bb-metrics-certs\" (UniqueName: \"kubernetes.io/secret/f41cd753-2c38-4a4a-a1a4-ec1bbacf75bb-metrics-certs\") pod \"f41cd753-2c38-4a4a-a1a4-ec1bbacf75bb\" (UID: \"f41cd753-2c38-4a4a-a1a4-ec1bbacf75bb\") "
Oct 31 00:40:56.524137 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:56.524051    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f41cd753-2c38-4a4a-a1a4-ec1bbacf75bb-kube-api-access-p225q\" (UniqueName: \"kubernetes.io/projected/f41cd753-2c38-4a4a-a1a4-ec1bbacf75bb-kube-api-access-p225q\") pod \"f41cd753-2c38-4a4a-a1a4-ec1bbacf75bb\" (UID: \"f41cd753-2c38-4a4a-a1a4-ec1bbacf75bb\") "
Oct 31 00:40:56.524443 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:56.524419    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f41cd753-2c38-4a4a-a1a4-ec1bbacf75bb-openshift-service-ca" pod "f41cd753-2c38-4a4a-a1a4-ec1bbacf75bb" (UID: "f41cd753-2c38-4a4a-a1a4-ec1bbacf75bb"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 00:40:56.526348 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:56.526323    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f41cd753-2c38-4a4a-a1a4-ec1bbacf75bb-metrics-certs" pod "f41cd753-2c38-4a4a-a1a4-ec1bbacf75bb" (UID: "f41cd753-2c38-4a4a-a1a4-ec1bbacf75bb"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 00:40:56.526458 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:56.526345    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f41cd753-2c38-4a4a-a1a4-ec1bbacf75bb-maas-api-tls" pod "f41cd753-2c38-4a4a-a1a4-ec1bbacf75bb" (UID: "f41cd753-2c38-4a4a-a1a4-ec1bbacf75bb"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 00:40:56.526458 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:56.526357    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f41cd753-2c38-4a4a-a1a4-ec1bbacf75bb-kube-api-access-p225q" pod "f41cd753-2c38-4a4a-a1a4-ec1bbacf75bb" (UID: "f41cd753-2c38-4a4a-a1a4-ec1bbacf75bb"). InnerVolumeSpecName "kube-api-access-p225q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 00:40:56.625358 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:56.625316    4284 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f41cd753-2c38-4a4a-a1a4-ec1bbacf75bb-maas-api-tls\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:40:56.625358 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:56.625358    4284 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f41cd753-2c38-4a4a-a1a4-ec1bbacf75bb-openshift-service-ca\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:40:56.625558 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:56.625373    4284 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f41cd753-2c38-4a4a-a1a4-ec1bbacf75bb-metrics-certs\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:40:56.625558 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:56.625387    4284 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p225q\" (UniqueName: \"kubernetes.io/projected/f41cd753-2c38-4a4a-a1a4-ec1bbacf75bb-kube-api-access-p225q\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:40:57.268144 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:57.268109    4284 generic.go:356] "Generic (PLEG): container finished" podID="f41cd753-2c38-4a4a-a1a4-ec1bbacf75bb" containerID="df019eab9a2fd3932e754754cd70bdfcff01d264b8eeec7a60cdcf9dec1105be" exitCode=0
Oct 31 00:40:57.268676 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:57.268167    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-95fa4445-7c699f7c77-pnr6h"
Oct 31 00:40:57.268676 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:57.268195    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-95fa4445-7c699f7c77-pnr6h" event={"ID":"f41cd753-2c38-4a4a-a1a4-ec1bbacf75bb","Type":"ContainerDied","Data":"df019eab9a2fd3932e754754cd70bdfcff01d264b8eeec7a60cdcf9dec1105be"}
Oct 31 00:40:57.268676 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:57.268313    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-95fa4445-7c699f7c77-pnr6h" event={"ID":"f41cd753-2c38-4a4a-a1a4-ec1bbacf75bb","Type":"ContainerDied","Data":"5b3540ead3c98e07dae7e6343885f18f3fe20b954a3bebf2f539f2127df99b4e"}
Oct 31 00:40:57.268676 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:57.268350    4284 scope.go:122] "RemoveContainer" containerID="df019eab9a2fd3932e754754cd70bdfcff01d264b8eeec7a60cdcf9dec1105be"
Oct 31 00:40:57.278891 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:57.278872    4284 scope.go:122] "RemoveContainer" containerID="df019eab9a2fd3932e754754cd70bdfcff01d264b8eeec7a60cdcf9dec1105be"
Oct 31 00:40:57.279182 ip-10-0-132-173 kubenswrapper[4284]: E1031 00:40:57.279162    4284 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"df019eab9a2fd3932e754754cd70bdfcff01d264b8eeec7a60cdcf9dec1105be\": container with ID starting with df019eab9a2fd3932e754754cd70bdfcff01d264b8eeec7a60cdcf9dec1105be not found: ID does not exist" containerID="df019eab9a2fd3932e754754cd70bdfcff01d264b8eeec7a60cdcf9dec1105be"
Oct 31 00:40:57.279310 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:57.279190    4284 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"df019eab9a2fd3932e754754cd70bdfcff01d264b8eeec7a60cdcf9dec1105be"} err="failed to get container status \"df019eab9a2fd3932e754754cd70bdfcff01d264b8eeec7a60cdcf9dec1105be\": rpc error: code = NotFound desc = could not find container \"df019eab9a2fd3932e754754cd70bdfcff01d264b8eeec7a60cdcf9dec1105be\": container with ID starting with df019eab9a2fd3932e754754cd70bdfcff01d264b8eeec7a60cdcf9dec1105be not found: ID does not exist"
Oct 31 00:40:57.425205 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:57.425171    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-95fa4445-7c699f7c77-pnr6h"]
Oct 31 00:40:57.472732 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:57.472702    4284 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-95fa4445-7c699f7c77-pnr6h"]
Oct 31 00:40:57.907284 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:57.907256    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:40:57.962139 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:57.962096    4284 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f41cd753-2c38-4a4a-a1a4-ec1bbacf75bb" path="/var/lib/kubelet/pods/f41cd753-2c38-4a4a-a1a4-ec1bbacf75bb/volumes"
Oct 31 00:40:58.057036 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:58.056994    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:40:58.205830 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:58.205737    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:40:58.334929 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:58.334893    4284 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w1-b63c24-75cd4d9679-cpvtw"]
Oct 31 00:40:58.335344 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:58.335330    4284 cpu_manager.go:414] "containerMap: removing container" podUID="f41cd753-2c38-4a4a-a1a4-ec1bbacf75bb" containerName="maas-api"
Oct 31 00:40:58.335384 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:58.335345    4284 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f41cd753-2c38-4a4a-a1a4-ec1bbacf75bb" containerName="maas-api"
Oct 31 00:40:58.335384 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:58.335364    4284 cpu_manager.go:414] "containerMap: removing container" podUID="8585003b-9127-409c-a2a6-e8cce1e5241f" containerName="payload-processing"
Oct 31 00:40:58.335384 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:58.335369    4284 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8585003b-9127-409c-a2a6-e8cce1e5241f" containerName="payload-processing"
Oct 31 00:40:58.335475 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:58.335439    4284 memory_manager.go:365] "RemoveStaleState removing state" podUID="8585003b-9127-409c-a2a6-e8cce1e5241f" containerName="payload-processing"
Oct 31 00:40:58.335475 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:58.335455    4284 memory_manager.go:365] "RemoveStaleState removing state" podUID="f41cd753-2c38-4a4a-a1a4-ec1bbacf75bb" containerName="maas-api"
Oct 31 00:40:58.339801 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:58.339783    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-b63c24-75cd4d9679-cpvtw"
Oct 31 00:40:58.359284 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:58.359252    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:40:58.399059 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:58.399028    4284 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w1-b63c24-dockercfg-whht5\""
Oct 31 00:40:58.399395 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:58.399377    4284 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w1-b63c24\""
Oct 31 00:40:58.399609 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:58.399584    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w1-b63c24-75cd4d9679-cpvtw"]
Oct 31 00:40:58.441313 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:58.441285    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fea709fd-6209-4b40-95f6-ea5104c1cb09-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w1-b63c24-75cd4d9679-cpvtw\" (UID: \"fea709fd-6209-4b40-95f6-ea5104c1cb09\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-b63c24-75cd4d9679-cpvtw"
Oct 31 00:40:58.441747 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:58.441692    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bn2dl\" (UniqueName: \"kubernetes.io/projected/fea709fd-6209-4b40-95f6-ea5104c1cb09-kube-api-access-bn2dl\") pod \"payload-pre-processing-e2e-worker-w1-b63c24-75cd4d9679-cpvtw\" (UID: \"fea709fd-6209-4b40-95f6-ea5104c1cb09\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-b63c24-75cd4d9679-cpvtw"
Oct 31 00:40:58.542868 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:58.542782    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fea709fd-6209-4b40-95f6-ea5104c1cb09-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w1-b63c24-75cd4d9679-cpvtw\" (UID: \"fea709fd-6209-4b40-95f6-ea5104c1cb09\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-b63c24-75cd4d9679-cpvtw"
Oct 31 00:40:58.543026 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:58.542891    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bn2dl\" (UniqueName: \"kubernetes.io/projected/fea709fd-6209-4b40-95f6-ea5104c1cb09-kube-api-access-bn2dl\") pod \"payload-pre-processing-e2e-worker-w1-b63c24-75cd4d9679-cpvtw\" (UID: \"fea709fd-6209-4b40-95f6-ea5104c1cb09\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-b63c24-75cd4d9679-cpvtw"
Oct 31 00:40:58.543566 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:58.543546    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fea709fd-6209-4b40-95f6-ea5104c1cb09-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w1-b63c24-75cd4d9679-cpvtw\" (UID: \"fea709fd-6209-4b40-95f6-ea5104c1cb09\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-b63c24-75cd4d9679-cpvtw"
Oct 31 00:40:58.601683 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:58.601650    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bn2dl\" (UniqueName: \"kubernetes.io/projected/fea709fd-6209-4b40-95f6-ea5104c1cb09-kube-api-access-bn2dl\") pod \"payload-pre-processing-e2e-worker-w1-b63c24-75cd4d9679-cpvtw\" (UID: \"fea709fd-6209-4b40-95f6-ea5104c1cb09\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-b63c24-75cd4d9679-cpvtw"
Oct 31 00:40:58.653675 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:58.653638    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-b63c24-75cd4d9679-cpvtw"
Oct 31 00:40:58.975947 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:58.975909    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-95fa4445-openshift-default-c9bd77677-wz7qd"]
Oct 31 00:40:58.976201 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:58.976171    4284 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-95fa4445-openshift-default-c9bd77677-wz7qd" podUID="06987c79-2521-487f-a60d-3050698f9719" containerName="istio-proxy" containerID="cri-o://f5a24fe6446e45bf1f8d243f02e2759b02f8b8316400142e14a7c27c8c3f686c" gracePeriod=30
Oct 31 00:40:59.017607 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:59.017577    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:40:59.024151 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:59.024126    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w1-b63c24-75cd4d9679-cpvtw"]
Oct 31 00:40:59.025124 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:40:59.025099    4284 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfea709fd_6209_4b40_95f6_ea5104c1cb09.slice/crio-053a606aa32adfd9b6746aa1115497d0709dd6ead01e25768c78f9189d0271dc WatchSource:0}: Error finding container 053a606aa32adfd9b6746aa1115497d0709dd6ead01e25768c78f9189d0271dc: Status 404 returned error can't find the container with id 053a606aa32adfd9b6746aa1115497d0709dd6ead01e25768c78f9189d0271dc
Oct 31 00:40:59.281451 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:59.281352    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-b63c24-75cd4d9679-cpvtw" event={"ID":"fea709fd-6209-4b40-95f6-ea5104c1cb09","Type":"ContainerStarted","Data":"8bfc236166f01984405d4ec5670d53d05a5e6080fce6955bb83541e25c862230"}
Oct 31 00:40:59.281451 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:59.281407    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-b63c24-75cd4d9679-cpvtw" event={"ID":"fea709fd-6209-4b40-95f6-ea5104c1cb09","Type":"ContainerStarted","Data":"053a606aa32adfd9b6746aa1115497d0709dd6ead01e25768c78f9189d0271dc"}
Oct 31 00:40:59.281666 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:59.281509    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-b63c24-75cd4d9679-cpvtw"
Oct 31 00:40:59.393849 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:59.393795    4284 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-b63c24-75cd4d9679-cpvtw" podStartSLOduration=1.393779876 podStartE2EDuration="1.393779876s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 00:40:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:40:59.392689891 +0000 UTC m=+773.537696809" watchObservedRunningTime="2026-10-31 00:40:59.393779876 +0000 UTC m=+773.538786791"
Oct 31 00:40:59.891796 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:40:59.891757    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:41:00.336092 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:00.336060    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:41:00.431237 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:00.431191    4284 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-nc722"]
Oct 31 00:41:00.434792 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:00.434773    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-nc722"
Oct 31 00:41:00.506224 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:00.506168    4284 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w0-8bec73-openshift-default-dockercfg-xqgfq\""
Oct 31 00:41:00.517099 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:00.517067    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:41:00.537051 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:00.537021    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-nc722"]
Oct 31 00:41:00.562866 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:00.562831    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e999d104-c9ea-45a7-9930-7f829013c52e-workload-certs\") pod \"e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-nc722\" (UID: \"e999d104-c9ea-45a7-9930-7f829013c52e\") " pod="openshift-ingress/e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-nc722"
Oct 31 00:41:00.563008 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:00.562878    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e999d104-c9ea-45a7-9930-7f829013c52e-istio-podinfo\") pod \"e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-nc722\" (UID: \"e999d104-c9ea-45a7-9930-7f829013c52e\") " pod="openshift-ingress/e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-nc722"
Oct 31 00:41:00.563008 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:00.562903    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e999d104-c9ea-45a7-9930-7f829013c52e-credential-socket\") pod \"e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-nc722\" (UID: \"e999d104-c9ea-45a7-9930-7f829013c52e\") " pod="openshift-ingress/e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-nc722"
Oct 31 00:41:00.563008 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:00.562996    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e999d104-c9ea-45a7-9930-7f829013c52e-workload-socket\") pod \"e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-nc722\" (UID: \"e999d104-c9ea-45a7-9930-7f829013c52e\") " pod="openshift-ingress/e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-nc722"
Oct 31 00:41:00.563138 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:00.563034    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vl9hm\" (UniqueName: \"kubernetes.io/projected/e999d104-c9ea-45a7-9930-7f829013c52e-kube-api-access-vl9hm\") pod \"e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-nc722\" (UID: \"e999d104-c9ea-45a7-9930-7f829013c52e\") " pod="openshift-ingress/e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-nc722"
Oct 31 00:41:00.563138 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:00.563108    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e999d104-c9ea-45a7-9930-7f829013c52e-istiod-ca-cert\") pod \"e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-nc722\" (UID: \"e999d104-c9ea-45a7-9930-7f829013c52e\") " pod="openshift-ingress/e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-nc722"
Oct 31 00:41:00.563195 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:00.563178    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e999d104-c9ea-45a7-9930-7f829013c52e-istio-envoy\") pod \"e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-nc722\" (UID: \"e999d104-c9ea-45a7-9930-7f829013c52e\") " pod="openshift-ingress/e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-nc722"
Oct 31 00:41:00.563304 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:00.563195    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e999d104-c9ea-45a7-9930-7f829013c52e-istio-token\") pod \"e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-nc722\" (UID: \"e999d104-c9ea-45a7-9930-7f829013c52e\") " pod="openshift-ingress/e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-nc722"
Oct 31 00:41:00.563304 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:00.563241    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e999d104-c9ea-45a7-9930-7f829013c52e-istio-data\") pod \"e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-nc722\" (UID: \"e999d104-c9ea-45a7-9930-7f829013c52e\") " pod="openshift-ingress/e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-nc722"
Oct 31 00:41:00.664838 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:00.664797    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e999d104-c9ea-45a7-9930-7f829013c52e-workload-certs\") pod \"e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-nc722\" (UID: \"e999d104-c9ea-45a7-9930-7f829013c52e\") " pod="openshift-ingress/e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-nc722"
Oct 31 00:41:00.665017 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:00.664848    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e999d104-c9ea-45a7-9930-7f829013c52e-istio-podinfo\") pod \"e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-nc722\" (UID: \"e999d104-c9ea-45a7-9930-7f829013c52e\") " pod="openshift-ingress/e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-nc722"
Oct 31 00:41:00.665017 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:00.664880    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e999d104-c9ea-45a7-9930-7f829013c52e-credential-socket\") pod \"e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-nc722\" (UID: \"e999d104-c9ea-45a7-9930-7f829013c52e\") " pod="openshift-ingress/e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-nc722"
Oct 31 00:41:00.665017 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:00.664920    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e999d104-c9ea-45a7-9930-7f829013c52e-workload-socket\") pod \"e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-nc722\" (UID: \"e999d104-c9ea-45a7-9930-7f829013c52e\") " pod="openshift-ingress/e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-nc722"
Oct 31 00:41:00.665017 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:00.664962    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vl9hm\" (UniqueName: \"kubernetes.io/projected/e999d104-c9ea-45a7-9930-7f829013c52e-kube-api-access-vl9hm\") pod \"e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-nc722\" (UID: \"e999d104-c9ea-45a7-9930-7f829013c52e\") " pod="openshift-ingress/e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-nc722"
Oct 31 00:41:00.665017 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:00.664998    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e999d104-c9ea-45a7-9930-7f829013c52e-istiod-ca-cert\") pod \"e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-nc722\" (UID: \"e999d104-c9ea-45a7-9930-7f829013c52e\") " pod="openshift-ingress/e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-nc722"
Oct 31 00:41:00.665315 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:00.665042    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e999d104-c9ea-45a7-9930-7f829013c52e-istio-envoy\") pod \"e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-nc722\" (UID: \"e999d104-c9ea-45a7-9930-7f829013c52e\") " pod="openshift-ingress/e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-nc722"
Oct 31 00:41:00.665315 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:00.665067    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e999d104-c9ea-45a7-9930-7f829013c52e-istio-token\") pod \"e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-nc722\" (UID: \"e999d104-c9ea-45a7-9930-7f829013c52e\") " pod="openshift-ingress/e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-nc722"
Oct 31 00:41:00.665315 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:00.665093    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e999d104-c9ea-45a7-9930-7f829013c52e-istio-data\") pod \"e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-nc722\" (UID: \"e999d104-c9ea-45a7-9930-7f829013c52e\") " pod="openshift-ingress/e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-nc722"
Oct 31 00:41:00.665425 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:00.665318    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e999d104-c9ea-45a7-9930-7f829013c52e-workload-certs\") pod \"e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-nc722\" (UID: \"e999d104-c9ea-45a7-9930-7f829013c52e\") " pod="openshift-ingress/e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-nc722"
Oct 31 00:41:00.665800 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:00.665578    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e999d104-c9ea-45a7-9930-7f829013c52e-workload-socket\") pod \"e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-nc722\" (UID: \"e999d104-c9ea-45a7-9930-7f829013c52e\") " pod="openshift-ingress/e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-nc722"
Oct 31 00:41:00.665800 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:00.665679    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e999d104-c9ea-45a7-9930-7f829013c52e-istio-data\") pod \"e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-nc722\" (UID: \"e999d104-c9ea-45a7-9930-7f829013c52e\") " pod="openshift-ingress/e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-nc722"
Oct 31 00:41:00.665800 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:00.665711    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e999d104-c9ea-45a7-9930-7f829013c52e-credential-socket\") pod \"e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-nc722\" (UID: \"e999d104-c9ea-45a7-9930-7f829013c52e\") " pod="openshift-ingress/e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-nc722"
Oct 31 00:41:00.666053 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:00.666033    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e999d104-c9ea-45a7-9930-7f829013c52e-istiod-ca-cert\") pod \"e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-nc722\" (UID: \"e999d104-c9ea-45a7-9930-7f829013c52e\") " pod="openshift-ingress/e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-nc722"
Oct 31 00:41:00.667728 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:00.667706    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e999d104-c9ea-45a7-9930-7f829013c52e-istio-envoy\") pod \"e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-nc722\" (UID: \"e999d104-c9ea-45a7-9930-7f829013c52e\") " pod="openshift-ingress/e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-nc722"
Oct 31 00:41:00.668305 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:00.668283    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e999d104-c9ea-45a7-9930-7f829013c52e-istio-podinfo\") pod \"e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-nc722\" (UID: \"e999d104-c9ea-45a7-9930-7f829013c52e\") " pod="openshift-ingress/e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-nc722"
Oct 31 00:41:00.669203 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:00.669180    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:41:00.723756 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:00.723727    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e999d104-c9ea-45a7-9930-7f829013c52e-istio-token\") pod \"e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-nc722\" (UID: \"e999d104-c9ea-45a7-9930-7f829013c52e\") " pod="openshift-ingress/e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-nc722"
Oct 31 00:41:00.723951 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:00.723928    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vl9hm\" (UniqueName: \"kubernetes.io/projected/e999d104-c9ea-45a7-9930-7f829013c52e-kube-api-access-vl9hm\") pod \"e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-nc722\" (UID: \"e999d104-c9ea-45a7-9930-7f829013c52e\") " pod="openshift-ingress/e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-nc722"
Oct 31 00:41:00.750689 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:00.750657    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-nc722"
Oct 31 00:41:00.912259 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:00.911293    4284 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w3-dd98f0-7cf4455d8c-979g4"]
Oct 31 00:41:00.915969 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:00.915901    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w3-dd98f0-7cf4455d8c-979g4"
Oct 31 00:41:00.918540 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:00.918500    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:41:00.966599 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:00.966541    4284 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w3-dd98f0-dockercfg-6zqrd\""
Oct 31 00:41:00.966599 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:00.966554    4284 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w3-dd98f0\""
Oct 31 00:41:00.968200 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:00.968164    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6m98s\" (UniqueName: \"kubernetes.io/projected/1a294983-e4ba-4253-8e51-24481d476dd6-kube-api-access-6m98s\") pod \"payload-processing-e2e-worker-w3-dd98f0-7cf4455d8c-979g4\" (UID: \"1a294983-e4ba-4253-8e51-24481d476dd6\") " pod="openshift-ingress/payload-processing-e2e-worker-w3-dd98f0-7cf4455d8c-979g4"
Oct 31 00:41:00.968200 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:00.968351    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1a294983-e4ba-4253-8e51-24481d476dd6-plugins-config-volume\") pod \"payload-processing-e2e-worker-w3-dd98f0-7cf4455d8c-979g4\" (UID: \"1a294983-e4ba-4253-8e51-24481d476dd6\") " pod="openshift-ingress/payload-processing-e2e-worker-w3-dd98f0-7cf4455d8c-979g4"
Oct 31 00:41:00.969093 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:00.969076    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w3-dd98f0-7cf4455d8c-979g4"]
Oct 31 00:41:01.069343 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:01.069292    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1a294983-e4ba-4253-8e51-24481d476dd6-plugins-config-volume\") pod \"payload-processing-e2e-worker-w3-dd98f0-7cf4455d8c-979g4\" (UID: \"1a294983-e4ba-4253-8e51-24481d476dd6\") " pod="openshift-ingress/payload-processing-e2e-worker-w3-dd98f0-7cf4455d8c-979g4"
Oct 31 00:41:01.069537 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:01.069401    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6m98s\" (UniqueName: \"kubernetes.io/projected/1a294983-e4ba-4253-8e51-24481d476dd6-kube-api-access-6m98s\") pod \"payload-processing-e2e-worker-w3-dd98f0-7cf4455d8c-979g4\" (UID: \"1a294983-e4ba-4253-8e51-24481d476dd6\") " pod="openshift-ingress/payload-processing-e2e-worker-w3-dd98f0-7cf4455d8c-979g4"
Oct 31 00:41:01.070013 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:01.069993    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1a294983-e4ba-4253-8e51-24481d476dd6-plugins-config-volume\") pod \"payload-processing-e2e-worker-w3-dd98f0-7cf4455d8c-979g4\" (UID: \"1a294983-e4ba-4253-8e51-24481d476dd6\") " pod="openshift-ingress/payload-processing-e2e-worker-w3-dd98f0-7cf4455d8c-979g4"
Oct 31 00:41:01.128260 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:01.128231    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6m98s\" (UniqueName: \"kubernetes.io/projected/1a294983-e4ba-4253-8e51-24481d476dd6-kube-api-access-6m98s\") pod \"payload-processing-e2e-worker-w3-dd98f0-7cf4455d8c-979g4\" (UID: \"1a294983-e4ba-4253-8e51-24481d476dd6\") " pod="openshift-ingress/payload-processing-e2e-worker-w3-dd98f0-7cf4455d8c-979g4"
Oct 31 00:41:01.181560 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:01.181538    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-nc722"]
Oct 31 00:41:01.183039 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:41:01.183013    4284 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode999d104_c9ea_45a7_9930_7f829013c52e.slice/crio-7f8dca014920dab7f7db7d6a317608194db2aa8f2ff3d46fa0e48adce5090040 WatchSource:0}: Error finding container 7f8dca014920dab7f7db7d6a317608194db2aa8f2ff3d46fa0e48adce5090040: Status 404 returned error can't find the container with id 7f8dca014920dab7f7db7d6a317608194db2aa8f2ff3d46fa0e48adce5090040
Oct 31 00:41:01.185452 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:01.185415    4284 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 00:41:01.185536 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:01.185476    4284 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 00:41:01.185536 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:01.185507    4284 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 00:41:01.237887 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:01.237856    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w3-dd98f0-7cf4455d8c-979g4"
Oct 31 00:41:01.295669 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:01.295623    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-nc722" event={"ID":"e999d104-c9ea-45a7-9930-7f829013c52e","Type":"ContainerStarted","Data":"08276160cce04ccd7627275ee1008a8ae05b058341b2e2d9f29fa4049acfdb7d"}
Oct 31 00:41:01.295827 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:01.295700    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-nc722" event={"ID":"e999d104-c9ea-45a7-9930-7f829013c52e","Type":"ContainerStarted","Data":"7f8dca014920dab7f7db7d6a317608194db2aa8f2ff3d46fa0e48adce5090040"}
Oct 31 00:41:01.416391 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:01.416332    4284 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-nc722" podStartSLOduration=1.416313057 podStartE2EDuration="1.416313057s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 00:41:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:41:01.414790486 +0000 UTC m=+775.559797405" watchObservedRunningTime="2026-10-31 00:41:01.416313057 +0000 UTC m=+775.561319975"
Oct 31 00:41:01.464955 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:01.464879    4284 patch_prober.go:29] interesting pod/e2e-ait-95fa4445-openshift-default-c9bd77677-wz7qd container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 31 00:41:01.464955 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:01.464935    4284 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-95fa4445-openshift-default-c9bd77677-wz7qd" podUID="06987c79-2521-487f-a60d-3050698f9719" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 31 00:41:01.621012 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:01.620988    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w3-dd98f0-7cf4455d8c-979g4"]
Oct 31 00:41:01.622772 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:41:01.622753    4284 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1a294983_e4ba_4253_8e51_24481d476dd6.slice/crio-8d95411bdd4fdf355955af0dfba470741d2300bc948439a0550aa8186c6224db WatchSource:0}: Error finding container 8d95411bdd4fdf355955af0dfba470741d2300bc948439a0550aa8186c6224db: Status 404 returned error can't find the container with id 8d95411bdd4fdf355955af0dfba470741d2300bc948439a0550aa8186c6224db
Oct 31 00:41:01.751712 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:01.751612    4284 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-nc722"
Oct 31 00:41:02.302673 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:02.302636    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w3-dd98f0-7cf4455d8c-979g4" event={"ID":"1a294983-e4ba-4253-8e51-24481d476dd6","Type":"ContainerStarted","Data":"4cd17abeec3e7f3281c3a48b75a8eaa53bba63c3524ff6079dbbf23588047847"}
Oct 31 00:41:02.302865 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:02.302700    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w3-dd98f0-7cf4455d8c-979g4" event={"ID":"1a294983-e4ba-4253-8e51-24481d476dd6","Type":"ContainerStarted","Data":"8d95411bdd4fdf355955af0dfba470741d2300bc948439a0550aa8186c6224db"}
Oct 31 00:41:02.302939 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:02.302920    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-worker-w3-dd98f0-7cf4455d8c-979g4"
Oct 31 00:41:02.414578 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:02.414525    4284 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-worker-w3-dd98f0-7cf4455d8c-979g4" podStartSLOduration=2.414509753 podStartE2EDuration="2.414509753s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 00:41:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:41:02.41438484 +0000 UTC m=+776.559391755" watchObservedRunningTime="2026-10-31 00:41:02.414509753 +0000 UTC m=+776.559516669"
Oct 31 00:41:02.756569 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:02.756531    4284 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-nc722"
Oct 31 00:41:03.309052 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:03.309019    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-nc722"
Oct 31 00:41:03.310115 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:03.310094    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-nc722"
Oct 31 00:41:03.340394 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:03.340359    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:41:04.233818 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:04.233790    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-95fa4445-openshift-default-c9bd77677-wz7qd"
Oct 31 00:41:04.309493 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:04.309398    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/06987c79-2521-487f-a60d-3050698f9719-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/06987c79-2521-487f-a60d-3050698f9719-credential-socket\") pod \"06987c79-2521-487f-a60d-3050698f9719\" (UID: \"06987c79-2521-487f-a60d-3050698f9719\") "
Oct 31 00:41:04.309493 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:04.309481    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/06987c79-2521-487f-a60d-3050698f9719-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/06987c79-2521-487f-a60d-3050698f9719-istio-podinfo\") pod \"06987c79-2521-487f-a60d-3050698f9719\" (UID: \"06987c79-2521-487f-a60d-3050698f9719\") "
Oct 31 00:41:04.309736 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:04.309505    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/06987c79-2521-487f-a60d-3050698f9719-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/06987c79-2521-487f-a60d-3050698f9719-workload-certs\") pod \"06987c79-2521-487f-a60d-3050698f9719\" (UID: \"06987c79-2521-487f-a60d-3050698f9719\") "
Oct 31 00:41:04.309736 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:04.309559    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/06987c79-2521-487f-a60d-3050698f9719-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/06987c79-2521-487f-a60d-3050698f9719-istio-data\") pod \"06987c79-2521-487f-a60d-3050698f9719\" (UID: \"06987c79-2521-487f-a60d-3050698f9719\") "
Oct 31 00:41:04.309736 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:04.309599    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/06987c79-2521-487f-a60d-3050698f9719-kube-api-access-7vr8w\" (UniqueName: \"kubernetes.io/projected/06987c79-2521-487f-a60d-3050698f9719-kube-api-access-7vr8w\") pod \"06987c79-2521-487f-a60d-3050698f9719\" (UID: \"06987c79-2521-487f-a60d-3050698f9719\") "
Oct 31 00:41:04.309736 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:04.309643    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/06987c79-2521-487f-a60d-3050698f9719-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/06987c79-2521-487f-a60d-3050698f9719-istio-envoy\") pod \"06987c79-2521-487f-a60d-3050698f9719\" (UID: \"06987c79-2521-487f-a60d-3050698f9719\") "
Oct 31 00:41:04.309736 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:04.309674    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/06987c79-2521-487f-a60d-3050698f9719-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/06987c79-2521-487f-a60d-3050698f9719-istiod-ca-cert\") pod \"06987c79-2521-487f-a60d-3050698f9719\" (UID: \"06987c79-2521-487f-a60d-3050698f9719\") "
Oct 31 00:41:04.309736 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:04.309694    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/06987c79-2521-487f-a60d-3050698f9719-credential-socket" pod "06987c79-2521-487f-a60d-3050698f9719" (UID: "06987c79-2521-487f-a60d-3050698f9719"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:41:04.309736 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:04.309708    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/06987c79-2521-487f-a60d-3050698f9719-istio-token\" (UniqueName: \"kubernetes.io/projected/06987c79-2521-487f-a60d-3050698f9719-istio-token\") pod \"06987c79-2521-487f-a60d-3050698f9719\" (UID: \"06987c79-2521-487f-a60d-3050698f9719\") "
Oct 31 00:41:04.310096 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:04.309788    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/06987c79-2521-487f-a60d-3050698f9719-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/06987c79-2521-487f-a60d-3050698f9719-workload-socket\") pod \"06987c79-2521-487f-a60d-3050698f9719\" (UID: \"06987c79-2521-487f-a60d-3050698f9719\") "
Oct 31 00:41:04.310096 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:04.309999    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/06987c79-2521-487f-a60d-3050698f9719-istio-data" pod "06987c79-2521-487f-a60d-3050698f9719" (UID: "06987c79-2521-487f-a60d-3050698f9719"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:41:04.310245 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:04.310095    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/06987c79-2521-487f-a60d-3050698f9719-istiod-ca-cert" pod "06987c79-2521-487f-a60d-3050698f9719" (UID: "06987c79-2521-487f-a60d-3050698f9719"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 00:41:04.310356 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:04.310206    4284 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/06987c79-2521-487f-a60d-3050698f9719-credential-socket\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:41:04.310441 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:04.310367    4284 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/06987c79-2521-487f-a60d-3050698f9719-istio-data\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:41:04.310441 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:04.310382    4284 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/06987c79-2521-487f-a60d-3050698f9719-istiod-ca-cert\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:41:04.310441 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:04.310343    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/06987c79-2521-487f-a60d-3050698f9719-workload-socket" pod "06987c79-2521-487f-a60d-3050698f9719" (UID: "06987c79-2521-487f-a60d-3050698f9719"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:41:04.310441 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:04.310361    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/06987c79-2521-487f-a60d-3050698f9719-workload-certs" pod "06987c79-2521-487f-a60d-3050698f9719" (UID: "06987c79-2521-487f-a60d-3050698f9719"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:41:04.312447 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:04.312426    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/06987c79-2521-487f-a60d-3050698f9719-kube-api-access-7vr8w" pod "06987c79-2521-487f-a60d-3050698f9719" (UID: "06987c79-2521-487f-a60d-3050698f9719"). InnerVolumeSpecName "kube-api-access-7vr8w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 00:41:04.313755 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:04.313726    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/06987c79-2521-487f-a60d-3050698f9719-istio-envoy" pod "06987c79-2521-487f-a60d-3050698f9719" (UID: "06987c79-2521-487f-a60d-3050698f9719"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:41:04.313755 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:04.313735    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/06987c79-2521-487f-a60d-3050698f9719-istio-podinfo" pod "06987c79-2521-487f-a60d-3050698f9719" (UID: "06987c79-2521-487f-a60d-3050698f9719"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 00:41:04.314006 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:04.313732    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/06987c79-2521-487f-a60d-3050698f9719-istio-token" pod "06987c79-2521-487f-a60d-3050698f9719" (UID: "06987c79-2521-487f-a60d-3050698f9719"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 00:41:04.317019 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:04.316995    4284 generic.go:356] "Generic (PLEG): container finished" podID="06987c79-2521-487f-a60d-3050698f9719" containerID="f5a24fe6446e45bf1f8d243f02e2759b02f8b8316400142e14a7c27c8c3f686c" exitCode=0
Oct 31 00:41:04.317113 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:04.317057    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-95fa4445-openshift-default-c9bd77677-wz7qd"
Oct 31 00:41:04.317113 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:04.317079    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-95fa4445-openshift-default-c9bd77677-wz7qd" event={"ID":"06987c79-2521-487f-a60d-3050698f9719","Type":"ContainerDied","Data":"f5a24fe6446e45bf1f8d243f02e2759b02f8b8316400142e14a7c27c8c3f686c"}
Oct 31 00:41:04.317191 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:04.317176    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-95fa4445-openshift-default-c9bd77677-wz7qd" event={"ID":"06987c79-2521-487f-a60d-3050698f9719","Type":"ContainerDied","Data":"3f5b227ddc44e4d90770709a5a0fd851c4ea6011b4f9b3370f580ae290dcc3a6"}
Oct 31 00:41:04.317316 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:04.317244    4284 scope.go:122] "RemoveContainer" containerID="f5a24fe6446e45bf1f8d243f02e2759b02f8b8316400142e14a7c27c8c3f686c"
Oct 31 00:41:04.337135 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:04.337115    4284 scope.go:122] "RemoveContainer" containerID="5078f568d9db91b92e652db9f85003d72d143a1c54dbf308c657860aac539164"
Oct 31 00:41:04.348814 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:04.348709    4284 scope.go:122] "RemoveContainer" containerID="f5a24fe6446e45bf1f8d243f02e2759b02f8b8316400142e14a7c27c8c3f686c"
Oct 31 00:41:04.349189 ip-10-0-132-173 kubenswrapper[4284]: E1031 00:41:04.349145    4284 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f5a24fe6446e45bf1f8d243f02e2759b02f8b8316400142e14a7c27c8c3f686c\": container with ID starting with f5a24fe6446e45bf1f8d243f02e2759b02f8b8316400142e14a7c27c8c3f686c not found: ID does not exist" containerID="f5a24fe6446e45bf1f8d243f02e2759b02f8b8316400142e14a7c27c8c3f686c"
Oct 31 00:41:04.349189 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:04.349183    4284 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f5a24fe6446e45bf1f8d243f02e2759b02f8b8316400142e14a7c27c8c3f686c"} err="failed to get container status \"f5a24fe6446e45bf1f8d243f02e2759b02f8b8316400142e14a7c27c8c3f686c\": rpc error: code = NotFound desc = could not find container \"f5a24fe6446e45bf1f8d243f02e2759b02f8b8316400142e14a7c27c8c3f686c\": container with ID starting with f5a24fe6446e45bf1f8d243f02e2759b02f8b8316400142e14a7c27c8c3f686c not found: ID does not exist"
Oct 31 00:41:04.349423 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:04.349223    4284 scope.go:122] "RemoveContainer" containerID="5078f568d9db91b92e652db9f85003d72d143a1c54dbf308c657860aac539164"
Oct 31 00:41:04.349596 ip-10-0-132-173 kubenswrapper[4284]: E1031 00:41:04.349571    4284 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5078f568d9db91b92e652db9f85003d72d143a1c54dbf308c657860aac539164\": container with ID starting with 5078f568d9db91b92e652db9f85003d72d143a1c54dbf308c657860aac539164 not found: ID does not exist" containerID="5078f568d9db91b92e652db9f85003d72d143a1c54dbf308c657860aac539164"
Oct 31 00:41:04.349656 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:04.349605    4284 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5078f568d9db91b92e652db9f85003d72d143a1c54dbf308c657860aac539164"} err="failed to get container status \"5078f568d9db91b92e652db9f85003d72d143a1c54dbf308c657860aac539164\": rpc error: code = NotFound desc = could not find container \"5078f568d9db91b92e652db9f85003d72d143a1c54dbf308c657860aac539164\": container with ID starting with 5078f568d9db91b92e652db9f85003d72d143a1c54dbf308c657860aac539164 not found: ID does not exist"
Oct 31 00:41:04.411678 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:04.411638    4284 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7vr8w\" (UniqueName: \"kubernetes.io/projected/06987c79-2521-487f-a60d-3050698f9719-kube-api-access-7vr8w\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:41:04.411678 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:04.411675    4284 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/06987c79-2521-487f-a60d-3050698f9719-istio-envoy\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:41:04.411895 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:04.411694    4284 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/06987c79-2521-487f-a60d-3050698f9719-istio-token\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:41:04.411895 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:04.411708    4284 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/06987c79-2521-487f-a60d-3050698f9719-workload-socket\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:41:04.411895 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:04.411720    4284 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/06987c79-2521-487f-a60d-3050698f9719-istio-podinfo\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:41:04.411895 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:04.411731    4284 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/06987c79-2521-487f-a60d-3050698f9719-workload-certs\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:41:04.511547 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:04.511475    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-95fa4445-openshift-default-c9bd77677-wz7qd"]
Oct 31 00:41:04.517919 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:04.515568    4284 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-95fa4445-openshift-default-c9bd77677-wz7qd"]
Oct 31 00:41:04.810470 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:04.810429    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:41:04.974581 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:04.974544    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:41:05.130409 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:05.130315    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:41:05.963866 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:05.963827    4284 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="06987c79-2521-487f-a60d-3050698f9719" path="/var/lib/kubelet/pods/06987c79-2521-487f-a60d-3050698f9719/volumes"
Oct 31 00:41:08.412240 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:08.410009    4284 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w1-b63c24-8544f46fb-2qm2b"]
Oct 31 00:41:08.412240 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:08.410763    4284 cpu_manager.go:414] "containerMap: removing container" podUID="06987c79-2521-487f-a60d-3050698f9719" containerName="istio-proxy"
Oct 31 00:41:08.412240 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:08.410782    4284 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="06987c79-2521-487f-a60d-3050698f9719" containerName="istio-proxy"
Oct 31 00:41:08.412240 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:08.410815    4284 cpu_manager.go:414] "containerMap: removing container" podUID="06987c79-2521-487f-a60d-3050698f9719" containerName="istio-proxy"
Oct 31 00:41:08.412240 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:08.410823    4284 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="06987c79-2521-487f-a60d-3050698f9719" containerName="istio-proxy"
Oct 31 00:41:08.412240 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:08.410933    4284 memory_manager.go:365] "RemoveStaleState removing state" podUID="06987c79-2521-487f-a60d-3050698f9719" containerName="istio-proxy"
Oct 31 00:41:08.417332 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:08.417309    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-b63c24-8544f46fb-2qm2b"
Oct 31 00:41:08.452696 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:08.452665    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w1-b63c24-8544f46fb-2qm2b"]
Oct 31 00:41:08.467415 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:08.467387    4284 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w1-b63c24-serving-cert\""
Oct 31 00:41:08.558547 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:08.558503    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b8c47f8b-d458-4669-9af4-95fd324dff37-metrics-certs\") pod \"maas-api-e2e-worker-w1-b63c24-8544f46fb-2qm2b\" (UID: \"b8c47f8b-d458-4669-9af4-95fd324dff37\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-b63c24-8544f46fb-2qm2b"
Oct 31 00:41:08.558547 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:08.558555    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-55bzz\" (UniqueName: \"kubernetes.io/projected/b8c47f8b-d458-4669-9af4-95fd324dff37-kube-api-access-55bzz\") pod \"maas-api-e2e-worker-w1-b63c24-8544f46fb-2qm2b\" (UID: \"b8c47f8b-d458-4669-9af4-95fd324dff37\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-b63c24-8544f46fb-2qm2b"
Oct 31 00:41:08.558820 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:08.558749    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b8c47f8b-d458-4669-9af4-95fd324dff37-maas-api-tls\") pod \"maas-api-e2e-worker-w1-b63c24-8544f46fb-2qm2b\" (UID: \"b8c47f8b-d458-4669-9af4-95fd324dff37\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-b63c24-8544f46fb-2qm2b"
Oct 31 00:41:08.558820 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:08.558789    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b8c47f8b-d458-4669-9af4-95fd324dff37-openshift-service-ca\") pod \"maas-api-e2e-worker-w1-b63c24-8544f46fb-2qm2b\" (UID: \"b8c47f8b-d458-4669-9af4-95fd324dff37\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-b63c24-8544f46fb-2qm2b"
Oct 31 00:41:08.584146 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:08.584119    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:41:08.660128 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:08.660089    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b8c47f8b-d458-4669-9af4-95fd324dff37-maas-api-tls\") pod \"maas-api-e2e-worker-w1-b63c24-8544f46fb-2qm2b\" (UID: \"b8c47f8b-d458-4669-9af4-95fd324dff37\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-b63c24-8544f46fb-2qm2b"
Oct 31 00:41:08.660128 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:08.660127    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b8c47f8b-d458-4669-9af4-95fd324dff37-openshift-service-ca\") pod \"maas-api-e2e-worker-w1-b63c24-8544f46fb-2qm2b\" (UID: \"b8c47f8b-d458-4669-9af4-95fd324dff37\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-b63c24-8544f46fb-2qm2b"
Oct 31 00:41:08.660389 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:08.660175    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b8c47f8b-d458-4669-9af4-95fd324dff37-metrics-certs\") pod \"maas-api-e2e-worker-w1-b63c24-8544f46fb-2qm2b\" (UID: \"b8c47f8b-d458-4669-9af4-95fd324dff37\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-b63c24-8544f46fb-2qm2b"
Oct 31 00:41:08.660389 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:08.660193    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-55bzz\" (UniqueName: \"kubernetes.io/projected/b8c47f8b-d458-4669-9af4-95fd324dff37-kube-api-access-55bzz\") pod \"maas-api-e2e-worker-w1-b63c24-8544f46fb-2qm2b\" (UID: \"b8c47f8b-d458-4669-9af4-95fd324dff37\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-b63c24-8544f46fb-2qm2b"
Oct 31 00:41:08.660589 ip-10-0-132-173 kubenswrapper[4284]: E1031 00:41:08.660571    4284 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-worker-w1-b63c24-serving-cert: secret "maas-api-e2e-worker-w1-b63c24-serving-cert" not found
Oct 31 00:41:08.660660 ip-10-0-132-173 kubenswrapper[4284]: E1031 00:41:08.660648    4284 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b8c47f8b-d458-4669-9af4-95fd324dff37-maas-api-tls podName:b8c47f8b-d458-4669-9af4-95fd324dff37 nodeName:}" failed. No retries permitted until 2026-10-31 00:41:09.160633241 +0000 UTC m=+783.305640136 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/b8c47f8b-d458-4669-9af4-95fd324dff37-maas-api-tls") pod "maas-api-e2e-worker-w1-b63c24-8544f46fb-2qm2b" (UID: "b8c47f8b-d458-4669-9af4-95fd324dff37") : secret "maas-api-e2e-worker-w1-b63c24-serving-cert" not found
Oct 31 00:41:08.661398 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:08.661380    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b8c47f8b-d458-4669-9af4-95fd324dff37-openshift-service-ca\") pod \"maas-api-e2e-worker-w1-b63c24-8544f46fb-2qm2b\" (UID: \"b8c47f8b-d458-4669-9af4-95fd324dff37\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-b63c24-8544f46fb-2qm2b"
Oct 31 00:41:08.663168 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:08.663119    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b8c47f8b-d458-4669-9af4-95fd324dff37-metrics-certs\") pod \"maas-api-e2e-worker-w1-b63c24-8544f46fb-2qm2b\" (UID: \"b8c47f8b-d458-4669-9af4-95fd324dff37\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-b63c24-8544f46fb-2qm2b"
Oct 31 00:41:08.719525 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:08.719489    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-55bzz\" (UniqueName: \"kubernetes.io/projected/b8c47f8b-d458-4669-9af4-95fd324dff37-kube-api-access-55bzz\") pod \"maas-api-e2e-worker-w1-b63c24-8544f46fb-2qm2b\" (UID: \"b8c47f8b-d458-4669-9af4-95fd324dff37\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-b63c24-8544f46fb-2qm2b"
Oct 31 00:41:08.777687 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:08.777650    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:41:08.932464 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:08.932361    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:41:09.164294 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:09.164260    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b8c47f8b-d458-4669-9af4-95fd324dff37-maas-api-tls\") pod \"maas-api-e2e-worker-w1-b63c24-8544f46fb-2qm2b\" (UID: \"b8c47f8b-d458-4669-9af4-95fd324dff37\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-b63c24-8544f46fb-2qm2b"
Oct 31 00:41:09.166810 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:09.166788    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b8c47f8b-d458-4669-9af4-95fd324dff37-maas-api-tls\") pod \"maas-api-e2e-worker-w1-b63c24-8544f46fb-2qm2b\" (UID: \"b8c47f8b-d458-4669-9af4-95fd324dff37\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-b63c24-8544f46fb-2qm2b"
Oct 31 00:41:09.330465 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:09.330369    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-b63c24-8544f46fb-2qm2b"
Oct 31 00:41:09.730297 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:09.730268    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w1-b63c24-8544f46fb-2qm2b"]
Oct 31 00:41:09.731825 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:41:09.731788    4284 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb8c47f8b_d458_4669_9af4_95fd324dff37.slice/crio-1334f961521998acf799b953c7324001930c2f82f14678b234cf24371422d47a WatchSource:0}: Error finding container 1334f961521998acf799b953c7324001930c2f82f14678b234cf24371422d47a: Status 404 returned error can't find the container with id 1334f961521998acf799b953c7324001930c2f82f14678b234cf24371422d47a
Oct 31 00:41:10.299290 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:10.299259    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-b63c24-75cd4d9679-cpvtw"
Oct 31 00:41:10.350261 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:10.350222    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-b63c24-8544f46fb-2qm2b" event={"ID":"b8c47f8b-d458-4669-9af4-95fd324dff37","Type":"ContainerStarted","Data":"1334f961521998acf799b953c7324001930c2f82f14678b234cf24371422d47a"}
Oct 31 00:41:11.358411 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:11.358373    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-b63c24-8544f46fb-2qm2b" event={"ID":"b8c47f8b-d458-4669-9af4-95fd324dff37","Type":"ContainerStarted","Data":"f6cfd624fbfeb47c5f8efb01043161aaf58053495ffd354b49fac0ffdef9ceae"}
Oct 31 00:41:11.358876 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:11.358506    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-b63c24-8544f46fb-2qm2b"
Oct 31 00:41:11.474148 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:11.474085    4284 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-b63c24-8544f46fb-2qm2b" podStartSLOduration=2.614906544 podStartE2EDuration="3.474065614s" totalImagesPullingTime="859.15907ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 00:41:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:41:11.47193268 +0000 UTC m=+785.616939609" watchObservedRunningTime="2026-10-31 00:41:11.474065614 +0000 UTC m=+785.619072532"
Oct 31 00:41:13.312381 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:13.312344    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-worker-w3-dd98f0-7cf4455d8c-979g4"
Oct 31 00:41:13.682447 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:13.682415    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:41:13.879618 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:13.879583    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:41:14.032187 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:14.032099    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:41:14.209834 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:14.209793    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:41:14.336043 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:14.335972    4284 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-8bec73-6b85599f55-5vzdh"]
Oct 31 00:41:14.336500 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:14.336485    4284 memory_manager.go:365] "RemoveStaleState removing state" podUID="06987c79-2521-487f-a60d-3050698f9719" containerName="istio-proxy"
Oct 31 00:41:14.338889 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:14.338868    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-8bec73-6b85599f55-5vzdh"
Oct 31 00:41:14.375786 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:14.375759    4284 generic.go:356] "Generic (PLEG): container finished" podID="a75a6e60-7634-4ae3-ac95-f5cdd33c1af8" containerID="c285f65ea39e06cf577b1d0d137fe831a23be8ad8243cd2b79bb8e8b9a1b7c23" exitCode=0
Oct 31 00:41:14.375963 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:14.375830    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-knm5g" event={"ID":"a75a6e60-7634-4ae3-ac95-f5cdd33c1af8","Type":"ContainerDied","Data":"c285f65ea39e06cf577b1d0d137fe831a23be8ad8243cd2b79bb8e8b9a1b7c23"}
Oct 31 00:41:14.376226 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:14.376194    4284 scope.go:122] "RemoveContainer" containerID="c285f65ea39e06cf577b1d0d137fe831a23be8ad8243cd2b79bb8e8b9a1b7c23"
Oct 31 00:41:14.377563 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:14.377541    4284 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 00:41:14.377628 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:14.377604    4284 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 00:41:14.377667 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:14.377633    4284 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 00:41:14.398849 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:14.398819    4284 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w0-8bec73\""
Oct 31 00:41:14.398954 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:14.398902    4284 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w0-8bec73-dockercfg-wbq7c\""
Oct 31 00:41:14.412971 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:14.412941    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:41:14.421594 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:14.421552    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fc15a078-6a56-4f96-b6a7-ff31044d75c6-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w0-8bec73-6b85599f55-5vzdh\" (UID: \"fc15a078-6a56-4f96-b6a7-ff31044d75c6\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-8bec73-6b85599f55-5vzdh"
Oct 31 00:41:14.421723 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:14.421670    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g8zkq\" (UniqueName: \"kubernetes.io/projected/fc15a078-6a56-4f96-b6a7-ff31044d75c6-kube-api-access-g8zkq\") pod \"payload-pre-processing-e2e-worker-w0-8bec73-6b85599f55-5vzdh\" (UID: \"fc15a078-6a56-4f96-b6a7-ff31044d75c6\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-8bec73-6b85599f55-5vzdh"
Oct 31 00:41:14.457987 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:14.456642    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-8bec73-6b85599f55-5vzdh"]
Oct 31 00:41:14.464357 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:14.464323    4284 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-8bec73-74b4bf6479-9vmvq"]
Oct 31 00:41:14.469970 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:14.469367    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w0-8bec73-74b4bf6479-9vmvq"
Oct 31 00:41:14.518364 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:14.518332    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-8bec73-74b4bf6479-9vmvq"]
Oct 31 00:41:14.522965 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:14.522932    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e9f98efa-eb3e-4a3f-bf43-9b1603d0b725-plugins-config-volume\") pod \"payload-processing-e2e-worker-w0-8bec73-74b4bf6479-9vmvq\" (UID: \"e9f98efa-eb3e-4a3f-bf43-9b1603d0b725\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-8bec73-74b4bf6479-9vmvq"
Oct 31 00:41:14.523144 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:14.523003    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fc15a078-6a56-4f96-b6a7-ff31044d75c6-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w0-8bec73-6b85599f55-5vzdh\" (UID: \"fc15a078-6a56-4f96-b6a7-ff31044d75c6\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-8bec73-6b85599f55-5vzdh"
Oct 31 00:41:14.523144 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:14.523083    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p6969\" (UniqueName: \"kubernetes.io/projected/e9f98efa-eb3e-4a3f-bf43-9b1603d0b725-kube-api-access-p6969\") pod \"payload-processing-e2e-worker-w0-8bec73-74b4bf6479-9vmvq\" (UID: \"e9f98efa-eb3e-4a3f-bf43-9b1603d0b725\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-8bec73-74b4bf6479-9vmvq"
Oct 31 00:41:14.523288 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:14.523139    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g8zkq\" (UniqueName: \"kubernetes.io/projected/fc15a078-6a56-4f96-b6a7-ff31044d75c6-kube-api-access-g8zkq\") pod \"payload-pre-processing-e2e-worker-w0-8bec73-6b85599f55-5vzdh\" (UID: \"fc15a078-6a56-4f96-b6a7-ff31044d75c6\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-8bec73-6b85599f55-5vzdh"
Oct 31 00:41:14.524090 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:14.524066    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fc15a078-6a56-4f96-b6a7-ff31044d75c6-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w0-8bec73-6b85599f55-5vzdh\" (UID: \"fc15a078-6a56-4f96-b6a7-ff31044d75c6\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-8bec73-6b85599f55-5vzdh"
Oct 31 00:41:14.583967 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:14.583930    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g8zkq\" (UniqueName: \"kubernetes.io/projected/fc15a078-6a56-4f96-b6a7-ff31044d75c6-kube-api-access-g8zkq\") pod \"payload-pre-processing-e2e-worker-w0-8bec73-6b85599f55-5vzdh\" (UID: \"fc15a078-6a56-4f96-b6a7-ff31044d75c6\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-8bec73-6b85599f55-5vzdh"
Oct 31 00:41:14.624188 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:14.624066    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e9f98efa-eb3e-4a3f-bf43-9b1603d0b725-plugins-config-volume\") pod \"payload-processing-e2e-worker-w0-8bec73-74b4bf6479-9vmvq\" (UID: \"e9f98efa-eb3e-4a3f-bf43-9b1603d0b725\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-8bec73-74b4bf6479-9vmvq"
Oct 31 00:41:14.624188 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:14.624121    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p6969\" (UniqueName: \"kubernetes.io/projected/e9f98efa-eb3e-4a3f-bf43-9b1603d0b725-kube-api-access-p6969\") pod \"payload-processing-e2e-worker-w0-8bec73-74b4bf6479-9vmvq\" (UID: \"e9f98efa-eb3e-4a3f-bf43-9b1603d0b725\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-8bec73-74b4bf6479-9vmvq"
Oct 31 00:41:14.624895 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:14.624873    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e9f98efa-eb3e-4a3f-bf43-9b1603d0b725-plugins-config-volume\") pod \"payload-processing-e2e-worker-w0-8bec73-74b4bf6479-9vmvq\" (UID: \"e9f98efa-eb3e-4a3f-bf43-9b1603d0b725\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-8bec73-74b4bf6479-9vmvq"
Oct 31 00:41:14.650422 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:14.650381    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-8bec73-6b85599f55-5vzdh"
Oct 31 00:41:14.683232 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:14.683176    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p6969\" (UniqueName: \"kubernetes.io/projected/e9f98efa-eb3e-4a3f-bf43-9b1603d0b725-kube-api-access-p6969\") pod \"payload-processing-e2e-worker-w0-8bec73-74b4bf6479-9vmvq\" (UID: \"e9f98efa-eb3e-4a3f-bf43-9b1603d0b725\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-8bec73-74b4bf6479-9vmvq"
Oct 31 00:41:14.787190 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:14.787154    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w0-8bec73-74b4bf6479-9vmvq"
Oct 31 00:41:14.926933 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:14.926900    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:41:15.032049 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:15.032022    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-8bec73-6b85599f55-5vzdh"]
Oct 31 00:41:15.032935 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:41:15.032909    4284 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfc15a078_6a56_4f96_b6a7_ff31044d75c6.slice/crio-b78cde91ee5e5ec00978c066aad31e22261264941388d854be1c8eff4f64f505 WatchSource:0}: Error finding container b78cde91ee5e5ec00978c066aad31e22261264941388d854be1c8eff4f64f505: Status 404 returned error can't find the container with id b78cde91ee5e5ec00978c066aad31e22261264941388d854be1c8eff4f64f505
Oct 31 00:41:15.078155 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:15.078101    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:41:15.165358 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:15.165332    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-8bec73-74b4bf6479-9vmvq"]
Oct 31 00:41:15.166442 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:41:15.166414    4284 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode9f98efa_eb3e_4a3f_bf43_9b1603d0b725.slice/crio-bbb75327449d55bd07c4ad02fdc735e7e081e1ac5c50153d9795becd5082e8b5 WatchSource:0}: Error finding container bbb75327449d55bd07c4ad02fdc735e7e081e1ac5c50153d9795becd5082e8b5: Status 404 returned error can't find the container with id bbb75327449d55bd07c4ad02fdc735e7e081e1ac5c50153d9795becd5082e8b5
Oct 31 00:41:15.202682 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:15.201317    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-knm5g"
Oct 31 00:41:15.232764 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:15.232732    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:41:15.382714 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:15.382676    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w0-8bec73-74b4bf6479-9vmvq" event={"ID":"e9f98efa-eb3e-4a3f-bf43-9b1603d0b725","Type":"ContainerStarted","Data":"9f1eafe8bb29a0117f13f39e0a2be7f5c1b6f53647d41d58da4587867a669e37"}
Oct 31 00:41:15.383200 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:15.382746    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-worker-w0-8bec73-74b4bf6479-9vmvq"
Oct 31 00:41:15.383200 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:15.382766    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w0-8bec73-74b4bf6479-9vmvq" event={"ID":"e9f98efa-eb3e-4a3f-bf43-9b1603d0b725","Type":"ContainerStarted","Data":"bbb75327449d55bd07c4ad02fdc735e7e081e1ac5c50153d9795becd5082e8b5"}
Oct 31 00:41:15.415825 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:15.415790    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-knm5g" event={"ID":"a75a6e60-7634-4ae3-ac95-f5cdd33c1af8","Type":"ContainerStarted","Data":"2470536176c418a1f51ecc2c20cfd05d60004bc21cb16997d01a3a87cfa9c85a"}
Oct 31 00:41:15.418603 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:15.418574    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-8bec73-6b85599f55-5vzdh" event={"ID":"fc15a078-6a56-4f96-b6a7-ff31044d75c6","Type":"ContainerStarted","Data":"24e9dee9b31c2674a821f1c7e1a29bc9c6eb97bef1e61399f5c5593833e423c2"}
Oct 31 00:41:15.418788 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:15.418634    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-8bec73-6b85599f55-5vzdh"
Oct 31 00:41:15.418788 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:15.418655    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-8bec73-6b85599f55-5vzdh" event={"ID":"fc15a078-6a56-4f96-b6a7-ff31044d75c6","Type":"ContainerStarted","Data":"b78cde91ee5e5ec00978c066aad31e22261264941388d854be1c8eff4f64f505"}
Oct 31 00:41:15.559895 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:15.559813    4284 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-knm5g"
Oct 31 00:41:15.561089 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:15.561064    4284 patch_prober.go:29] interesting pod/e2e-worker-w4-ef2572-openshift-default-7649dd87-knm5g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" start-of-body=
Oct 31 00:41:15.561232 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:15.561107    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-knm5g" podUID="a75a6e60-7634-4ae3-ac95-f5cdd33c1af8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused"
Oct 31 00:41:15.627856 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:15.627809    4284 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-worker-w0-8bec73-74b4bf6479-9vmvq" podStartSLOduration=1.627794312 podStartE2EDuration="1.627794312s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 00:41:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:41:15.507515036 +0000 UTC m=+789.652521954" watchObservedRunningTime="2026-10-31 00:41:15.627794312 +0000 UTC m=+789.772801298"
Oct 31 00:41:16.561299 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:16.561263    4284 patch_prober.go:29] interesting pod/e2e-worker-w4-ef2572-openshift-default-7649dd87-knm5g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" start-of-body=
Oct 31 00:41:16.561767 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:16.561340    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-knm5g" podUID="a75a6e60-7634-4ae3-ac95-f5cdd33c1af8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused"
Oct 31 00:41:17.040450 ip-10-0-132-173 kubenswrapper[4284]: E1031 00:41:17.040406    4284 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fd8sp" containerName="extractor"
Oct 31 00:41:17.370520 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:17.370438    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-b63c24-8544f46fb-2qm2b"
Oct 31 00:41:17.496237 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:17.490020    4284 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-8bec73-6b85599f55-5vzdh" podStartSLOduration=3.489995384 podStartE2EDuration="3.489995384s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 00:41:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:41:15.739832479 +0000 UTC m=+789.884839395" watchObservedRunningTime="2026-10-31 00:41:17.489995384 +0000 UTC m=+791.635002301"
Oct 31 00:41:17.561233 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:17.561186    4284 patch_prober.go:29] interesting pod/e2e-worker-w4-ef2572-openshift-default-7649dd87-knm5g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" start-of-body=
Oct 31 00:41:17.561397 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:17.561294    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-knm5g" podUID="a75a6e60-7634-4ae3-ac95-f5cdd33c1af8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused"
Oct 31 00:41:18.560533 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:18.560478    4284 patch_prober.go:29] interesting pod/e2e-worker-w4-ef2572-openshift-default-7649dd87-knm5g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" start-of-body=
Oct 31 00:41:18.560533 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:18.560531    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-knm5g" podUID="a75a6e60-7634-4ae3-ac95-f5cdd33c1af8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused"
Oct 31 00:41:18.613238 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:18.609604    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:41:19.401240 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:19.398397    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:41:19.559948 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:19.559916    4284 patch_prober.go:29] interesting pod/e2e-worker-w4-ef2572-openshift-default-7649dd87-knm5g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" start-of-body=
Oct 31 00:41:19.560131 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:19.559967    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-knm5g" podUID="a75a6e60-7634-4ae3-ac95-f5cdd33c1af8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused"
Oct 31 00:41:20.560610 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:20.560575    4284 patch_prober.go:29] interesting pod/e2e-worker-w4-ef2572-openshift-default-7649dd87-knm5g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" start-of-body=
Oct 31 00:41:20.561009 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:20.560631    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-knm5g" podUID="a75a6e60-7634-4ae3-ac95-f5cdd33c1af8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused"
Oct 31 00:41:21.564415 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:21.564384    4284 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-knm5g"
Oct 31 00:41:21.564796 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:21.564681    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-knm5g"
Oct 31 00:41:21.565452 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:21.565434    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-knm5g"
Oct 31 00:41:24.397828 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:24.397793    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:41:26.426832 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:26.426806    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-worker-w0-8bec73-74b4bf6479-9vmvq"
Oct 31 00:41:26.427319 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:26.426916    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-8bec73-6b85599f55-5vzdh"
Oct 31 00:41:26.493258 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:26.493193    4284 generic.go:356] "Generic (PLEG): container finished" podID="ccf1138e-2e92-4ab6-bfdd-9979f0b69893" containerID="352c1902de2c418962324852d5b808029015706578d88f6a05bb07b1fe519795" exitCode=0
Oct 31 00:41:26.493438 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:26.493289    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-b63c24-openshift-default-d8c64855f-6mgf7" event={"ID":"ccf1138e-2e92-4ab6-bfdd-9979f0b69893","Type":"ContainerDied","Data":"352c1902de2c418962324852d5b808029015706578d88f6a05bb07b1fe519795"}
Oct 31 00:41:26.493764 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:26.493749    4284 scope.go:122] "RemoveContainer" containerID="352c1902de2c418962324852d5b808029015706578d88f6a05bb07b1fe519795"
Oct 31 00:41:26.495117 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:26.495088    4284 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 00:41:26.495243 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:26.495160    4284 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 00:41:26.495243 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:26.495194    4284 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 00:41:27.514287 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:27.514254    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-b63c24-openshift-default-d8c64855f-6mgf7" event={"ID":"ccf1138e-2e92-4ab6-bfdd-9979f0b69893","Type":"ContainerStarted","Data":"c6896cb617d6c6bc4e67b08c0156fea421b127c0b5542bb89fb23b3d63f2643c"}
Oct 31 00:41:28.316359 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:28.316327    4284 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w1-b63c24-openshift-default-d8c64855f-6mgf7"
Oct 31 00:41:28.317662 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:28.317632    4284 patch_prober.go:29] interesting pod/e2e-worker-w1-b63c24-openshift-default-d8c64855f-6mgf7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused" start-of-body=
Oct 31 00:41:28.317801 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:28.317689    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-b63c24-openshift-default-d8c64855f-6mgf7" podUID="ccf1138e-2e92-4ab6-bfdd-9979f0b69893" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused"
Oct 31 00:41:29.216838 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:29.216798    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:41:29.315801 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:29.315764    4284 patch_prober.go:29] interesting pod/e2e-worker-w1-b63c24-openshift-default-d8c64855f-6mgf7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused" start-of-body=
Oct 31 00:41:29.316088 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:29.316053    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-b63c24-openshift-default-d8c64855f-6mgf7" podUID="ccf1138e-2e92-4ab6-bfdd-9979f0b69893" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused"
Oct 31 00:41:29.367280 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:29.367243    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:41:29.522040 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:29.521957    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:41:30.316179 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:30.316143    4284 patch_prober.go:29] interesting pod/e2e-worker-w1-b63c24-openshift-default-d8c64855f-6mgf7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused" start-of-body=
Oct 31 00:41:30.316643 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:30.316196    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-b63c24-openshift-default-d8c64855f-6mgf7" podUID="ccf1138e-2e92-4ab6-bfdd-9979f0b69893" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused"
Oct 31 00:41:31.209386 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:31.209342    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-b63c24-openshift-default-d8c64855f-6mgf7"
Oct 31 00:41:31.316336 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:31.316302    4284 patch_prober.go:29] interesting pod/e2e-worker-w1-b63c24-openshift-default-d8c64855f-6mgf7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused" start-of-body=
Oct 31 00:41:31.316750 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:31.316356    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-b63c24-openshift-default-d8c64855f-6mgf7" podUID="ccf1138e-2e92-4ab6-bfdd-9979f0b69893" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused"
Oct 31 00:41:31.773121 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:31.773083    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:41:32.317205 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:32.317172    4284 patch_prober.go:29] interesting pod/e2e-worker-w1-b63c24-openshift-default-d8c64855f-6mgf7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused" start-of-body=
Oct 31 00:41:32.317619 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:32.317253    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-b63c24-openshift-default-d8c64855f-6mgf7" podUID="ccf1138e-2e92-4ab6-bfdd-9979f0b69893" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused"
Oct 31 00:41:32.381713 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:32.381669    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:41:32.603672 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:32.603569    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:41:32.762485 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:32.762444    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:41:33.320328 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:33.320301    4284 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w1-b63c24-openshift-default-d8c64855f-6mgf7"
Oct 31 00:41:33.321611 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:33.321581    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w1-b63c24-openshift-default-d8c64855f-6mgf7"
Oct 31 00:41:34.659559 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:34.659523    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:41:39.432535 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:39.432493    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:41:44.109555 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:44.109512    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:41:44.395296 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:44.389441    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:41:47.016481 ip-10-0-132-173 kubenswrapper[4284]: E1031 00:41:47.016444    4284 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fd8sp" containerName="extractor"
Oct 31 00:41:48.920057 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:48.919601    4284 patch_prober.go:29] interesting pod/e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-nc722 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.54:15021/healthz/ready\": read tcp 10.134.0.2:53794->10.134.0.54:15021: read: connection reset by peer" start-of-body=
Oct 31 00:41:48.920057 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:48.919693    4284 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-nc722" podUID="e999d104-c9ea-45a7-9930-7f829013c52e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": read tcp 10.134.0.2:53794->10.134.0.54:15021: read: connection reset by peer"
Oct 31 00:41:49.293706 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:49.293619    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:41:49.441936 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:49.441890    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:41:49.661098 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:49.661069    4284 generic.go:356] "Generic (PLEG): container finished" podID="e999d104-c9ea-45a7-9930-7f829013c52e" containerID="08276160cce04ccd7627275ee1008a8ae05b058341b2e2d9f29fa4049acfdb7d" exitCode=0
Oct 31 00:41:49.661371 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:49.661137    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-nc722" event={"ID":"e999d104-c9ea-45a7-9930-7f829013c52e","Type":"ContainerDied","Data":"08276160cce04ccd7627275ee1008a8ae05b058341b2e2d9f29fa4049acfdb7d"}
Oct 31 00:41:49.661542 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:49.661526    4284 scope.go:122] "RemoveContainer" containerID="08276160cce04ccd7627275ee1008a8ae05b058341b2e2d9f29fa4049acfdb7d"
Oct 31 00:41:49.662823 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:49.662790    4284 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 00:41:49.662946 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:49.662882    4284 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 00:41:49.662946 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:49.662933    4284 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 00:41:49.972261 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:49.972153    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:41:50.121713 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:50.121673    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:41:50.303292 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:50.303007    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:41:50.667618 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:50.667583    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-nc722" event={"ID":"e999d104-c9ea-45a7-9930-7f829013c52e","Type":"ContainerStarted","Data":"4c17ea82450732e49ab6f3eafde1abf9c57f07d3fa62849d8311a5eaf9435ccf"}
Oct 31 00:41:50.751923 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:50.751888    4284 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-nc722"
Oct 31 00:41:50.752093 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:50.752069    4284 patch_prober.go:29] interesting pod/e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-nc722 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" start-of-body=
Oct 31 00:41:50.752137 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:50.752115    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-nc722" podUID="e999d104-c9ea-45a7-9930-7f829013c52e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused"
Oct 31 00:41:51.753586 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:51.753549    4284 patch_prober.go:29] interesting pod/e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-nc722 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" start-of-body=
Oct 31 00:41:51.754016 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:51.753643    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-nc722" podUID="e999d104-c9ea-45a7-9930-7f829013c52e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused"
Oct 31 00:41:52.751855 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:52.751821    4284 patch_prober.go:29] interesting pod/e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-nc722 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" start-of-body=
Oct 31 00:41:52.752045 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:52.751889    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-nc722" podUID="e999d104-c9ea-45a7-9930-7f829013c52e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused"
Oct 31 00:41:53.042094 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:53.041992    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:41:53.322787 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:53.322702    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:41:53.649641 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:53.649551    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:41:53.751848 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:53.751810    4284 patch_prober.go:29] interesting pod/e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-nc722 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" start-of-body=
Oct 31 00:41:53.752033 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:53.751875    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-nc722" podUID="e999d104-c9ea-45a7-9930-7f829013c52e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused"
Oct 31 00:41:53.801238 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:53.799681    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:41:53.949803 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:53.949772    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:41:54.426105 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:54.426063    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:41:54.751416 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:54.751328    4284 patch_prober.go:29] interesting pod/e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-nc722 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" start-of-body=
Oct 31 00:41:54.751416 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:54.751402    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-nc722" podUID="e999d104-c9ea-45a7-9930-7f829013c52e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused"
Oct 31 00:41:55.806897 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:55.806850    4284 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-nc722"
Oct 31 00:41:55.807717 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:55.807696    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-nc722"
Oct 31 00:41:55.809661 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:55.809639    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-nc722"
Oct 31 00:41:59.187041 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:41:59.186998    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:42:01.055587 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:42:01.055553    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:42:04.033644 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:42:04.033602    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:42:05.333063 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:42:05.333031    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:42:05.956018 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:42:05.955982    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:42:06.151817 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:42:06.151774    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:42:09.100784 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:42:09.100740    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:42:13.902924 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:42:13.902879    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:42:15.934326 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:42:15.934287    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:42:17.013796 ip-10-0-132-173 kubenswrapper[4284]: E1031 00:42:17.013722    4284 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fd8sp" containerName="extractor"
Oct 31 00:42:18.690228 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:42:18.690160    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:42:21.192914 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:42:21.192870    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:42:23.601655 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:42:23.601615    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:42:28.388865 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:42:28.388835    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:42:33.189576 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:42:33.189539    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:42:34.301349 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:42:34.301307    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:42:37.191661 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:42:37.191620    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:42:37.999257 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:42:37.999226    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:42:42.835732 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:42:42.835666    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:42:47.014186 ip-10-0-132-173 kubenswrapper[4284]: E1031 00:42:47.014148    4284 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fd8sp" containerName="extractor"
Oct 31 00:42:47.514429 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:42:47.514387    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:42:52.306088 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:42:52.306047    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:42:53.639028 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:42:53.638990    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:42:57.183309 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:42:57.183270    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:43:02.000573 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:43:02.000536    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:43:06.309624 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:43:06.309588    4284 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w1-b63c24/premium-w1-b63c24-kserve-549f786d9-k7g8j"]
Oct 31 00:43:06.313562 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:43:06.313538    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-b63c24/premium-w1-b63c24-kserve-549f786d9-k7g8j"
Oct 31 00:43:06.367126 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:43:06.367095    4284 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w1-b63c24\"/\"kube-root-ca.crt\""
Oct 31 00:43:06.408305 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:43:06.408275    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:43:06.411344 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:43:06.411320    4284 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w1-b63c24\"/\"openshift-service-ca.crt\""
Oct 31 00:43:06.411448 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:43:06.411427    4284 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w1-b63c24\"/\"default-dockercfg-mkgsq\""
Oct 31 00:43:06.411520 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:43:06.411501    4284 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w1-b63c24\"/\"premium-w1-b63c24-kserve-self-signed-certs\""
Oct 31 00:43:06.413585 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:43:06.413569    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w1-b63c24/premium-w1-b63c24-kserve-549f786d9-k7g8j"]
Oct 31 00:43:06.460173 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:43:06.459606    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b6846d68-7b58-4637-a47f-cd55bbf79b98-tls-certs\") pod \"premium-w1-b63c24-kserve-549f786d9-k7g8j\" (UID: \"b6846d68-7b58-4637-a47f-cd55bbf79b98\") " pod="e2e-models-e2e-worker-w1-b63c24/premium-w1-b63c24-kserve-549f786d9-k7g8j"
Oct 31 00:43:06.460173 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:43:06.459706    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qgb87\" (UniqueName: \"kubernetes.io/projected/b6846d68-7b58-4637-a47f-cd55bbf79b98-kube-api-access-qgb87\") pod \"premium-w1-b63c24-kserve-549f786d9-k7g8j\" (UID: \"b6846d68-7b58-4637-a47f-cd55bbf79b98\") " pod="e2e-models-e2e-worker-w1-b63c24/premium-w1-b63c24-kserve-549f786d9-k7g8j"
Oct 31 00:43:06.460173 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:43:06.459891    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b6846d68-7b58-4637-a47f-cd55bbf79b98-tmp-dir\") pod \"premium-w1-b63c24-kserve-549f786d9-k7g8j\" (UID: \"b6846d68-7b58-4637-a47f-cd55bbf79b98\") " pod="e2e-models-e2e-worker-w1-b63c24/premium-w1-b63c24-kserve-549f786d9-k7g8j"
Oct 31 00:43:06.460173 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:43:06.459922    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b6846d68-7b58-4637-a47f-cd55bbf79b98-model-cache\") pod \"premium-w1-b63c24-kserve-549f786d9-k7g8j\" (UID: \"b6846d68-7b58-4637-a47f-cd55bbf79b98\") " pod="e2e-models-e2e-worker-w1-b63c24/premium-w1-b63c24-kserve-549f786d9-k7g8j"
Oct 31 00:43:06.460173 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:43:06.459961    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b6846d68-7b58-4637-a47f-cd55bbf79b98-dshm\") pod \"premium-w1-b63c24-kserve-549f786d9-k7g8j\" (UID: \"b6846d68-7b58-4637-a47f-cd55bbf79b98\") " pod="e2e-models-e2e-worker-w1-b63c24/premium-w1-b63c24-kserve-549f786d9-k7g8j"
Oct 31 00:43:06.460173 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:43:06.460000    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b6846d68-7b58-4637-a47f-cd55bbf79b98-home\") pod \"premium-w1-b63c24-kserve-549f786d9-k7g8j\" (UID: \"b6846d68-7b58-4637-a47f-cd55bbf79b98\") " pod="e2e-models-e2e-worker-w1-b63c24/premium-w1-b63c24-kserve-549f786d9-k7g8j"
Oct 31 00:43:06.561500 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:43:06.561405    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b6846d68-7b58-4637-a47f-cd55bbf79b98-dshm\") pod \"premium-w1-b63c24-kserve-549f786d9-k7g8j\" (UID: \"b6846d68-7b58-4637-a47f-cd55bbf79b98\") " pod="e2e-models-e2e-worker-w1-b63c24/premium-w1-b63c24-kserve-549f786d9-k7g8j"
Oct 31 00:43:06.561500 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:43:06.561458    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b6846d68-7b58-4637-a47f-cd55bbf79b98-home\") pod \"premium-w1-b63c24-kserve-549f786d9-k7g8j\" (UID: \"b6846d68-7b58-4637-a47f-cd55bbf79b98\") " pod="e2e-models-e2e-worker-w1-b63c24/premium-w1-b63c24-kserve-549f786d9-k7g8j"
Oct 31 00:43:06.561825 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:43:06.561526    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b6846d68-7b58-4637-a47f-cd55bbf79b98-tls-certs\") pod \"premium-w1-b63c24-kserve-549f786d9-k7g8j\" (UID: \"b6846d68-7b58-4637-a47f-cd55bbf79b98\") " pod="e2e-models-e2e-worker-w1-b63c24/premium-w1-b63c24-kserve-549f786d9-k7g8j"
Oct 31 00:43:06.561825 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:43:06.561557    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qgb87\" (UniqueName: \"kubernetes.io/projected/b6846d68-7b58-4637-a47f-cd55bbf79b98-kube-api-access-qgb87\") pod \"premium-w1-b63c24-kserve-549f786d9-k7g8j\" (UID: \"b6846d68-7b58-4637-a47f-cd55bbf79b98\") " pod="e2e-models-e2e-worker-w1-b63c24/premium-w1-b63c24-kserve-549f786d9-k7g8j"
Oct 31 00:43:06.561825 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:43:06.561668    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b6846d68-7b58-4637-a47f-cd55bbf79b98-tmp-dir\") pod \"premium-w1-b63c24-kserve-549f786d9-k7g8j\" (UID: \"b6846d68-7b58-4637-a47f-cd55bbf79b98\") " pod="e2e-models-e2e-worker-w1-b63c24/premium-w1-b63c24-kserve-549f786d9-k7g8j"
Oct 31 00:43:06.561825 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:43:06.561696    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b6846d68-7b58-4637-a47f-cd55bbf79b98-model-cache\") pod \"premium-w1-b63c24-kserve-549f786d9-k7g8j\" (UID: \"b6846d68-7b58-4637-a47f-cd55bbf79b98\") " pod="e2e-models-e2e-worker-w1-b63c24/premium-w1-b63c24-kserve-549f786d9-k7g8j"
Oct 31 00:43:06.562131 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:43:06.562106    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b6846d68-7b58-4637-a47f-cd55bbf79b98-model-cache\") pod \"premium-w1-b63c24-kserve-549f786d9-k7g8j\" (UID: \"b6846d68-7b58-4637-a47f-cd55bbf79b98\") " pod="e2e-models-e2e-worker-w1-b63c24/premium-w1-b63c24-kserve-549f786d9-k7g8j"
Oct 31 00:43:06.562415 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:43:06.562389    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b6846d68-7b58-4637-a47f-cd55bbf79b98-home\") pod \"premium-w1-b63c24-kserve-549f786d9-k7g8j\" (UID: \"b6846d68-7b58-4637-a47f-cd55bbf79b98\") " pod="e2e-models-e2e-worker-w1-b63c24/premium-w1-b63c24-kserve-549f786d9-k7g8j"
Oct 31 00:43:06.562541 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:43:06.562422    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b6846d68-7b58-4637-a47f-cd55bbf79b98-tmp-dir\") pod \"premium-w1-b63c24-kserve-549f786d9-k7g8j\" (UID: \"b6846d68-7b58-4637-a47f-cd55bbf79b98\") " pod="e2e-models-e2e-worker-w1-b63c24/premium-w1-b63c24-kserve-549f786d9-k7g8j"
Oct 31 00:43:06.564041 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:43:06.564016    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b6846d68-7b58-4637-a47f-cd55bbf79b98-dshm\") pod \"premium-w1-b63c24-kserve-549f786d9-k7g8j\" (UID: \"b6846d68-7b58-4637-a47f-cd55bbf79b98\") " pod="e2e-models-e2e-worker-w1-b63c24/premium-w1-b63c24-kserve-549f786d9-k7g8j"
Oct 31 00:43:06.564410 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:43:06.564393    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b6846d68-7b58-4637-a47f-cd55bbf79b98-tls-certs\") pod \"premium-w1-b63c24-kserve-549f786d9-k7g8j\" (UID: \"b6846d68-7b58-4637-a47f-cd55bbf79b98\") " pod="e2e-models-e2e-worker-w1-b63c24/premium-w1-b63c24-kserve-549f786d9-k7g8j"
Oct 31 00:43:06.623327 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:43:06.623289    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qgb87\" (UniqueName: \"kubernetes.io/projected/b6846d68-7b58-4637-a47f-cd55bbf79b98-kube-api-access-qgb87\") pod \"premium-w1-b63c24-kserve-549f786d9-k7g8j\" (UID: \"b6846d68-7b58-4637-a47f-cd55bbf79b98\") " pod="e2e-models-e2e-worker-w1-b63c24/premium-w1-b63c24-kserve-549f786d9-k7g8j"
Oct 31 00:43:06.641793 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:43:06.641766    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-b63c24/premium-w1-b63c24-kserve-549f786d9-k7g8j"
Oct 31 00:43:06.834849 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:43:06.834760    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:43:07.028663 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:43:07.028632    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w1-b63c24/premium-w1-b63c24-kserve-549f786d9-k7g8j"]
Oct 31 00:43:07.030279 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:43:07.030252    4284 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb6846d68_7b58_4637_a47f_cd55bbf79b98.slice/crio-1125046877e9650a0960bbc9c501d59767fd291a929eb8a566e671226cbe443a WatchSource:0}: Error finding container 1125046877e9650a0960bbc9c501d59767fd291a929eb8a566e671226cbe443a: Status 404 returned error can't find the container with id 1125046877e9650a0960bbc9c501d59767fd291a929eb8a566e671226cbe443a
Oct 31 00:43:07.078985 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:43:07.078952    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-b63c24/premium-w1-b63c24-kserve-549f786d9-k7g8j" event={"ID":"b6846d68-7b58-4637-a47f-cd55bbf79b98","Type":"ContainerStarted","Data":"1125046877e9650a0960bbc9c501d59767fd291a929eb8a566e671226cbe443a"}
Oct 31 00:43:08.085192 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:43:08.085149    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-b63c24/premium-w1-b63c24-kserve-549f786d9-k7g8j" event={"ID":"b6846d68-7b58-4637-a47f-cd55bbf79b98","Type":"ContainerStarted","Data":"d441c0d2e5722e306c81c1f810ff4601f5dd44d407ba999997d31bdc7d65cd3a"}
Oct 31 00:43:08.197679 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:43:08.197628    4284 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w1-b63c24/premium-w1-b63c24-kserve-549f786d9-k7g8j" podStartSLOduration=2.197615124 podStartE2EDuration="2.197615124s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 00:43:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:43:08.19716494 +0000 UTC m=+902.342171856" watchObservedRunningTime="2026-10-31 00:43:08.197615124 +0000 UTC m=+902.342622040"
Oct 31 00:43:08.641904 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:43:08.641865    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-b63c24/premium-w1-b63c24-kserve-549f786d9-k7g8j"
Oct 31 00:43:11.633675 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:43:11.633638    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:43:12.536424 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:43:12.536387    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:43:13.617970 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:43:13.617919    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:43:16.386403 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:43:16.386365    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:43:16.642245 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:43:16.642124    4284 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w1-b63c24/premium-w1-b63c24-kserve-549f786d9-k7g8j"
Oct 31 00:43:16.654589 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:43:16.654562    4284 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w1-b63c24/premium-w1-b63c24-kserve-549f786d9-k7g8j"
Oct 31 00:43:17.014074 ip-10-0-132-173 kubenswrapper[4284]: E1031 00:43:17.014045    4284 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fd8sp" containerName="extractor"
Oct 31 00:43:17.149253 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:43:17.149224    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-b63c24/premium-w1-b63c24-kserve-549f786d9-k7g8j"
Oct 31 00:43:21.196307 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:43:21.196270    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:43:26.003485 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:43:26.003447    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:43:26.567364 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:43:26.567332    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:43:30.817036 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:43:30.817000    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:43:35.524396 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:43:35.524356    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:43:38.655390 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:43:38.655357    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:43:40.409703 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:43:40.409670    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:43:43.952132 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:43:43.952095    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:43:45.185261 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:43:45.185206    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:43:47.013978 ip-10-0-132-173 kubenswrapper[4284]: E1031 00:43:47.013898    4284 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fd8sp" containerName="extractor"
Oct 31 00:43:49.991549 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:43:49.991505    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:43:53.975870 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:43:53.975819    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:43:54.803410 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:43:54.803365    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:43:59.597037 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:43:59.596994    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:44:04.402722 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:04.402688    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:44:04.731442 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:04.731409    4284 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-wqns9"]
Oct 31 00:44:04.735478 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:04.735453    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-wqns9"
Oct 31 00:44:04.785996 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:04.785956    4284 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w3-dd98f0-openshift-default-dockercfg-6r29c\""
Oct 31 00:44:04.825585 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:04.825535    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fc082395-6c5f-4a62-8d0a-1e7d5d562c12-workload-certs\") pod \"e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-wqns9\" (UID: \"fc082395-6c5f-4a62-8d0a-1e7d5d562c12\") " pod="openshift-ingress/e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-wqns9"
Oct 31 00:44:04.825801 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:04.825714    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fc082395-6c5f-4a62-8d0a-1e7d5d562c12-istio-data\") pod \"e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-wqns9\" (UID: \"fc082395-6c5f-4a62-8d0a-1e7d5d562c12\") " pod="openshift-ingress/e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-wqns9"
Oct 31 00:44:04.825801 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:04.825762    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fc082395-6c5f-4a62-8d0a-1e7d5d562c12-istio-podinfo\") pod \"e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-wqns9\" (UID: \"fc082395-6c5f-4a62-8d0a-1e7d5d562c12\") " pod="openshift-ingress/e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-wqns9"
Oct 31 00:44:04.825919 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:04.825889    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7qzjm\" (UniqueName: \"kubernetes.io/projected/fc082395-6c5f-4a62-8d0a-1e7d5d562c12-kube-api-access-7qzjm\") pod \"e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-wqns9\" (UID: \"fc082395-6c5f-4a62-8d0a-1e7d5d562c12\") " pod="openshift-ingress/e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-wqns9"
Oct 31 00:44:04.825967 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:04.825941    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fc082395-6c5f-4a62-8d0a-1e7d5d562c12-workload-socket\") pod \"e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-wqns9\" (UID: \"fc082395-6c5f-4a62-8d0a-1e7d5d562c12\") " pod="openshift-ingress/e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-wqns9"
Oct 31 00:44:04.826031 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:04.826010    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fc082395-6c5f-4a62-8d0a-1e7d5d562c12-istiod-ca-cert\") pod \"e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-wqns9\" (UID: \"fc082395-6c5f-4a62-8d0a-1e7d5d562c12\") " pod="openshift-ingress/e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-wqns9"
Oct 31 00:44:04.826082 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:04.826067    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fc082395-6c5f-4a62-8d0a-1e7d5d562c12-credential-socket\") pod \"e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-wqns9\" (UID: \"fc082395-6c5f-4a62-8d0a-1e7d5d562c12\") " pod="openshift-ingress/e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-wqns9"
Oct 31 00:44:04.826164 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:04.826151    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fc082395-6c5f-4a62-8d0a-1e7d5d562c12-istio-envoy\") pod \"e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-wqns9\" (UID: \"fc082395-6c5f-4a62-8d0a-1e7d5d562c12\") " pod="openshift-ingress/e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-wqns9"
Oct 31 00:44:04.826202 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:04.826177    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fc082395-6c5f-4a62-8d0a-1e7d5d562c12-istio-token\") pod \"e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-wqns9\" (UID: \"fc082395-6c5f-4a62-8d0a-1e7d5d562c12\") " pod="openshift-ingress/e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-wqns9"
Oct 31 00:44:04.845899 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:04.845858    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-wqns9"]
Oct 31 00:44:04.929796 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:04.929754    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fc082395-6c5f-4a62-8d0a-1e7d5d562c12-istio-podinfo\") pod \"e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-wqns9\" (UID: \"fc082395-6c5f-4a62-8d0a-1e7d5d562c12\") " pod="openshift-ingress/e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-wqns9"
Oct 31 00:44:04.953392 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:04.953350    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7qzjm\" (UniqueName: \"kubernetes.io/projected/fc082395-6c5f-4a62-8d0a-1e7d5d562c12-kube-api-access-7qzjm\") pod \"e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-wqns9\" (UID: \"fc082395-6c5f-4a62-8d0a-1e7d5d562c12\") " pod="openshift-ingress/e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-wqns9"
Oct 31 00:44:04.953670 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:04.953647    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fc082395-6c5f-4a62-8d0a-1e7d5d562c12-workload-socket\") pod \"e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-wqns9\" (UID: \"fc082395-6c5f-4a62-8d0a-1e7d5d562c12\") " pod="openshift-ingress/e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-wqns9"
Oct 31 00:44:04.953811 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:04.953794    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fc082395-6c5f-4a62-8d0a-1e7d5d562c12-istiod-ca-cert\") pod \"e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-wqns9\" (UID: \"fc082395-6c5f-4a62-8d0a-1e7d5d562c12\") " pod="openshift-ingress/e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-wqns9"
Oct 31 00:44:04.953963 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:04.953947    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fc082395-6c5f-4a62-8d0a-1e7d5d562c12-credential-socket\") pod \"e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-wqns9\" (UID: \"fc082395-6c5f-4a62-8d0a-1e7d5d562c12\") " pod="openshift-ingress/e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-wqns9"
Oct 31 00:44:04.954118 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:04.954100    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fc082395-6c5f-4a62-8d0a-1e7d5d562c12-istio-envoy\") pod \"e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-wqns9\" (UID: \"fc082395-6c5f-4a62-8d0a-1e7d5d562c12\") " pod="openshift-ingress/e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-wqns9"
Oct 31 00:44:04.954333 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:04.954314    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fc082395-6c5f-4a62-8d0a-1e7d5d562c12-istio-token\") pod \"e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-wqns9\" (UID: \"fc082395-6c5f-4a62-8d0a-1e7d5d562c12\") " pod="openshift-ingress/e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-wqns9"
Oct 31 00:44:04.954519 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:04.954504    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fc082395-6c5f-4a62-8d0a-1e7d5d562c12-workload-certs\") pod \"e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-wqns9\" (UID: \"fc082395-6c5f-4a62-8d0a-1e7d5d562c12\") " pod="openshift-ingress/e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-wqns9"
Oct 31 00:44:04.954727 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:04.954714    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fc082395-6c5f-4a62-8d0a-1e7d5d562c12-istio-data\") pod \"e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-wqns9\" (UID: \"fc082395-6c5f-4a62-8d0a-1e7d5d562c12\") " pod="openshift-ingress/e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-wqns9"
Oct 31 00:44:04.968236 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:04.955319    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fc082395-6c5f-4a62-8d0a-1e7d5d562c12-istio-data\") pod \"e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-wqns9\" (UID: \"fc082395-6c5f-4a62-8d0a-1e7d5d562c12\") " pod="openshift-ingress/e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-wqns9"
Oct 31 00:44:04.972236 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:04.969255    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fc082395-6c5f-4a62-8d0a-1e7d5d562c12-istiod-ca-cert\") pod \"e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-wqns9\" (UID: \"fc082395-6c5f-4a62-8d0a-1e7d5d562c12\") " pod="openshift-ingress/e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-wqns9"
Oct 31 00:44:04.972464 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:04.971580    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fc082395-6c5f-4a62-8d0a-1e7d5d562c12-workload-socket\") pod \"e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-wqns9\" (UID: \"fc082395-6c5f-4a62-8d0a-1e7d5d562c12\") " pod="openshift-ingress/e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-wqns9"
Oct 31 00:44:04.972532 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:04.972148    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fc082395-6c5f-4a62-8d0a-1e7d5d562c12-credential-socket\") pod \"e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-wqns9\" (UID: \"fc082395-6c5f-4a62-8d0a-1e7d5d562c12\") " pod="openshift-ingress/e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-wqns9"
Oct 31 00:44:04.972532 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:04.972384    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fc082395-6c5f-4a62-8d0a-1e7d5d562c12-workload-certs\") pod \"e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-wqns9\" (UID: \"fc082395-6c5f-4a62-8d0a-1e7d5d562c12\") " pod="openshift-ingress/e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-wqns9"
Oct 31 00:44:04.974821 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:04.974801    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fc082395-6c5f-4a62-8d0a-1e7d5d562c12-istio-envoy\") pod \"e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-wqns9\" (UID: \"fc082395-6c5f-4a62-8d0a-1e7d5d562c12\") " pod="openshift-ingress/e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-wqns9"
Oct 31 00:44:04.974937 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:04.974919    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fc082395-6c5f-4a62-8d0a-1e7d5d562c12-istio-podinfo\") pod \"e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-wqns9\" (UID: \"fc082395-6c5f-4a62-8d0a-1e7d5d562c12\") " pod="openshift-ingress/e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-wqns9"
Oct 31 00:44:05.034378 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:05.034257    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7qzjm\" (UniqueName: \"kubernetes.io/projected/fc082395-6c5f-4a62-8d0a-1e7d5d562c12-kube-api-access-7qzjm\") pod \"e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-wqns9\" (UID: \"fc082395-6c5f-4a62-8d0a-1e7d5d562c12\") " pod="openshift-ingress/e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-wqns9"
Oct 31 00:44:05.035283 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:05.035202    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fc082395-6c5f-4a62-8d0a-1e7d5d562c12-istio-token\") pod \"e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-wqns9\" (UID: \"fc082395-6c5f-4a62-8d0a-1e7d5d562c12\") " pod="openshift-ingress/e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-wqns9"
Oct 31 00:44:05.052588 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:05.052532    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-wqns9"
Oct 31 00:44:05.488390 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:05.488366    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-wqns9"]
Oct 31 00:44:05.489909 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:44:05.489888    4284 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfc082395_6c5f_4a62_8d0a_1e7d5d562c12.slice/crio-cfb589e2fd6d38dc979463ee7c57705800f85a52a08fa33b526f3a5861e84e40 WatchSource:0}: Error finding container cfb589e2fd6d38dc979463ee7c57705800f85a52a08fa33b526f3a5861e84e40: Status 404 returned error can't find the container with id cfb589e2fd6d38dc979463ee7c57705800f85a52a08fa33b526f3a5861e84e40
Oct 31 00:44:05.492165 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:05.492134    4284 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 00:44:05.492280 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:05.492198    4284 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 00:44:05.492280 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:05.492247    4284 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 00:44:06.416865 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:06.416830    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-wqns9" event={"ID":"fc082395-6c5f-4a62-8d0a-1e7d5d562c12","Type":"ContainerStarted","Data":"57366ca8c190c348c75806d2546bdbc28b788c48c95551a1dde53188378ad7a4"}
Oct 31 00:44:06.417128 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:06.416907    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-wqns9" event={"ID":"fc082395-6c5f-4a62-8d0a-1e7d5d562c12","Type":"ContainerStarted","Data":"cfb589e2fd6d38dc979463ee7c57705800f85a52a08fa33b526f3a5861e84e40"}
Oct 31 00:44:06.610390 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:06.610339    4284 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-wqns9" podStartSLOduration=2.610324599 podStartE2EDuration="2.610324599s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 00:44:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:44:06.609837341 +0000 UTC m=+960.754844289" watchObservedRunningTime="2026-10-31 00:44:06.610324599 +0000 UTC m=+960.755331515"
Oct 31 00:44:07.052946 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:07.052913    4284 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-wqns9"
Oct 31 00:44:07.054614 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:07.054585    4284 patch_prober.go:29] interesting pod/e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-wqns9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused" start-of-body=
Oct 31 00:44:07.054747 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:07.054660    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-wqns9" podUID="fc082395-6c5f-4a62-8d0a-1e7d5d562c12" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused"
Oct 31 00:44:08.053065 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:08.053019    4284 patch_prober.go:29] interesting pod/e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-wqns9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused" start-of-body=
Oct 31 00:44:08.053618 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:08.053080    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-wqns9" podUID="fc082395-6c5f-4a62-8d0a-1e7d5d562c12" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused"
Oct 31 00:44:09.053474 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:09.053442    4284 patch_prober.go:29] interesting pod/e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-wqns9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused" start-of-body=
Oct 31 00:44:09.053902 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:09.053489    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-wqns9" podUID="fc082395-6c5f-4a62-8d0a-1e7d5d562c12" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused"
Oct 31 00:44:09.340061 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:09.339951    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:44:10.053646 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:10.053617    4284 patch_prober.go:29] interesting pod/e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-wqns9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused" start-of-body=
Oct 31 00:44:10.054103 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:10.053676    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-wqns9" podUID="fc082395-6c5f-4a62-8d0a-1e7d5d562c12" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused"
Oct 31 00:44:11.052988 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:11.052949    4284 patch_prober.go:29] interesting pod/e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-wqns9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused" start-of-body=
Oct 31 00:44:11.053197 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:11.053013    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-wqns9" podUID="fc082395-6c5f-4a62-8d0a-1e7d5d562c12" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused"
Oct 31 00:44:12.163566 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:12.163534    4284 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-wqns9"
Oct 31 00:44:12.164064 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:12.163944    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-wqns9"
Oct 31 00:44:12.164918 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:12.164901    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-wqns9"
Oct 31 00:44:14.099039 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:14.099001    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:44:14.947689 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:14.947648    4284 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w2-29ece5-openshift-default-6fcb86dbc-hl96f"]
Oct 31 00:44:14.951461 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:14.951440    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-29ece5-openshift-default-6fcb86dbc-hl96f"
Oct 31 00:44:14.979072 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:14.979045    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/19acc006-2388-488d-bfff-6ce6bab123a8-istio-podinfo\") pod \"e2e-worker-w2-29ece5-openshift-default-6fcb86dbc-hl96f\" (UID: \"19acc006-2388-488d-bfff-6ce6bab123a8\") " pod="openshift-ingress/e2e-worker-w2-29ece5-openshift-default-6fcb86dbc-hl96f"
Oct 31 00:44:14.979268 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:14.979098    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/19acc006-2388-488d-bfff-6ce6bab123a8-workload-socket\") pod \"e2e-worker-w2-29ece5-openshift-default-6fcb86dbc-hl96f\" (UID: \"19acc006-2388-488d-bfff-6ce6bab123a8\") " pod="openshift-ingress/e2e-worker-w2-29ece5-openshift-default-6fcb86dbc-hl96f"
Oct 31 00:44:14.979268 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:14.979252    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/19acc006-2388-488d-bfff-6ce6bab123a8-istio-token\") pod \"e2e-worker-w2-29ece5-openshift-default-6fcb86dbc-hl96f\" (UID: \"19acc006-2388-488d-bfff-6ce6bab123a8\") " pod="openshift-ingress/e2e-worker-w2-29ece5-openshift-default-6fcb86dbc-hl96f"
Oct 31 00:44:14.979383 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:14.979298    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jrs5w\" (UniqueName: \"kubernetes.io/projected/19acc006-2388-488d-bfff-6ce6bab123a8-kube-api-access-jrs5w\") pod \"e2e-worker-w2-29ece5-openshift-default-6fcb86dbc-hl96f\" (UID: \"19acc006-2388-488d-bfff-6ce6bab123a8\") " pod="openshift-ingress/e2e-worker-w2-29ece5-openshift-default-6fcb86dbc-hl96f"
Oct 31 00:44:14.979383 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:14.979361    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/19acc006-2388-488d-bfff-6ce6bab123a8-workload-certs\") pod \"e2e-worker-w2-29ece5-openshift-default-6fcb86dbc-hl96f\" (UID: \"19acc006-2388-488d-bfff-6ce6bab123a8\") " pod="openshift-ingress/e2e-worker-w2-29ece5-openshift-default-6fcb86dbc-hl96f"
Oct 31 00:44:14.979478 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:14.979394    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/19acc006-2388-488d-bfff-6ce6bab123a8-istio-data\") pod \"e2e-worker-w2-29ece5-openshift-default-6fcb86dbc-hl96f\" (UID: \"19acc006-2388-488d-bfff-6ce6bab123a8\") " pod="openshift-ingress/e2e-worker-w2-29ece5-openshift-default-6fcb86dbc-hl96f"
Oct 31 00:44:14.979478 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:14.979443    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/19acc006-2388-488d-bfff-6ce6bab123a8-istiod-ca-cert\") pod \"e2e-worker-w2-29ece5-openshift-default-6fcb86dbc-hl96f\" (UID: \"19acc006-2388-488d-bfff-6ce6bab123a8\") " pod="openshift-ingress/e2e-worker-w2-29ece5-openshift-default-6fcb86dbc-hl96f"
Oct 31 00:44:14.979587 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:14.979531    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/19acc006-2388-488d-bfff-6ce6bab123a8-credential-socket\") pod \"e2e-worker-w2-29ece5-openshift-default-6fcb86dbc-hl96f\" (UID: \"19acc006-2388-488d-bfff-6ce6bab123a8\") " pod="openshift-ingress/e2e-worker-w2-29ece5-openshift-default-6fcb86dbc-hl96f"
Oct 31 00:44:14.979587 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:14.979572    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/19acc006-2388-488d-bfff-6ce6bab123a8-istio-envoy\") pod \"e2e-worker-w2-29ece5-openshift-default-6fcb86dbc-hl96f\" (UID: \"19acc006-2388-488d-bfff-6ce6bab123a8\") " pod="openshift-ingress/e2e-worker-w2-29ece5-openshift-default-6fcb86dbc-hl96f"
Oct 31 00:44:15.001927 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:15.001888    4284 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w2-29ece5-openshift-default-dockercfg-6vh8l\""
Oct 31 00:44:15.052033 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:15.052002    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-29ece5-openshift-default-6fcb86dbc-hl96f"]
Oct 31 00:44:15.080396 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:15.080358    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jrs5w\" (UniqueName: \"kubernetes.io/projected/19acc006-2388-488d-bfff-6ce6bab123a8-kube-api-access-jrs5w\") pod \"e2e-worker-w2-29ece5-openshift-default-6fcb86dbc-hl96f\" (UID: \"19acc006-2388-488d-bfff-6ce6bab123a8\") " pod="openshift-ingress/e2e-worker-w2-29ece5-openshift-default-6fcb86dbc-hl96f"
Oct 31 00:44:15.080396 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:15.080396    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/19acc006-2388-488d-bfff-6ce6bab123a8-workload-certs\") pod \"e2e-worker-w2-29ece5-openshift-default-6fcb86dbc-hl96f\" (UID: \"19acc006-2388-488d-bfff-6ce6bab123a8\") " pod="openshift-ingress/e2e-worker-w2-29ece5-openshift-default-6fcb86dbc-hl96f"
Oct 31 00:44:15.080672 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:15.080426    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/19acc006-2388-488d-bfff-6ce6bab123a8-istio-data\") pod \"e2e-worker-w2-29ece5-openshift-default-6fcb86dbc-hl96f\" (UID: \"19acc006-2388-488d-bfff-6ce6bab123a8\") " pod="openshift-ingress/e2e-worker-w2-29ece5-openshift-default-6fcb86dbc-hl96f"
Oct 31 00:44:15.080672 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:15.080459    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/19acc006-2388-488d-bfff-6ce6bab123a8-istiod-ca-cert\") pod \"e2e-worker-w2-29ece5-openshift-default-6fcb86dbc-hl96f\" (UID: \"19acc006-2388-488d-bfff-6ce6bab123a8\") " pod="openshift-ingress/e2e-worker-w2-29ece5-openshift-default-6fcb86dbc-hl96f"
Oct 31 00:44:15.080672 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:15.080505    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/19acc006-2388-488d-bfff-6ce6bab123a8-credential-socket\") pod \"e2e-worker-w2-29ece5-openshift-default-6fcb86dbc-hl96f\" (UID: \"19acc006-2388-488d-bfff-6ce6bab123a8\") " pod="openshift-ingress/e2e-worker-w2-29ece5-openshift-default-6fcb86dbc-hl96f"
Oct 31 00:44:15.080672 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:15.080554    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/19acc006-2388-488d-bfff-6ce6bab123a8-istio-envoy\") pod \"e2e-worker-w2-29ece5-openshift-default-6fcb86dbc-hl96f\" (UID: \"19acc006-2388-488d-bfff-6ce6bab123a8\") " pod="openshift-ingress/e2e-worker-w2-29ece5-openshift-default-6fcb86dbc-hl96f"
Oct 31 00:44:15.080672 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:15.080584    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/19acc006-2388-488d-bfff-6ce6bab123a8-istio-podinfo\") pod \"e2e-worker-w2-29ece5-openshift-default-6fcb86dbc-hl96f\" (UID: \"19acc006-2388-488d-bfff-6ce6bab123a8\") " pod="openshift-ingress/e2e-worker-w2-29ece5-openshift-default-6fcb86dbc-hl96f"
Oct 31 00:44:15.080672 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:15.080634    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/19acc006-2388-488d-bfff-6ce6bab123a8-workload-socket\") pod \"e2e-worker-w2-29ece5-openshift-default-6fcb86dbc-hl96f\" (UID: \"19acc006-2388-488d-bfff-6ce6bab123a8\") " pod="openshift-ingress/e2e-worker-w2-29ece5-openshift-default-6fcb86dbc-hl96f"
Oct 31 00:44:15.080979 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:15.080699    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/19acc006-2388-488d-bfff-6ce6bab123a8-istio-token\") pod \"e2e-worker-w2-29ece5-openshift-default-6fcb86dbc-hl96f\" (UID: \"19acc006-2388-488d-bfff-6ce6bab123a8\") " pod="openshift-ingress/e2e-worker-w2-29ece5-openshift-default-6fcb86dbc-hl96f"
Oct 31 00:44:15.080979 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:15.080856    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/19acc006-2388-488d-bfff-6ce6bab123a8-workload-certs\") pod \"e2e-worker-w2-29ece5-openshift-default-6fcb86dbc-hl96f\" (UID: \"19acc006-2388-488d-bfff-6ce6bab123a8\") " pod="openshift-ingress/e2e-worker-w2-29ece5-openshift-default-6fcb86dbc-hl96f"
Oct 31 00:44:15.080979 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:15.080929    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/19acc006-2388-488d-bfff-6ce6bab123a8-credential-socket\") pod \"e2e-worker-w2-29ece5-openshift-default-6fcb86dbc-hl96f\" (UID: \"19acc006-2388-488d-bfff-6ce6bab123a8\") " pod="openshift-ingress/e2e-worker-w2-29ece5-openshift-default-6fcb86dbc-hl96f"
Oct 31 00:44:15.081143 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:15.081107    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/19acc006-2388-488d-bfff-6ce6bab123a8-workload-socket\") pod \"e2e-worker-w2-29ece5-openshift-default-6fcb86dbc-hl96f\" (UID: \"19acc006-2388-488d-bfff-6ce6bab123a8\") " pod="openshift-ingress/e2e-worker-w2-29ece5-openshift-default-6fcb86dbc-hl96f"
Oct 31 00:44:15.081196 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:15.081158    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/19acc006-2388-488d-bfff-6ce6bab123a8-istio-data\") pod \"e2e-worker-w2-29ece5-openshift-default-6fcb86dbc-hl96f\" (UID: \"19acc006-2388-488d-bfff-6ce6bab123a8\") " pod="openshift-ingress/e2e-worker-w2-29ece5-openshift-default-6fcb86dbc-hl96f"
Oct 31 00:44:15.081419 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:15.081385    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/19acc006-2388-488d-bfff-6ce6bab123a8-istiod-ca-cert\") pod \"e2e-worker-w2-29ece5-openshift-default-6fcb86dbc-hl96f\" (UID: \"19acc006-2388-488d-bfff-6ce6bab123a8\") " pod="openshift-ingress/e2e-worker-w2-29ece5-openshift-default-6fcb86dbc-hl96f"
Oct 31 00:44:15.083065 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:15.083040    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/19acc006-2388-488d-bfff-6ce6bab123a8-istio-envoy\") pod \"e2e-worker-w2-29ece5-openshift-default-6fcb86dbc-hl96f\" (UID: \"19acc006-2388-488d-bfff-6ce6bab123a8\") " pod="openshift-ingress/e2e-worker-w2-29ece5-openshift-default-6fcb86dbc-hl96f"
Oct 31 00:44:15.083413 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:15.083397    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/19acc006-2388-488d-bfff-6ce6bab123a8-istio-podinfo\") pod \"e2e-worker-w2-29ece5-openshift-default-6fcb86dbc-hl96f\" (UID: \"19acc006-2388-488d-bfff-6ce6bab123a8\") " pod="openshift-ingress/e2e-worker-w2-29ece5-openshift-default-6fcb86dbc-hl96f"
Oct 31 00:44:15.139940 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:15.139904    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jrs5w\" (UniqueName: \"kubernetes.io/projected/19acc006-2388-488d-bfff-6ce6bab123a8-kube-api-access-jrs5w\") pod \"e2e-worker-w2-29ece5-openshift-default-6fcb86dbc-hl96f\" (UID: \"19acc006-2388-488d-bfff-6ce6bab123a8\") " pod="openshift-ingress/e2e-worker-w2-29ece5-openshift-default-6fcb86dbc-hl96f"
Oct 31 00:44:15.140321 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:15.140290    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/19acc006-2388-488d-bfff-6ce6bab123a8-istio-token\") pod \"e2e-worker-w2-29ece5-openshift-default-6fcb86dbc-hl96f\" (UID: \"19acc006-2388-488d-bfff-6ce6bab123a8\") " pod="openshift-ingress/e2e-worker-w2-29ece5-openshift-default-6fcb86dbc-hl96f"
Oct 31 00:44:15.264431 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:15.264348    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-29ece5-openshift-default-6fcb86dbc-hl96f"
Oct 31 00:44:15.681068 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:15.681036    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-29ece5-openshift-default-6fcb86dbc-hl96f"]
Oct 31 00:44:15.682251 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:44:15.682196    4284 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod19acc006_2388_488d_bfff_6ce6bab123a8.slice/crio-511cc46aa456ba4633a13d1258deabe17a03dcd191811a49effce1d4838f38de WatchSource:0}: Error finding container 511cc46aa456ba4633a13d1258deabe17a03dcd191811a49effce1d4838f38de: Status 404 returned error can't find the container with id 511cc46aa456ba4633a13d1258deabe17a03dcd191811a49effce1d4838f38de
Oct 31 00:44:15.684602 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:15.684571    4284 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 00:44:15.684665 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:15.684637    4284 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 00:44:15.684705 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:15.684665    4284 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 00:44:16.475379 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:16.475343    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-29ece5-openshift-default-6fcb86dbc-hl96f" event={"ID":"19acc006-2388-488d-bfff-6ce6bab123a8","Type":"ContainerStarted","Data":"cc637542466f434173e4554b90b9e78e20e03f3c87b39f900985c90e5e524c3a"}
Oct 31 00:44:16.475822 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:16.475402    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-29ece5-openshift-default-6fcb86dbc-hl96f" event={"ID":"19acc006-2388-488d-bfff-6ce6bab123a8","Type":"ContainerStarted","Data":"511cc46aa456ba4633a13d1258deabe17a03dcd191811a49effce1d4838f38de"}
Oct 31 00:44:16.615826 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:16.615770    4284 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w2-29ece5-openshift-default-6fcb86dbc-hl96f" podStartSLOduration=2.615752476 podStartE2EDuration="2.615752476s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 00:44:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:44:16.614083416 +0000 UTC m=+970.759090334" watchObservedRunningTime="2026-10-31 00:44:16.615752476 +0000 UTC m=+970.760759392"
Oct 31 00:44:17.032321 ip-10-0-132-173 kubenswrapper[4284]: E1031 00:44:17.032288    4284 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fd8sp" containerName="extractor"
Oct 31 00:44:17.265478 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:17.265441    4284 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-29ece5-openshift-default-6fcb86dbc-hl96f"
Oct 31 00:44:17.269282 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:17.269237    4284 patch_prober.go:29] interesting pod/e2e-worker-w2-29ece5-openshift-default-6fcb86dbc-hl96f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" start-of-body=
Oct 31 00:44:17.269450 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:17.269301    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-29ece5-openshift-default-6fcb86dbc-hl96f" podUID="19acc006-2388-488d-bfff-6ce6bab123a8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused"
Oct 31 00:44:18.265454 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:18.265423    4284 patch_prober.go:29] interesting pod/e2e-worker-w2-29ece5-openshift-default-6fcb86dbc-hl96f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" start-of-body=
Oct 31 00:44:18.265839 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:18.265488    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-29ece5-openshift-default-6fcb86dbc-hl96f" podUID="19acc006-2388-488d-bfff-6ce6bab123a8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused"
Oct 31 00:44:19.039083 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:19.039048    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:44:19.265622 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:19.265584    4284 patch_prober.go:29] interesting pod/e2e-worker-w2-29ece5-openshift-default-6fcb86dbc-hl96f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" start-of-body=
Oct 31 00:44:19.266070 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:19.265635    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-29ece5-openshift-default-6fcb86dbc-hl96f" podUID="19acc006-2388-488d-bfff-6ce6bab123a8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused"
Oct 31 00:44:20.265240 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:20.265180    4284 patch_prober.go:29] interesting pod/e2e-worker-w2-29ece5-openshift-default-6fcb86dbc-hl96f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" start-of-body=
Oct 31 00:44:20.265433 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:20.265257    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-29ece5-openshift-default-6fcb86dbc-hl96f" podUID="19acc006-2388-488d-bfff-6ce6bab123a8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused"
Oct 31 00:44:21.265147 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:21.265108    4284 patch_prober.go:29] interesting pod/e2e-worker-w2-29ece5-openshift-default-6fcb86dbc-hl96f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" start-of-body=
Oct 31 00:44:21.265617 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:21.265169    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-29ece5-openshift-default-6fcb86dbc-hl96f" podUID="19acc006-2388-488d-bfff-6ce6bab123a8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused"
Oct 31 00:44:22.265121 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:22.265085    4284 patch_prober.go:29] interesting pod/e2e-worker-w2-29ece5-openshift-default-6fcb86dbc-hl96f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" start-of-body=
Oct 31 00:44:22.265341 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:22.265149    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-29ece5-openshift-default-6fcb86dbc-hl96f" podUID="19acc006-2388-488d-bfff-6ce6bab123a8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused"
Oct 31 00:44:22.796057 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:22.795979    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:44:22.997001 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:22.996969    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:44:23.151401 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:23.151310    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:44:23.480446 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:23.480411    4284 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-29ece5-openshift-default-6fcb86dbc-hl96f"
Oct 31 00:44:23.481346 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:23.481323    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-29ece5-openshift-default-6fcb86dbc-hl96f"
Oct 31 00:44:23.482676 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:23.482654    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-29ece5-openshift-default-6fcb86dbc-hl96f"
Oct 31 00:44:24.002752 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:24.002711    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:44:25.130109 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:25.130073    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:44:25.276560 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:25.276524    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:44:28.798404 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:28.798371    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:44:31.371407 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:31.371366    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:44:31.876537 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:31.876501    4284 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["models-as-a-service/payload-processing-external-model-56f84cd566-gzxbm"]
Oct 31 00:44:31.879894 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:31.879878    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="models-as-a-service/payload-processing-external-model-56f84cd566-gzxbm"
Oct 31 00:44:31.975681 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:31.975644    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/a5b686fc-3da6-40c0-9619-86357c50fd1e-routing-overlay-volume\") pod \"payload-processing-external-model-56f84cd566-gzxbm\" (UID: \"a5b686fc-3da6-40c0-9619-86357c50fd1e\") " pod="models-as-a-service/payload-processing-external-model-56f84cd566-gzxbm"
Oct 31 00:44:31.975681 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:31.975681    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a5b686fc-3da6-40c0-9619-86357c50fd1e-plugins-config-volume\") pod \"payload-processing-external-model-56f84cd566-gzxbm\" (UID: \"a5b686fc-3da6-40c0-9619-86357c50fd1e\") " pod="models-as-a-service/payload-processing-external-model-56f84cd566-gzxbm"
Oct 31 00:44:31.975873 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:31.975800    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/a5b686fc-3da6-40c0-9619-86357c50fd1e-provider-credentials\") pod \"payload-processing-external-model-56f84cd566-gzxbm\" (UID: \"a5b686fc-3da6-40c0-9619-86357c50fd1e\") " pod="models-as-a-service/payload-processing-external-model-56f84cd566-gzxbm"
Oct 31 00:44:31.975873 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:31.975856    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6bxnn\" (UniqueName: \"kubernetes.io/projected/a5b686fc-3da6-40c0-9619-86357c50fd1e-kube-api-access-6bxnn\") pod \"payload-processing-external-model-56f84cd566-gzxbm\" (UID: \"a5b686fc-3da6-40c0-9619-86357c50fd1e\") " pod="models-as-a-service/payload-processing-external-model-56f84cd566-gzxbm"
Oct 31 00:44:31.980470 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:31.980441    4284 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"models-as-a-service\"/\"agc-cred-bf4919d00529\""
Oct 31 00:44:31.980652 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:31.980441    4284 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"models-as-a-service\"/\"openshift-service-ca.crt\""
Oct 31 00:44:31.980652 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:31.980468    4284 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"models-as-a-service\"/\"routing-overlay\""
Oct 31 00:44:31.980853 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:31.980479    4284 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"models-as-a-service\"/\"kube-root-ca.crt\""
Oct 31 00:44:31.980853 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:31.980491    4284 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"models-as-a-service\"/\"payload-processing-external-model-plugins\""
Oct 31 00:44:31.980853 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:31.980498    4284 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"models-as-a-service\"/\"agc-cred-1a4f67076b57\""
Oct 31 00:44:31.982866 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:31.982832    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["models-as-a-service/payload-processing-external-model-56f84cd566-gzxbm"]
Oct 31 00:44:32.077418 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:32.077375    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6bxnn\" (UniqueName: \"kubernetes.io/projected/a5b686fc-3da6-40c0-9619-86357c50fd1e-kube-api-access-6bxnn\") pod \"payload-processing-external-model-56f84cd566-gzxbm\" (UID: \"a5b686fc-3da6-40c0-9619-86357c50fd1e\") " pod="models-as-a-service/payload-processing-external-model-56f84cd566-gzxbm"
Oct 31 00:44:32.077597 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:32.077485    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/a5b686fc-3da6-40c0-9619-86357c50fd1e-routing-overlay-volume\") pod \"payload-processing-external-model-56f84cd566-gzxbm\" (UID: \"a5b686fc-3da6-40c0-9619-86357c50fd1e\") " pod="models-as-a-service/payload-processing-external-model-56f84cd566-gzxbm"
Oct 31 00:44:32.077597 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:32.077518    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a5b686fc-3da6-40c0-9619-86357c50fd1e-plugins-config-volume\") pod \"payload-processing-external-model-56f84cd566-gzxbm\" (UID: \"a5b686fc-3da6-40c0-9619-86357c50fd1e\") " pod="models-as-a-service/payload-processing-external-model-56f84cd566-gzxbm"
Oct 31 00:44:32.077695 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:32.077658    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/a5b686fc-3da6-40c0-9619-86357c50fd1e-provider-credentials\") pod \"payload-processing-external-model-56f84cd566-gzxbm\" (UID: \"a5b686fc-3da6-40c0-9619-86357c50fd1e\") " pod="models-as-a-service/payload-processing-external-model-56f84cd566-gzxbm"
Oct 31 00:44:32.078171 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:32.078149    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/a5b686fc-3da6-40c0-9619-86357c50fd1e-routing-overlay-volume\") pod \"payload-processing-external-model-56f84cd566-gzxbm\" (UID: \"a5b686fc-3da6-40c0-9619-86357c50fd1e\") " pod="models-as-a-service/payload-processing-external-model-56f84cd566-gzxbm"
Oct 31 00:44:32.078355 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:32.078333    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a5b686fc-3da6-40c0-9619-86357c50fd1e-plugins-config-volume\") pod \"payload-processing-external-model-56f84cd566-gzxbm\" (UID: \"a5b686fc-3da6-40c0-9619-86357c50fd1e\") " pod="models-as-a-service/payload-processing-external-model-56f84cd566-gzxbm"
Oct 31 00:44:32.080325 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:32.080304    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/a5b686fc-3da6-40c0-9619-86357c50fd1e-provider-credentials\") pod \"payload-processing-external-model-56f84cd566-gzxbm\" (UID: \"a5b686fc-3da6-40c0-9619-86357c50fd1e\") " pod="models-as-a-service/payload-processing-external-model-56f84cd566-gzxbm"
Oct 31 00:44:32.136780 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:32.136707    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6bxnn\" (UniqueName: \"kubernetes.io/projected/a5b686fc-3da6-40c0-9619-86357c50fd1e-kube-api-access-6bxnn\") pod \"payload-processing-external-model-56f84cd566-gzxbm\" (UID: \"a5b686fc-3da6-40c0-9619-86357c50fd1e\") " pod="models-as-a-service/payload-processing-external-model-56f84cd566-gzxbm"
Oct 31 00:44:32.193598 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:32.193535    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="models-as-a-service/payload-processing-external-model-56f84cd566-gzxbm"
Oct 31 00:44:32.411652 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:32.411614    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:44:32.623605 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:32.623582    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["models-as-a-service/payload-processing-external-model-56f84cd566-gzxbm"]
Oct 31 00:44:32.625155 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:44:32.625131    4284 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda5b686fc_3da6_40c0_9619_86357c50fd1e.slice/crio-344a21ab19a139d03cc2142d230960ce5c23acef51152a68e02c4f1367c1b8cd WatchSource:0}: Error finding container 344a21ab19a139d03cc2142d230960ce5c23acef51152a68e02c4f1367c1b8cd: Status 404 returned error can't find the container with id 344a21ab19a139d03cc2142d230960ce5c23acef51152a68e02c4f1367c1b8cd
Oct 31 00:44:33.593134 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:33.593070    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:44:33.593635 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:33.593173    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="models-as-a-service/payload-processing-external-model-56f84cd566-gzxbm" event={"ID":"a5b686fc-3da6-40c0-9619-86357c50fd1e","Type":"ContainerStarted","Data":"dbbaf7d79605dcee18a5b1c7407f8a1ad606705ea7237bff8212989856d34f11"}
Oct 31 00:44:33.593635 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:33.593249    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="models-as-a-service/payload-processing-external-model-56f84cd566-gzxbm" event={"ID":"a5b686fc-3da6-40c0-9619-86357c50fd1e","Type":"ContainerStarted","Data":"344a21ab19a139d03cc2142d230960ce5c23acef51152a68e02c4f1367c1b8cd"}
Oct 31 00:44:33.593635 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:33.593415    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="models-as-a-service/payload-processing-external-model-56f84cd566-gzxbm"
Oct 31 00:44:33.707097 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:33.707047    4284 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="models-as-a-service/payload-processing-external-model-56f84cd566-gzxbm" podStartSLOduration=2.707031961 podStartE2EDuration="2.707031961s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 00:44:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:44:33.705710249 +0000 UTC m=+987.850717166" watchObservedRunningTime="2026-10-31 00:44:33.707031961 +0000 UTC m=+987.852038937"
Oct 31 00:44:37.354373 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:37.354337    4284 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-hvt6d"]
Oct 31 00:44:37.358170 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:37.358144    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-hvt6d"
Oct 31 00:44:37.401783 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:37.401754    4284 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-d4s6f"]
Oct 31 00:44:37.405486 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:37.405467    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-d4s6f"
Oct 31 00:44:37.443128 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:37.443092    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-grl28\" (UniqueName: \"kubernetes.io/projected/907b625f-2ccc-4de2-8181-34ff51ab2620-kube-api-access-grl28\") pod \"maas-default-gateway-openshift-default-6cc458d887-hvt6d\" (UID: \"907b625f-2ccc-4de2-8181-34ff51ab2620\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-hvt6d"
Oct 31 00:44:37.443128 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:37.443139    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/907b625f-2ccc-4de2-8181-34ff51ab2620-workload-socket\") pod \"maas-default-gateway-openshift-default-6cc458d887-hvt6d\" (UID: \"907b625f-2ccc-4de2-8181-34ff51ab2620\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-hvt6d"
Oct 31 00:44:37.443428 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:37.443164    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/907b625f-2ccc-4de2-8181-34ff51ab2620-istio-envoy\") pod \"maas-default-gateway-openshift-default-6cc458d887-hvt6d\" (UID: \"907b625f-2ccc-4de2-8181-34ff51ab2620\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-hvt6d"
Oct 31 00:44:37.443428 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:37.443314    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/907b625f-2ccc-4de2-8181-34ff51ab2620-istio-token\") pod \"maas-default-gateway-openshift-default-6cc458d887-hvt6d\" (UID: \"907b625f-2ccc-4de2-8181-34ff51ab2620\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-hvt6d"
Oct 31 00:44:37.443530 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:37.443435    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/907b625f-2ccc-4de2-8181-34ff51ab2620-credential-socket\") pod \"maas-default-gateway-openshift-default-6cc458d887-hvt6d\" (UID: \"907b625f-2ccc-4de2-8181-34ff51ab2620\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-hvt6d"
Oct 31 00:44:37.443530 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:37.443464    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/907b625f-2ccc-4de2-8181-34ff51ab2620-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-6cc458d887-hvt6d\" (UID: \"907b625f-2ccc-4de2-8181-34ff51ab2620\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-hvt6d"
Oct 31 00:44:37.443530 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:37.443517    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/907b625f-2ccc-4de2-8181-34ff51ab2620-workload-certs\") pod \"maas-default-gateway-openshift-default-6cc458d887-hvt6d\" (UID: \"907b625f-2ccc-4de2-8181-34ff51ab2620\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-hvt6d"
Oct 31 00:44:37.443634 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:37.443602    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/907b625f-2ccc-4de2-8181-34ff51ab2620-istio-data\") pod \"maas-default-gateway-openshift-default-6cc458d887-hvt6d\" (UID: \"907b625f-2ccc-4de2-8181-34ff51ab2620\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-hvt6d"
Oct 31 00:44:37.443668 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:37.443636    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/907b625f-2ccc-4de2-8181-34ff51ab2620-istio-podinfo\") pod \"maas-default-gateway-openshift-default-6cc458d887-hvt6d\" (UID: \"907b625f-2ccc-4de2-8181-34ff51ab2620\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-hvt6d"
Oct 31 00:44:37.456479 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:37.456445    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-hvt6d"]
Oct 31 00:44:37.504847 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:37.504814    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-d4s6f"]
Oct 31 00:44:37.544867 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:37.544832    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b398d9e9-9d78-4185-ac8c-4a9eecca9f48-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-6cc458d887-d4s6f\" (UID: \"b398d9e9-9d78-4185-ac8c-4a9eecca9f48\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-d4s6f"
Oct 31 00:44:37.545039 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:37.544876    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/907b625f-2ccc-4de2-8181-34ff51ab2620-workload-certs\") pod \"maas-default-gateway-openshift-default-6cc458d887-hvt6d\" (UID: \"907b625f-2ccc-4de2-8181-34ff51ab2620\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-hvt6d"
Oct 31 00:44:37.545039 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:37.544900    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/907b625f-2ccc-4de2-8181-34ff51ab2620-istio-data\") pod \"maas-default-gateway-openshift-default-6cc458d887-hvt6d\" (UID: \"907b625f-2ccc-4de2-8181-34ff51ab2620\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-hvt6d"
Oct 31 00:44:37.545039 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:37.544969    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b398d9e9-9d78-4185-ac8c-4a9eecca9f48-credential-socket\") pod \"maas-default-gateway-openshift-default-6cc458d887-d4s6f\" (UID: \"b398d9e9-9d78-4185-ac8c-4a9eecca9f48\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-d4s6f"
Oct 31 00:44:37.545039 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:37.545015    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/907b625f-2ccc-4de2-8181-34ff51ab2620-istio-podinfo\") pod \"maas-default-gateway-openshift-default-6cc458d887-hvt6d\" (UID: \"907b625f-2ccc-4de2-8181-34ff51ab2620\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-hvt6d"
Oct 31 00:44:37.545200 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:37.545077    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b398d9e9-9d78-4185-ac8c-4a9eecca9f48-istio-data\") pod \"maas-default-gateway-openshift-default-6cc458d887-d4s6f\" (UID: \"b398d9e9-9d78-4185-ac8c-4a9eecca9f48\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-d4s6f"
Oct 31 00:44:37.545200 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:37.545112    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b398d9e9-9d78-4185-ac8c-4a9eecca9f48-istio-token\") pod \"maas-default-gateway-openshift-default-6cc458d887-d4s6f\" (UID: \"b398d9e9-9d78-4185-ac8c-4a9eecca9f48\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-d4s6f"
Oct 31 00:44:37.545331 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:37.545272    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b398d9e9-9d78-4185-ac8c-4a9eecca9f48-istio-envoy\") pod \"maas-default-gateway-openshift-default-6cc458d887-d4s6f\" (UID: \"b398d9e9-9d78-4185-ac8c-4a9eecca9f48\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-d4s6f"
Oct 31 00:44:37.545331 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:37.545321    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-grl28\" (UniqueName: \"kubernetes.io/projected/907b625f-2ccc-4de2-8181-34ff51ab2620-kube-api-access-grl28\") pod \"maas-default-gateway-openshift-default-6cc458d887-hvt6d\" (UID: \"907b625f-2ccc-4de2-8181-34ff51ab2620\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-hvt6d"
Oct 31 00:44:37.545444 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:37.545403    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/907b625f-2ccc-4de2-8181-34ff51ab2620-istio-data\") pod \"maas-default-gateway-openshift-default-6cc458d887-hvt6d\" (UID: \"907b625f-2ccc-4de2-8181-34ff51ab2620\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-hvt6d"
Oct 31 00:44:37.545444 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:37.545407    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xkwp6\" (UniqueName: \"kubernetes.io/projected/b398d9e9-9d78-4185-ac8c-4a9eecca9f48-kube-api-access-xkwp6\") pod \"maas-default-gateway-openshift-default-6cc458d887-d4s6f\" (UID: \"b398d9e9-9d78-4185-ac8c-4a9eecca9f48\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-d4s6f"
Oct 31 00:44:37.545444 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:37.545418    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/907b625f-2ccc-4de2-8181-34ff51ab2620-workload-certs\") pod \"maas-default-gateway-openshift-default-6cc458d887-hvt6d\" (UID: \"907b625f-2ccc-4de2-8181-34ff51ab2620\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-hvt6d"
Oct 31 00:44:37.545585 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:37.545469    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/907b625f-2ccc-4de2-8181-34ff51ab2620-workload-socket\") pod \"maas-default-gateway-openshift-default-6cc458d887-hvt6d\" (UID: \"907b625f-2ccc-4de2-8181-34ff51ab2620\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-hvt6d"
Oct 31 00:44:37.545585 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:37.545520    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b398d9e9-9d78-4185-ac8c-4a9eecca9f48-workload-socket\") pod \"maas-default-gateway-openshift-default-6cc458d887-d4s6f\" (UID: \"b398d9e9-9d78-4185-ac8c-4a9eecca9f48\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-d4s6f"
Oct 31 00:44:37.545585 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:37.545550    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/907b625f-2ccc-4de2-8181-34ff51ab2620-istio-envoy\") pod \"maas-default-gateway-openshift-default-6cc458d887-hvt6d\" (UID: \"907b625f-2ccc-4de2-8181-34ff51ab2620\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-hvt6d"
Oct 31 00:44:37.545585 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:37.545566    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/907b625f-2ccc-4de2-8181-34ff51ab2620-istio-token\") pod \"maas-default-gateway-openshift-default-6cc458d887-hvt6d\" (UID: \"907b625f-2ccc-4de2-8181-34ff51ab2620\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-hvt6d"
Oct 31 00:44:37.545774 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:37.545661    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b398d9e9-9d78-4185-ac8c-4a9eecca9f48-workload-certs\") pod \"maas-default-gateway-openshift-default-6cc458d887-d4s6f\" (UID: \"b398d9e9-9d78-4185-ac8c-4a9eecca9f48\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-d4s6f"
Oct 31 00:44:37.545774 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:37.545760    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/907b625f-2ccc-4de2-8181-34ff51ab2620-workload-socket\") pod \"maas-default-gateway-openshift-default-6cc458d887-hvt6d\" (UID: \"907b625f-2ccc-4de2-8181-34ff51ab2620\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-hvt6d"
Oct 31 00:44:37.545855 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:37.545771    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/907b625f-2ccc-4de2-8181-34ff51ab2620-credential-socket\") pod \"maas-default-gateway-openshift-default-6cc458d887-hvt6d\" (UID: \"907b625f-2ccc-4de2-8181-34ff51ab2620\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-hvt6d"
Oct 31 00:44:37.545855 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:37.545833    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/907b625f-2ccc-4de2-8181-34ff51ab2620-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-6cc458d887-hvt6d\" (UID: \"907b625f-2ccc-4de2-8181-34ff51ab2620\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-hvt6d"
Oct 31 00:44:37.545928 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:37.545869    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b398d9e9-9d78-4185-ac8c-4a9eecca9f48-istio-podinfo\") pod \"maas-default-gateway-openshift-default-6cc458d887-d4s6f\" (UID: \"b398d9e9-9d78-4185-ac8c-4a9eecca9f48\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-d4s6f"
Oct 31 00:44:37.546024 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:37.546005    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/907b625f-2ccc-4de2-8181-34ff51ab2620-credential-socket\") pod \"maas-default-gateway-openshift-default-6cc458d887-hvt6d\" (UID: \"907b625f-2ccc-4de2-8181-34ff51ab2620\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-hvt6d"
Oct 31 00:44:37.546427 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:37.546410    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/907b625f-2ccc-4de2-8181-34ff51ab2620-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-6cc458d887-hvt6d\" (UID: \"907b625f-2ccc-4de2-8181-34ff51ab2620\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-hvt6d"
Oct 31 00:44:37.547701 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:37.547667    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/907b625f-2ccc-4de2-8181-34ff51ab2620-istio-podinfo\") pod \"maas-default-gateway-openshift-default-6cc458d887-hvt6d\" (UID: \"907b625f-2ccc-4de2-8181-34ff51ab2620\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-hvt6d"
Oct 31 00:44:37.547811 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:37.547687    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/907b625f-2ccc-4de2-8181-34ff51ab2620-istio-envoy\") pod \"maas-default-gateway-openshift-default-6cc458d887-hvt6d\" (UID: \"907b625f-2ccc-4de2-8181-34ff51ab2620\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-hvt6d"
Oct 31 00:44:37.604640 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:37.604565    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/907b625f-2ccc-4de2-8181-34ff51ab2620-istio-token\") pod \"maas-default-gateway-openshift-default-6cc458d887-hvt6d\" (UID: \"907b625f-2ccc-4de2-8181-34ff51ab2620\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-hvt6d"
Oct 31 00:44:37.604640 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:37.604631    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-grl28\" (UniqueName: \"kubernetes.io/projected/907b625f-2ccc-4de2-8181-34ff51ab2620-kube-api-access-grl28\") pod \"maas-default-gateway-openshift-default-6cc458d887-hvt6d\" (UID: \"907b625f-2ccc-4de2-8181-34ff51ab2620\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-hvt6d"
Oct 31 00:44:37.647341 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:37.647309    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b398d9e9-9d78-4185-ac8c-4a9eecca9f48-credential-socket\") pod \"maas-default-gateway-openshift-default-6cc458d887-d4s6f\" (UID: \"b398d9e9-9d78-4185-ac8c-4a9eecca9f48\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-d4s6f"
Oct 31 00:44:37.647547 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:37.647357    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b398d9e9-9d78-4185-ac8c-4a9eecca9f48-istio-data\") pod \"maas-default-gateway-openshift-default-6cc458d887-d4s6f\" (UID: \"b398d9e9-9d78-4185-ac8c-4a9eecca9f48\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-d4s6f"
Oct 31 00:44:37.647547 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:37.647376    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b398d9e9-9d78-4185-ac8c-4a9eecca9f48-istio-token\") pod \"maas-default-gateway-openshift-default-6cc458d887-d4s6f\" (UID: \"b398d9e9-9d78-4185-ac8c-4a9eecca9f48\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-d4s6f"
Oct 31 00:44:37.647547 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:37.647404    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b398d9e9-9d78-4185-ac8c-4a9eecca9f48-istio-envoy\") pod \"maas-default-gateway-openshift-default-6cc458d887-d4s6f\" (UID: \"b398d9e9-9d78-4185-ac8c-4a9eecca9f48\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-d4s6f"
Oct 31 00:44:37.647547 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:37.647444    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xkwp6\" (UniqueName: \"kubernetes.io/projected/b398d9e9-9d78-4185-ac8c-4a9eecca9f48-kube-api-access-xkwp6\") pod \"maas-default-gateway-openshift-default-6cc458d887-d4s6f\" (UID: \"b398d9e9-9d78-4185-ac8c-4a9eecca9f48\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-d4s6f"
Oct 31 00:44:37.647547 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:37.647468    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b398d9e9-9d78-4185-ac8c-4a9eecca9f48-workload-socket\") pod \"maas-default-gateway-openshift-default-6cc458d887-d4s6f\" (UID: \"b398d9e9-9d78-4185-ac8c-4a9eecca9f48\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-d4s6f"
Oct 31 00:44:37.647547 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:37.647531    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b398d9e9-9d78-4185-ac8c-4a9eecca9f48-workload-certs\") pod \"maas-default-gateway-openshift-default-6cc458d887-d4s6f\" (UID: \"b398d9e9-9d78-4185-ac8c-4a9eecca9f48\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-d4s6f"
Oct 31 00:44:37.647882 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:37.647587    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b398d9e9-9d78-4185-ac8c-4a9eecca9f48-istio-podinfo\") pod \"maas-default-gateway-openshift-default-6cc458d887-d4s6f\" (UID: \"b398d9e9-9d78-4185-ac8c-4a9eecca9f48\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-d4s6f"
Oct 31 00:44:37.647882 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:37.647633    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b398d9e9-9d78-4185-ac8c-4a9eecca9f48-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-6cc458d887-d4s6f\" (UID: \"b398d9e9-9d78-4185-ac8c-4a9eecca9f48\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-d4s6f"
Oct 31 00:44:37.647882 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:37.647840    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b398d9e9-9d78-4185-ac8c-4a9eecca9f48-istio-data\") pod \"maas-default-gateway-openshift-default-6cc458d887-d4s6f\" (UID: \"b398d9e9-9d78-4185-ac8c-4a9eecca9f48\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-d4s6f"
Oct 31 00:44:37.647989 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:37.647961    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b398d9e9-9d78-4185-ac8c-4a9eecca9f48-credential-socket\") pod \"maas-default-gateway-openshift-default-6cc458d887-d4s6f\" (UID: \"b398d9e9-9d78-4185-ac8c-4a9eecca9f48\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-d4s6f"
Oct 31 00:44:37.648024 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:37.647990    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b398d9e9-9d78-4185-ac8c-4a9eecca9f48-workload-certs\") pod \"maas-default-gateway-openshift-default-6cc458d887-d4s6f\" (UID: \"b398d9e9-9d78-4185-ac8c-4a9eecca9f48\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-d4s6f"
Oct 31 00:44:37.648024 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:37.648013    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b398d9e9-9d78-4185-ac8c-4a9eecca9f48-workload-socket\") pod \"maas-default-gateway-openshift-default-6cc458d887-d4s6f\" (UID: \"b398d9e9-9d78-4185-ac8c-4a9eecca9f48\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-d4s6f"
Oct 31 00:44:37.648457 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:37.648434    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b398d9e9-9d78-4185-ac8c-4a9eecca9f48-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-6cc458d887-d4s6f\" (UID: \"b398d9e9-9d78-4185-ac8c-4a9eecca9f48\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-d4s6f"
Oct 31 00:44:37.649961 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:37.649933    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b398d9e9-9d78-4185-ac8c-4a9eecca9f48-istio-envoy\") pod \"maas-default-gateway-openshift-default-6cc458d887-d4s6f\" (UID: \"b398d9e9-9d78-4185-ac8c-4a9eecca9f48\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-d4s6f"
Oct 31 00:44:37.650872 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:37.650852    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b398d9e9-9d78-4185-ac8c-4a9eecca9f48-istio-podinfo\") pod \"maas-default-gateway-openshift-default-6cc458d887-d4s6f\" (UID: \"b398d9e9-9d78-4185-ac8c-4a9eecca9f48\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-d4s6f"
Oct 31 00:44:37.673583 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:37.673545    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-hvt6d"
Oct 31 00:44:37.706160 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:37.706132    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b398d9e9-9d78-4185-ac8c-4a9eecca9f48-istio-token\") pod \"maas-default-gateway-openshift-default-6cc458d887-d4s6f\" (UID: \"b398d9e9-9d78-4185-ac8c-4a9eecca9f48\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-d4s6f"
Oct 31 00:44:37.706337 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:37.706261    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xkwp6\" (UniqueName: \"kubernetes.io/projected/b398d9e9-9d78-4185-ac8c-4a9eecca9f48-kube-api-access-xkwp6\") pod \"maas-default-gateway-openshift-default-6cc458d887-d4s6f\" (UID: \"b398d9e9-9d78-4185-ac8c-4a9eecca9f48\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-d4s6f"
Oct 31 00:44:37.719174 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:37.719141    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-d4s6f"
Oct 31 00:44:38.096305 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:38.096279    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-hvt6d"]
Oct 31 00:44:38.097368 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:44:38.097342    4284 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod907b625f_2ccc_4de2_8181_34ff51ab2620.slice/crio-eb62cd8f227970dd8f6e04676452a583440adcde365ebe8fd044a04d800e4bed WatchSource:0}: Error finding container eb62cd8f227970dd8f6e04676452a583440adcde365ebe8fd044a04d800e4bed: Status 404 returned error can't find the container with id eb62cd8f227970dd8f6e04676452a583440adcde365ebe8fd044a04d800e4bed
Oct 31 00:44:38.099897 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:38.099858    4284 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 00:44:38.100009 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:38.099969    4284 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 00:44:38.100072 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:38.100014    4284 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 00:44:38.145940 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:38.145912    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-d4s6f"]
Oct 31 00:44:38.154054 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:44:38.154025    4284 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb398d9e9_9d78_4185_ac8c_4a9eecca9f48.slice/crio-05794f8793eea658e70157b252cf72341a1beda36ecb4e6ab561020aec311618 WatchSource:0}: Error finding container 05794f8793eea658e70157b252cf72341a1beda36ecb4e6ab561020aec311618: Status 404 returned error can't find the container with id 05794f8793eea658e70157b252cf72341a1beda36ecb4e6ab561020aec311618
Oct 31 00:44:38.156648 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:38.156612    4284 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 00:44:38.156759 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:38.156687    4284 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 00:44:38.156759 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:38.156715    4284 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 00:44:38.399474 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:38.399370    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:44:38.621503 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:38.621457    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-d4s6f" event={"ID":"b398d9e9-9d78-4185-ac8c-4a9eecca9f48","Type":"ContainerStarted","Data":"532045cc890f889522b7e6812eb8a00980310d5d0f7b93cc2dab220d84eb52c8"}
Oct 31 00:44:38.621693 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:38.621519    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-d4s6f" event={"ID":"b398d9e9-9d78-4185-ac8c-4a9eecca9f48","Type":"ContainerStarted","Data":"05794f8793eea658e70157b252cf72341a1beda36ecb4e6ab561020aec311618"}
Oct 31 00:44:38.623315 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:38.623288    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-hvt6d" event={"ID":"907b625f-2ccc-4de2-8181-34ff51ab2620","Type":"ContainerStarted","Data":"1207c899f144a2a77001bd83adccc1f195da972fc68fa2cf9e2379f470b4102f"}
Oct 31 00:44:38.623438 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:38.623335    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-hvt6d" event={"ID":"907b625f-2ccc-4de2-8181-34ff51ab2620","Type":"ContainerStarted","Data":"eb62cd8f227970dd8f6e04676452a583440adcde365ebe8fd044a04d800e4bed"}
Oct 31 00:44:38.674494 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:38.674461    4284 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-hvt6d"
Oct 31 00:44:38.723813 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:38.721323    4284 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-d4s6f"
Oct 31 00:44:38.745126 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:38.745052    4284 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-d4s6f" podStartSLOduration=1.7450362529999999 podStartE2EDuration="1.745036253s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 00:44:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:44:38.742511522 +0000 UTC m=+992.887518454" watchObservedRunningTime="2026-10-31 00:44:38.745036253 +0000 UTC m=+992.890043168"
Oct 31 00:44:38.868324 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:38.868269    4284 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-hvt6d" podStartSLOduration=1.868252219 podStartE2EDuration="1.868252219s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 00:44:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:44:38.867440024 +0000 UTC m=+993.012446936" watchObservedRunningTime="2026-10-31 00:44:38.868252219 +0000 UTC m=+993.013259136"
Oct 31 00:44:39.675347 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:39.675317    4284 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6cc458d887-hvt6d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" start-of-body=
Oct 31 00:44:39.675760 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:39.675368    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-hvt6d" podUID="907b625f-2ccc-4de2-8181-34ff51ab2620" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused"
Oct 31 00:44:39.729066 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:39.721974    4284 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6cc458d887-d4s6f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused" start-of-body=
Oct 31 00:44:39.729066 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:39.722042    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-d4s6f" podUID="b398d9e9-9d78-4185-ac8c-4a9eecca9f48" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused"
Oct 31 00:44:40.674851 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:40.674812    4284 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6cc458d887-hvt6d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" start-of-body=
Oct 31 00:44:40.675036 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:40.674879    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-hvt6d" podUID="907b625f-2ccc-4de2-8181-34ff51ab2620" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused"
Oct 31 00:44:40.720238 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:40.719677    4284 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6cc458d887-d4s6f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused" start-of-body=
Oct 31 00:44:40.720238 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:40.719742    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-d4s6f" podUID="b398d9e9-9d78-4185-ac8c-4a9eecca9f48" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused"
Oct 31 00:44:41.674262 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:41.674199    4284 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6cc458d887-hvt6d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" start-of-body=
Oct 31 00:44:41.674471 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:41.674282    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-hvt6d" podUID="907b625f-2ccc-4de2-8181-34ff51ab2620" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused"
Oct 31 00:44:41.727238 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:41.726808    4284 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6cc458d887-d4s6f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused" start-of-body=
Oct 31 00:44:41.727238 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:41.726867    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-d4s6f" podUID="b398d9e9-9d78-4185-ac8c-4a9eecca9f48" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused"
Oct 31 00:44:42.438980 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:42.438908    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:44:42.675639 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:42.675602    4284 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6cc458d887-hvt6d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" start-of-body=
Oct 31 00:44:42.676272 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:42.676236    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-hvt6d" podUID="907b625f-2ccc-4de2-8181-34ff51ab2620" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused"
Oct 31 00:44:42.729299 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:42.724614    4284 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6cc458d887-d4s6f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused" start-of-body=
Oct 31 00:44:42.729299 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:42.724679    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-d4s6f" podUID="b398d9e9-9d78-4185-ac8c-4a9eecca9f48" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused"
Oct 31 00:44:43.519239 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:43.518784    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:44:43.674290 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:43.674253    4284 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6cc458d887-hvt6d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" start-of-body=
Oct 31 00:44:43.674504 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:43.674311    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-hvt6d" podUID="907b625f-2ccc-4de2-8181-34ff51ab2620" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused"
Oct 31 00:44:43.722723 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:43.722631    4284 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6cc458d887-d4s6f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused" start-of-body=
Oct 31 00:44:43.722723 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:43.722693    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-d4s6f" podUID="b398d9e9-9d78-4185-ac8c-4a9eecca9f48" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused"
Oct 31 00:44:44.603526 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:44.603481    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="models-as-a-service/payload-processing-external-model-56f84cd566-gzxbm"
Oct 31 00:44:44.674723 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:44.674687    4284 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6cc458d887-hvt6d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" start-of-body=
Oct 31 00:44:44.674927 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:44.674749    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-hvt6d" podUID="907b625f-2ccc-4de2-8181-34ff51ab2620" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused"
Oct 31 00:44:44.725144 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:44.725107    4284 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6cc458d887-d4s6f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused" start-of-body=
Oct 31 00:44:44.729034 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:44.728995    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-d4s6f" podUID="b398d9e9-9d78-4185-ac8c-4a9eecca9f48" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused"
Oct 31 00:44:45.674806 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:45.674767    4284 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6cc458d887-hvt6d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" start-of-body=
Oct 31 00:44:45.675242 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:45.674833    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-hvt6d" podUID="907b625f-2ccc-4de2-8181-34ff51ab2620" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused"
Oct 31 00:44:45.720014 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:45.719970    4284 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6cc458d887-d4s6f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused" start-of-body=
Oct 31 00:44:45.720246 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:45.720036    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-d4s6f" podUID="b398d9e9-9d78-4185-ac8c-4a9eecca9f48" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused"
Oct 31 00:44:47.050635 ip-10-0-132-173 kubenswrapper[4284]: E1031 00:44:47.050597    4284 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fd8sp" containerName="extractor"
Oct 31 00:44:47.346913 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:47.346831    4284 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-d4s6f"
Oct 31 00:44:47.348186 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:47.348164    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-d4s6f"
Oct 31 00:44:47.349310 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:47.349285    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-d4s6f"
Oct 31 00:44:47.416112 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:47.416085    4284 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-hvt6d"
Oct 31 00:44:47.416482 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:47.416460    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-hvt6d"
Oct 31 00:44:47.417540 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:47.417517    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-hvt6d"
Oct 31 00:44:48.538238 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:48.537940    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:44:52.490121 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:52.490080    4284 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-r7hr5"]
Oct 31 00:44:52.495379 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:52.495355    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-r7hr5"
Oct 31 00:44:52.595504 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:52.595465    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-r7hr5"]
Oct 31 00:44:52.602726 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:52.602687    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6cce1cfc-7109-4827-a231-fe01dd9a4bf9-workload-certs\") pod \"maas-default-gateway-openshift-default-6cc458d887-r7hr5\" (UID: \"6cce1cfc-7109-4827-a231-fe01dd9a4bf9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-r7hr5"
Oct 31 00:44:52.602899 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:52.602753    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6cce1cfc-7109-4827-a231-fe01dd9a4bf9-credential-socket\") pod \"maas-default-gateway-openshift-default-6cc458d887-r7hr5\" (UID: \"6cce1cfc-7109-4827-a231-fe01dd9a4bf9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-r7hr5"
Oct 31 00:44:52.602899 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:52.602824    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6cce1cfc-7109-4827-a231-fe01dd9a4bf9-istio-envoy\") pod \"maas-default-gateway-openshift-default-6cc458d887-r7hr5\" (UID: \"6cce1cfc-7109-4827-a231-fe01dd9a4bf9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-r7hr5"
Oct 31 00:44:52.603026 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:52.602891    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6cce1cfc-7109-4827-a231-fe01dd9a4bf9-istio-data\") pod \"maas-default-gateway-openshift-default-6cc458d887-r7hr5\" (UID: \"6cce1cfc-7109-4827-a231-fe01dd9a4bf9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-r7hr5"
Oct 31 00:44:52.603026 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:52.602936    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6vpff\" (UniqueName: \"kubernetes.io/projected/6cce1cfc-7109-4827-a231-fe01dd9a4bf9-kube-api-access-6vpff\") pod \"maas-default-gateway-openshift-default-6cc458d887-r7hr5\" (UID: \"6cce1cfc-7109-4827-a231-fe01dd9a4bf9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-r7hr5"
Oct 31 00:44:52.603026 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:52.602979    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6cce1cfc-7109-4827-a231-fe01dd9a4bf9-istio-token\") pod \"maas-default-gateway-openshift-default-6cc458d887-r7hr5\" (UID: \"6cce1cfc-7109-4827-a231-fe01dd9a4bf9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-r7hr5"
Oct 31 00:44:52.603190 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:52.603049    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6cce1cfc-7109-4827-a231-fe01dd9a4bf9-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-6cc458d887-r7hr5\" (UID: \"6cce1cfc-7109-4827-a231-fe01dd9a4bf9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-r7hr5"
Oct 31 00:44:52.603190 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:52.603091    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6cce1cfc-7109-4827-a231-fe01dd9a4bf9-istio-podinfo\") pod \"maas-default-gateway-openshift-default-6cc458d887-r7hr5\" (UID: \"6cce1cfc-7109-4827-a231-fe01dd9a4bf9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-r7hr5"
Oct 31 00:44:52.603190 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:52.603108    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6cce1cfc-7109-4827-a231-fe01dd9a4bf9-workload-socket\") pod \"maas-default-gateway-openshift-default-6cc458d887-r7hr5\" (UID: \"6cce1cfc-7109-4827-a231-fe01dd9a4bf9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-r7hr5"
Oct 31 00:44:52.704496 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:52.704462    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6cce1cfc-7109-4827-a231-fe01dd9a4bf9-istio-data\") pod \"maas-default-gateway-openshift-default-6cc458d887-r7hr5\" (UID: \"6cce1cfc-7109-4827-a231-fe01dd9a4bf9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-r7hr5"
Oct 31 00:44:52.704712 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:52.704514    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6vpff\" (UniqueName: \"kubernetes.io/projected/6cce1cfc-7109-4827-a231-fe01dd9a4bf9-kube-api-access-6vpff\") pod \"maas-default-gateway-openshift-default-6cc458d887-r7hr5\" (UID: \"6cce1cfc-7109-4827-a231-fe01dd9a4bf9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-r7hr5"
Oct 31 00:44:52.704712 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:52.704548    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6cce1cfc-7109-4827-a231-fe01dd9a4bf9-istio-token\") pod \"maas-default-gateway-openshift-default-6cc458d887-r7hr5\" (UID: \"6cce1cfc-7109-4827-a231-fe01dd9a4bf9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-r7hr5"
Oct 31 00:44:52.704712 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:52.704570    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6cce1cfc-7109-4827-a231-fe01dd9a4bf9-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-6cc458d887-r7hr5\" (UID: \"6cce1cfc-7109-4827-a231-fe01dd9a4bf9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-r7hr5"
Oct 31 00:44:52.704712 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:52.704608    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6cce1cfc-7109-4827-a231-fe01dd9a4bf9-istio-podinfo\") pod \"maas-default-gateway-openshift-default-6cc458d887-r7hr5\" (UID: \"6cce1cfc-7109-4827-a231-fe01dd9a4bf9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-r7hr5"
Oct 31 00:44:52.704712 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:52.704628    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6cce1cfc-7109-4827-a231-fe01dd9a4bf9-workload-socket\") pod \"maas-default-gateway-openshift-default-6cc458d887-r7hr5\" (UID: \"6cce1cfc-7109-4827-a231-fe01dd9a4bf9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-r7hr5"
Oct 31 00:44:52.704991 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:52.704718    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6cce1cfc-7109-4827-a231-fe01dd9a4bf9-workload-certs\") pod \"maas-default-gateway-openshift-default-6cc458d887-r7hr5\" (UID: \"6cce1cfc-7109-4827-a231-fe01dd9a4bf9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-r7hr5"
Oct 31 00:44:52.704991 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:52.704755    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6cce1cfc-7109-4827-a231-fe01dd9a4bf9-credential-socket\") pod \"maas-default-gateway-openshift-default-6cc458d887-r7hr5\" (UID: \"6cce1cfc-7109-4827-a231-fe01dd9a4bf9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-r7hr5"
Oct 31 00:44:52.704991 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:52.704783    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6cce1cfc-7109-4827-a231-fe01dd9a4bf9-istio-envoy\") pod \"maas-default-gateway-openshift-default-6cc458d887-r7hr5\" (UID: \"6cce1cfc-7109-4827-a231-fe01dd9a4bf9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-r7hr5"
Oct 31 00:44:52.704991 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:52.704958    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6cce1cfc-7109-4827-a231-fe01dd9a4bf9-istio-data\") pod \"maas-default-gateway-openshift-default-6cc458d887-r7hr5\" (UID: \"6cce1cfc-7109-4827-a231-fe01dd9a4bf9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-r7hr5"
Oct 31 00:44:52.705149 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:52.705122    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6cce1cfc-7109-4827-a231-fe01dd9a4bf9-workload-socket\") pod \"maas-default-gateway-openshift-default-6cc458d887-r7hr5\" (UID: \"6cce1cfc-7109-4827-a231-fe01dd9a4bf9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-r7hr5"
Oct 31 00:44:52.705199 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:52.705146    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6cce1cfc-7109-4827-a231-fe01dd9a4bf9-workload-certs\") pod \"maas-default-gateway-openshift-default-6cc458d887-r7hr5\" (UID: \"6cce1cfc-7109-4827-a231-fe01dd9a4bf9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-r7hr5"
Oct 31 00:44:52.705259 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:52.705198    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6cce1cfc-7109-4827-a231-fe01dd9a4bf9-credential-socket\") pod \"maas-default-gateway-openshift-default-6cc458d887-r7hr5\" (UID: \"6cce1cfc-7109-4827-a231-fe01dd9a4bf9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-r7hr5"
Oct 31 00:44:52.705817 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:52.705791    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6cce1cfc-7109-4827-a231-fe01dd9a4bf9-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-6cc458d887-r7hr5\" (UID: \"6cce1cfc-7109-4827-a231-fe01dd9a4bf9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-r7hr5"
Oct 31 00:44:52.707251 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:52.707191    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6cce1cfc-7109-4827-a231-fe01dd9a4bf9-istio-envoy\") pod \"maas-default-gateway-openshift-default-6cc458d887-r7hr5\" (UID: \"6cce1cfc-7109-4827-a231-fe01dd9a4bf9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-r7hr5"
Oct 31 00:44:52.707355 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:52.707329    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6cce1cfc-7109-4827-a231-fe01dd9a4bf9-istio-podinfo\") pod \"maas-default-gateway-openshift-default-6cc458d887-r7hr5\" (UID: \"6cce1cfc-7109-4827-a231-fe01dd9a4bf9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-r7hr5"
Oct 31 00:44:52.763432 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:52.763356    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6cce1cfc-7109-4827-a231-fe01dd9a4bf9-istio-token\") pod \"maas-default-gateway-openshift-default-6cc458d887-r7hr5\" (UID: \"6cce1cfc-7109-4827-a231-fe01dd9a4bf9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-r7hr5"
Oct 31 00:44:52.763599 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:52.763432    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6vpff\" (UniqueName: \"kubernetes.io/projected/6cce1cfc-7109-4827-a231-fe01dd9a4bf9-kube-api-access-6vpff\") pod \"maas-default-gateway-openshift-default-6cc458d887-r7hr5\" (UID: \"6cce1cfc-7109-4827-a231-fe01dd9a4bf9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-r7hr5"
Oct 31 00:44:52.808034 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:52.808002    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-r7hr5"
Oct 31 00:44:53.224044 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:53.224015    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-r7hr5"]
Oct 31 00:44:53.225234 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:44:53.225193    4284 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6cce1cfc_7109_4827_a231_fe01dd9a4bf9.slice/crio-9da644d0541ae5f1391e81f9db8d0790c32cedc5f4125f2bbe34b83f7759914d WatchSource:0}: Error finding container 9da644d0541ae5f1391e81f9db8d0790c32cedc5f4125f2bbe34b83f7759914d: Status 404 returned error can't find the container with id 9da644d0541ae5f1391e81f9db8d0790c32cedc5f4125f2bbe34b83f7759914d
Oct 31 00:44:53.227723 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:53.227693    4284 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 00:44:53.227799 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:53.227754    4284 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 00:44:53.227799 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:53.227781    4284 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 00:44:53.390862 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:53.390816    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:44:53.731634 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:53.731599    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-r7hr5" event={"ID":"6cce1cfc-7109-4827-a231-fe01dd9a4bf9","Type":"ContainerStarted","Data":"7f51bafb86f4a8d560e43f5c45adb5803d37b3d0626fc00957bedeb001510d17"}
Oct 31 00:44:53.732059 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:53.731666    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-r7hr5" event={"ID":"6cce1cfc-7109-4827-a231-fe01dd9a4bf9","Type":"ContainerStarted","Data":"9da644d0541ae5f1391e81f9db8d0790c32cedc5f4125f2bbe34b83f7759914d"}
Oct 31 00:44:53.810383 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:53.810340    4284 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-r7hr5"
Oct 31 00:44:53.851164 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:53.851112    4284 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-r7hr5" podStartSLOduration=1.851096437 podStartE2EDuration="1.851096437s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 00:44:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:44:53.848632733 +0000 UTC m=+1007.993639660" watchObservedRunningTime="2026-10-31 00:44:53.851096437 +0000 UTC m=+1007.996103353"
Oct 31 00:44:54.342710 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:54.342676    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:44:54.815237 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:54.814339    4284 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6cc458d887-r7hr5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused" start-of-body=
Oct 31 00:44:54.815237 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:54.814410    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-r7hr5" podUID="6cce1cfc-7109-4827-a231-fe01dd9a4bf9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused"
Oct 31 00:44:55.815238 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:55.811849    4284 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6cc458d887-r7hr5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused" start-of-body=
Oct 31 00:44:55.815238 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:55.811908    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-r7hr5" podUID="6cce1cfc-7109-4827-a231-fe01dd9a4bf9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused"
Oct 31 00:44:56.830240 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:56.827734    4284 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6cc458d887-r7hr5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused" start-of-body=
Oct 31 00:44:56.830240 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:56.827796    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-r7hr5" podUID="6cce1cfc-7109-4827-a231-fe01dd9a4bf9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused"
Oct 31 00:44:57.811751 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:57.811716    4284 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6cc458d887-r7hr5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused" start-of-body=
Oct 31 00:44:57.812032 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:57.812002    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-r7hr5" podUID="6cce1cfc-7109-4827-a231-fe01dd9a4bf9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused"
Oct 31 00:44:58.727487 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:58.727449    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:44:58.824696 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:58.824667    4284 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6cc458d887-r7hr5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused" start-of-body=
Oct 31 00:44:58.824896 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:58.824721    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-r7hr5" podUID="6cce1cfc-7109-4827-a231-fe01dd9a4bf9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused"
Oct 31 00:44:59.808709 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:59.808672    4284 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6cc458d887-r7hr5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused" start-of-body=
Oct 31 00:44:59.809275 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:44:59.808739    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-r7hr5" podUID="6cce1cfc-7109-4827-a231-fe01dd9a4bf9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused"
Oct 31 00:45:00.208943 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:45:00.208910    4284 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-29ece5-29890125-cbvn7"]
Oct 31 00:45:00.212906 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:45:00.212882    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-29ece5-29890125-cbvn7"
Oct 31 00:45:00.265620 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:45:00.265583    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/32a7e4d5-1e58-4073-81f1-57a9b7b71999-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w2-29ece5-29890125-cbvn7\" (UID: \"32a7e4d5-1e58-4073-81f1-57a9b7b71999\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-29ece5-29890125-cbvn7"
Oct 31 00:45:00.265828 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:45:00.265695    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-28v9s\" (UniqueName: \"kubernetes.io/projected/32a7e4d5-1e58-4073-81f1-57a9b7b71999-kube-api-access-28v9s\") pod \"maas-api-key-cleanup-e2e-worker-w2-29ece5-29890125-cbvn7\" (UID: \"32a7e4d5-1e58-4073-81f1-57a9b7b71999\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-29ece5-29890125-cbvn7"
Oct 31 00:45:00.266746 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:45:00.266713    4284 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-8pzqp\""
Oct 31 00:45:00.367091 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:45:00.367060    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/32a7e4d5-1e58-4073-81f1-57a9b7b71999-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w2-29ece5-29890125-cbvn7\" (UID: \"32a7e4d5-1e58-4073-81f1-57a9b7b71999\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-29ece5-29890125-cbvn7"
Oct 31 00:45:00.367314 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:45:00.367107    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-28v9s\" (UniqueName: \"kubernetes.io/projected/32a7e4d5-1e58-4073-81f1-57a9b7b71999-kube-api-access-28v9s\") pod \"maas-api-key-cleanup-e2e-worker-w2-29ece5-29890125-cbvn7\" (UID: \"32a7e4d5-1e58-4073-81f1-57a9b7b71999\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-29ece5-29890125-cbvn7"
Oct 31 00:45:00.367768 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:45:00.367741    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/32a7e4d5-1e58-4073-81f1-57a9b7b71999-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w2-29ece5-29890125-cbvn7\" (UID: \"32a7e4d5-1e58-4073-81f1-57a9b7b71999\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-29ece5-29890125-cbvn7"
Oct 31 00:45:00.429450 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:45:00.428818    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-29ece5-29890125-cbvn7"]
Oct 31 00:45:00.439459 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:45:00.439428    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-28v9s\" (UniqueName: \"kubernetes.io/projected/32a7e4d5-1e58-4073-81f1-57a9b7b71999-kube-api-access-28v9s\") pod \"maas-api-key-cleanup-e2e-worker-w2-29ece5-29890125-cbvn7\" (UID: \"32a7e4d5-1e58-4073-81f1-57a9b7b71999\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-29ece5-29890125-cbvn7"
Oct 31 00:45:00.526149 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:45:00.526072    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-29ece5-29890125-cbvn7"
Oct 31 00:45:00.808936 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:45:00.808856    4284 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6cc458d887-r7hr5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused" start-of-body=
Oct 31 00:45:00.808936 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:45:00.808921    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-r7hr5" podUID="6cce1cfc-7109-4827-a231-fe01dd9a4bf9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused"
Oct 31 00:45:00.898324 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:45:00.898292    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-29ece5-29890125-cbvn7"]
Oct 31 00:45:00.900312 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:45:00.900281    4284 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod32a7e4d5_1e58_4073_81f1_57a9b7b71999.slice/crio-af76aca6996ff578b4714d9e130fb9b26effa76d5f5f78b1202f373df744f7a1 WatchSource:0}: Error finding container af76aca6996ff578b4714d9e130fb9b26effa76d5f5f78b1202f373df744f7a1: Status 404 returned error can't find the container with id af76aca6996ff578b4714d9e130fb9b26effa76d5f5f78b1202f373df744f7a1
Oct 31 00:45:00.902370 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:45:00.902353    4284 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 31 00:45:01.843570 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:45:01.843529    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-29ece5-29890125-cbvn7" event={"ID":"32a7e4d5-1e58-4073-81f1-57a9b7b71999","Type":"ContainerStarted","Data":"af76aca6996ff578b4714d9e130fb9b26effa76d5f5f78b1202f373df744f7a1"}
Oct 31 00:45:01.898412 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:45:01.898379    4284 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-r7hr5"
Oct 31 00:45:01.898823 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:45:01.898790    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-r7hr5"
Oct 31 00:45:01.899643 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:45:01.899623    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-r7hr5"
Oct 31 00:45:03.598152 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:45:03.598110    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:45:05.871325 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:45:05.871225    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-29ece5-29890125-cbvn7" event={"ID":"32a7e4d5-1e58-4073-81f1-57a9b7b71999","Type":"ContainerStarted","Data":"294f8f5ad83d20de7f7e0c62ec6303178e241296e91cdf6d842d74042d307e96"}
Oct 31 00:45:05.983998 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:45:05.983948    4284 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-29ece5-29890125-cbvn7" podStartSLOduration=1.699755487 podStartE2EDuration="5.983934807s" totalImagesPullingTime="4.28417932s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 00:45:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:45:05.981724848 +0000 UTC m=+1020.126731763" watchObservedRunningTime="2026-10-31 00:45:05.983934807 +0000 UTC m=+1020.128941761"
Oct 31 00:45:06.878194 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:45:06.878156    4284 generic.go:356] "Generic (PLEG): container finished" podID="32a7e4d5-1e58-4073-81f1-57a9b7b71999" containerID="294f8f5ad83d20de7f7e0c62ec6303178e241296e91cdf6d842d74042d307e96" exitCode=60
Oct 31 00:45:06.878616 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:45:06.878239    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-29ece5-29890125-cbvn7" event={"ID":"32a7e4d5-1e58-4073-81f1-57a9b7b71999","Type":"ContainerDied","Data":"294f8f5ad83d20de7f7e0c62ec6303178e241296e91cdf6d842d74042d307e96"}
Oct 31 00:45:06.878616 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:45:06.878545    4284 scope.go:122] "RemoveContainer" containerID="294f8f5ad83d20de7f7e0c62ec6303178e241296e91cdf6d842d74042d307e96"
Oct 31 00:45:07.885481 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:45:07.885454    4284 generic.go:356] "Generic (PLEG): container finished" podID="32a7e4d5-1e58-4073-81f1-57a9b7b71999" containerID="d24ca3b9b3137d02e15bc847e064687e838b8fcebfa3c2682e4913cb4db43e7d" exitCode=60
Oct 31 00:45:07.885888 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:45:07.885523    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-29ece5-29890125-cbvn7" event={"ID":"32a7e4d5-1e58-4073-81f1-57a9b7b71999","Type":"ContainerDied","Data":"d24ca3b9b3137d02e15bc847e064687e838b8fcebfa3c2682e4913cb4db43e7d"}
Oct 31 00:45:07.885888 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:45:07.885601    4284 scope.go:122] "RemoveContainer" containerID="294f8f5ad83d20de7f7e0c62ec6303178e241296e91cdf6d842d74042d307e96"
Oct 31 00:45:07.885888 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:45:07.885870    4284 scope.go:122] "RemoveContainer" containerID="d24ca3b9b3137d02e15bc847e064687e838b8fcebfa3c2682e4913cb4db43e7d"
Oct 31 00:45:07.886185 ip-10-0-132-173 kubenswrapper[4284]: E1031 00:45:07.886165    4284 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w2-29ece5-29890125-cbvn7_odh-ai-gateway-infra(32a7e4d5-1e58-4073-81f1-57a9b7b71999)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-29ece5-29890125-cbvn7" podUID="32a7e4d5-1e58-4073-81f1-57a9b7b71999"
Oct 31 00:45:08.381879 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:45:08.381839    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:45:08.892802 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:45:08.892773    4284 scope.go:122] "RemoveContainer" containerID="d24ca3b9b3137d02e15bc847e064687e838b8fcebfa3c2682e4913cb4db43e7d"
Oct 31 00:45:08.893181 ip-10-0-132-173 kubenswrapper[4284]: E1031 00:45:08.892994    4284 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w2-29ece5-29890125-cbvn7_odh-ai-gateway-infra(32a7e4d5-1e58-4073-81f1-57a9b7b71999)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-29ece5-29890125-cbvn7" podUID="32a7e4d5-1e58-4073-81f1-57a9b7b71999"
Oct 31 00:45:13.105754 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:45:13.105713    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:45:17.013649 ip-10-0-132-173 kubenswrapper[4284]: E1031 00:45:17.013562    4284 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fd8sp" containerName="extractor"
Oct 31 00:45:18.019068 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:45:18.019030    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:45:19.956142 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:45:19.956109    4284 scope.go:122] "RemoveContainer" containerID="d24ca3b9b3137d02e15bc847e064687e838b8fcebfa3c2682e4913cb4db43e7d"
Oct 31 00:45:20.956392 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:45:20.956360    4284 generic.go:356] "Generic (PLEG): container finished" podID="32a7e4d5-1e58-4073-81f1-57a9b7b71999" containerID="9905115de216cf19a48caa29a262cbaecac89674250114b47bd8679e4a7d6dda" exitCode=60
Oct 31 00:45:20.956820 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:45:20.956434    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-29ece5-29890125-cbvn7" event={"ID":"32a7e4d5-1e58-4073-81f1-57a9b7b71999","Type":"ContainerDied","Data":"9905115de216cf19a48caa29a262cbaecac89674250114b47bd8679e4a7d6dda"}
Oct 31 00:45:20.956820 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:45:20.956487    4284 scope.go:122] "RemoveContainer" containerID="d24ca3b9b3137d02e15bc847e064687e838b8fcebfa3c2682e4913cb4db43e7d"
Oct 31 00:45:20.956820 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:45:20.956699    4284 scope.go:122] "RemoveContainer" containerID="9905115de216cf19a48caa29a262cbaecac89674250114b47bd8679e4a7d6dda"
Oct 31 00:45:20.956956 ip-10-0-132-173 kubenswrapper[4284]: E1031 00:45:20.956888    4284 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w2-29ece5-29890125-cbvn7_odh-ai-gateway-infra(32a7e4d5-1e58-4073-81f1-57a9b7b71999)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-29ece5-29890125-cbvn7" podUID="32a7e4d5-1e58-4073-81f1-57a9b7b71999"
Oct 31 00:45:22.077657 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:45:22.077622    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-29ece5-29890125-cbvn7"]
Oct 31 00:45:22.221568 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:45:22.221545    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-29ece5-29890125-cbvn7"
Oct 31 00:45:22.321240 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:45:22.321186    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/32a7e4d5-1e58-4073-81f1-57a9b7b71999-kube-api-access-28v9s\" (UniqueName: \"kubernetes.io/projected/32a7e4d5-1e58-4073-81f1-57a9b7b71999-kube-api-access-28v9s\") pod \"32a7e4d5-1e58-4073-81f1-57a9b7b71999\" (UID: \"32a7e4d5-1e58-4073-81f1-57a9b7b71999\") "
Oct 31 00:45:22.321414 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:45:22.321277    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/32a7e4d5-1e58-4073-81f1-57a9b7b71999-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/32a7e4d5-1e58-4073-81f1-57a9b7b71999-maas-api-service-ca\") pod \"32a7e4d5-1e58-4073-81f1-57a9b7b71999\" (UID: \"32a7e4d5-1e58-4073-81f1-57a9b7b71999\") "
Oct 31 00:45:22.321823 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:45:22.321800    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/32a7e4d5-1e58-4073-81f1-57a9b7b71999-maas-api-service-ca" pod "32a7e4d5-1e58-4073-81f1-57a9b7b71999" (UID: "32a7e4d5-1e58-4073-81f1-57a9b7b71999"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 00:45:22.323409 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:45:22.323386    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/32a7e4d5-1e58-4073-81f1-57a9b7b71999-kube-api-access-28v9s" pod "32a7e4d5-1e58-4073-81f1-57a9b7b71999" (UID: "32a7e4d5-1e58-4073-81f1-57a9b7b71999"). InnerVolumeSpecName "kube-api-access-28v9s". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 00:45:22.422692 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:45:22.422664    4284 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-28v9s\" (UniqueName: \"kubernetes.io/projected/32a7e4d5-1e58-4073-81f1-57a9b7b71999-kube-api-access-28v9s\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:45:22.422692 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:45:22.422689    4284 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/32a7e4d5-1e58-4073-81f1-57a9b7b71999-maas-api-service-ca\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:45:22.799633 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:45:22.799601    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:45:22.977342 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:45:22.977313    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-29ece5-29890125-cbvn7"
Oct 31 00:45:22.977535 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:45:22.977350    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-29ece5-29890125-cbvn7" event={"ID":"32a7e4d5-1e58-4073-81f1-57a9b7b71999","Type":"ContainerDied","Data":"af76aca6996ff578b4714d9e130fb9b26effa76d5f5f78b1202f373df744f7a1"}
Oct 31 00:45:22.977535 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:45:22.977413    4284 scope.go:122] "RemoveContainer" containerID="9905115de216cf19a48caa29a262cbaecac89674250114b47bd8679e4a7d6dda"
Oct 31 00:45:23.136601 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:45:23.136511    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-29ece5-29890125-cbvn7"]
Oct 31 00:45:23.184066 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:45:23.184036    4284 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-29ece5-29890125-cbvn7"]
Oct 31 00:45:23.962965 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:45:23.962925    4284 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="32a7e4d5-1e58-4073-81f1-57a9b7b71999" path="/var/lib/kubelet/pods/32a7e4d5-1e58-4073-81f1-57a9b7b71999/volumes"
Oct 31 00:45:24.677513 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:45:24.677472    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:45:27.601985 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:45:27.601946    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:45:27.820424 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:45:27.820386    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:45:30.060273 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:45:30.060235    4284 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w2-29ece5-openshift-default-6fcb86dbc-n2k5v"]
Oct 31 00:45:30.060715 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:45:30.060703    4284 cpu_manager.go:414] "containerMap: removing container" podUID="32a7e4d5-1e58-4073-81f1-57a9b7b71999" containerName="cleanup"
Oct 31 00:45:30.060775 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:45:30.060717    4284 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="32a7e4d5-1e58-4073-81f1-57a9b7b71999" containerName="cleanup"
Oct 31 00:45:30.060775 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:45:30.060749    4284 cpu_manager.go:414] "containerMap: removing container" podUID="32a7e4d5-1e58-4073-81f1-57a9b7b71999" containerName="cleanup"
Oct 31 00:45:30.060775 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:45:30.060754    4284 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="32a7e4d5-1e58-4073-81f1-57a9b7b71999" containerName="cleanup"
Oct 31 00:45:30.060870 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:45:30.060816    4284 memory_manager.go:365] "RemoveStaleState removing state" podUID="32a7e4d5-1e58-4073-81f1-57a9b7b71999" containerName="cleanup"
Oct 31 00:45:30.060870 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:45:30.060824    4284 memory_manager.go:365] "RemoveStaleState removing state" podUID="32a7e4d5-1e58-4073-81f1-57a9b7b71999" containerName="cleanup"
Oct 31 00:45:30.060870 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:45:30.060831    4284 memory_manager.go:365] "RemoveStaleState removing state" podUID="32a7e4d5-1e58-4073-81f1-57a9b7b71999" containerName="cleanup"
Oct 31 00:45:30.066515 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:45:30.066484    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-29ece5-openshift-default-6fcb86dbc-n2k5v"
Oct 31 00:45:30.128967 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:45:30.128929    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5fd0d1e4-e8d2-43a1-ac72-1e16bb0cbaef-istio-podinfo\") pod \"e2e-worker-w2-29ece5-openshift-default-6fcb86dbc-n2k5v\" (UID: \"5fd0d1e4-e8d2-43a1-ac72-1e16bb0cbaef\") " pod="openshift-ingress/e2e-worker-w2-29ece5-openshift-default-6fcb86dbc-n2k5v"
Oct 31 00:45:30.129151 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:45:30.129007    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5fd0d1e4-e8d2-43a1-ac72-1e16bb0cbaef-istiod-ca-cert\") pod \"e2e-worker-w2-29ece5-openshift-default-6fcb86dbc-n2k5v\" (UID: \"5fd0d1e4-e8d2-43a1-ac72-1e16bb0cbaef\") " pod="openshift-ingress/e2e-worker-w2-29ece5-openshift-default-6fcb86dbc-n2k5v"
Oct 31 00:45:30.129151 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:45:30.129053    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5fd0d1e4-e8d2-43a1-ac72-1e16bb0cbaef-istio-data\") pod \"e2e-worker-w2-29ece5-openshift-default-6fcb86dbc-n2k5v\" (UID: \"5fd0d1e4-e8d2-43a1-ac72-1e16bb0cbaef\") " pod="openshift-ingress/e2e-worker-w2-29ece5-openshift-default-6fcb86dbc-n2k5v"
Oct 31 00:45:30.129151 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:45:30.129073    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5fd0d1e4-e8d2-43a1-ac72-1e16bb0cbaef-credential-socket\") pod \"e2e-worker-w2-29ece5-openshift-default-6fcb86dbc-n2k5v\" (UID: \"5fd0d1e4-e8d2-43a1-ac72-1e16bb0cbaef\") " pod="openshift-ingress/e2e-worker-w2-29ece5-openshift-default-6fcb86dbc-n2k5v"
Oct 31 00:45:30.129151 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:45:30.129100    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5fd0d1e4-e8d2-43a1-ac72-1e16bb0cbaef-workload-certs\") pod \"e2e-worker-w2-29ece5-openshift-default-6fcb86dbc-n2k5v\" (UID: \"5fd0d1e4-e8d2-43a1-ac72-1e16bb0cbaef\") " pod="openshift-ingress/e2e-worker-w2-29ece5-openshift-default-6fcb86dbc-n2k5v"
Oct 31 00:45:30.129343 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:45:30.129205    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5fd0d1e4-e8d2-43a1-ac72-1e16bb0cbaef-workload-socket\") pod \"e2e-worker-w2-29ece5-openshift-default-6fcb86dbc-n2k5v\" (UID: \"5fd0d1e4-e8d2-43a1-ac72-1e16bb0cbaef\") " pod="openshift-ingress/e2e-worker-w2-29ece5-openshift-default-6fcb86dbc-n2k5v"
Oct 31 00:45:30.129343 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:45:30.129317    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5fd0d1e4-e8d2-43a1-ac72-1e16bb0cbaef-istio-token\") pod \"e2e-worker-w2-29ece5-openshift-default-6fcb86dbc-n2k5v\" (UID: \"5fd0d1e4-e8d2-43a1-ac72-1e16bb0cbaef\") " pod="openshift-ingress/e2e-worker-w2-29ece5-openshift-default-6fcb86dbc-n2k5v"
Oct 31 00:45:30.129343 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:45:30.129338    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5fd0d1e4-e8d2-43a1-ac72-1e16bb0cbaef-istio-envoy\") pod \"e2e-worker-w2-29ece5-openshift-default-6fcb86dbc-n2k5v\" (UID: \"5fd0d1e4-e8d2-43a1-ac72-1e16bb0cbaef\") " pod="openshift-ingress/e2e-worker-w2-29ece5-openshift-default-6fcb86dbc-n2k5v"
Oct 31 00:45:30.129462 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:45:30.129358    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cpr4j\" (UniqueName: \"kubernetes.io/projected/5fd0d1e4-e8d2-43a1-ac72-1e16bb0cbaef-kube-api-access-cpr4j\") pod \"e2e-worker-w2-29ece5-openshift-default-6fcb86dbc-n2k5v\" (UID: \"5fd0d1e4-e8d2-43a1-ac72-1e16bb0cbaef\") " pod="openshift-ingress/e2e-worker-w2-29ece5-openshift-default-6fcb86dbc-n2k5v"
Oct 31 00:45:30.158311 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:45:30.158280    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-29ece5-openshift-default-6fcb86dbc-n2k5v"]
Oct 31 00:45:30.230077 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:45:30.230042    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5fd0d1e4-e8d2-43a1-ac72-1e16bb0cbaef-workload-socket\") pod \"e2e-worker-w2-29ece5-openshift-default-6fcb86dbc-n2k5v\" (UID: \"5fd0d1e4-e8d2-43a1-ac72-1e16bb0cbaef\") " pod="openshift-ingress/e2e-worker-w2-29ece5-openshift-default-6fcb86dbc-n2k5v"
Oct 31 00:45:30.230281 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:45:30.230101    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5fd0d1e4-e8d2-43a1-ac72-1e16bb0cbaef-istio-token\") pod \"e2e-worker-w2-29ece5-openshift-default-6fcb86dbc-n2k5v\" (UID: \"5fd0d1e4-e8d2-43a1-ac72-1e16bb0cbaef\") " pod="openshift-ingress/e2e-worker-w2-29ece5-openshift-default-6fcb86dbc-n2k5v"
Oct 31 00:45:30.230281 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:45:30.230120    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5fd0d1e4-e8d2-43a1-ac72-1e16bb0cbaef-istio-envoy\") pod \"e2e-worker-w2-29ece5-openshift-default-6fcb86dbc-n2k5v\" (UID: \"5fd0d1e4-e8d2-43a1-ac72-1e16bb0cbaef\") " pod="openshift-ingress/e2e-worker-w2-29ece5-openshift-default-6fcb86dbc-n2k5v"
Oct 31 00:45:30.230281 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:45:30.230138    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cpr4j\" (UniqueName: \"kubernetes.io/projected/5fd0d1e4-e8d2-43a1-ac72-1e16bb0cbaef-kube-api-access-cpr4j\") pod \"e2e-worker-w2-29ece5-openshift-default-6fcb86dbc-n2k5v\" (UID: \"5fd0d1e4-e8d2-43a1-ac72-1e16bb0cbaef\") " pod="openshift-ingress/e2e-worker-w2-29ece5-openshift-default-6fcb86dbc-n2k5v"
Oct 31 00:45:30.230281 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:45:30.230198    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5fd0d1e4-e8d2-43a1-ac72-1e16bb0cbaef-istio-podinfo\") pod \"e2e-worker-w2-29ece5-openshift-default-6fcb86dbc-n2k5v\" (UID: \"5fd0d1e4-e8d2-43a1-ac72-1e16bb0cbaef\") " pod="openshift-ingress/e2e-worker-w2-29ece5-openshift-default-6fcb86dbc-n2k5v"
Oct 31 00:45:30.230281 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:45:30.230275    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5fd0d1e4-e8d2-43a1-ac72-1e16bb0cbaef-istiod-ca-cert\") pod \"e2e-worker-w2-29ece5-openshift-default-6fcb86dbc-n2k5v\" (UID: \"5fd0d1e4-e8d2-43a1-ac72-1e16bb0cbaef\") " pod="openshift-ingress/e2e-worker-w2-29ece5-openshift-default-6fcb86dbc-n2k5v"
Oct 31 00:45:30.230599 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:45:30.230324    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5fd0d1e4-e8d2-43a1-ac72-1e16bb0cbaef-istio-data\") pod \"e2e-worker-w2-29ece5-openshift-default-6fcb86dbc-n2k5v\" (UID: \"5fd0d1e4-e8d2-43a1-ac72-1e16bb0cbaef\") " pod="openshift-ingress/e2e-worker-w2-29ece5-openshift-default-6fcb86dbc-n2k5v"
Oct 31 00:45:30.230599 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:45:30.230348    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5fd0d1e4-e8d2-43a1-ac72-1e16bb0cbaef-credential-socket\") pod \"e2e-worker-w2-29ece5-openshift-default-6fcb86dbc-n2k5v\" (UID: \"5fd0d1e4-e8d2-43a1-ac72-1e16bb0cbaef\") " pod="openshift-ingress/e2e-worker-w2-29ece5-openshift-default-6fcb86dbc-n2k5v"
Oct 31 00:45:30.230599 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:45:30.230386    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5fd0d1e4-e8d2-43a1-ac72-1e16bb0cbaef-workload-certs\") pod \"e2e-worker-w2-29ece5-openshift-default-6fcb86dbc-n2k5v\" (UID: \"5fd0d1e4-e8d2-43a1-ac72-1e16bb0cbaef\") " pod="openshift-ingress/e2e-worker-w2-29ece5-openshift-default-6fcb86dbc-n2k5v"
Oct 31 00:45:30.230762 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:45:30.230724    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5fd0d1e4-e8d2-43a1-ac72-1e16bb0cbaef-workload-certs\") pod \"e2e-worker-w2-29ece5-openshift-default-6fcb86dbc-n2k5v\" (UID: \"5fd0d1e4-e8d2-43a1-ac72-1e16bb0cbaef\") " pod="openshift-ingress/e2e-worker-w2-29ece5-openshift-default-6fcb86dbc-n2k5v"
Oct 31 00:45:30.230762 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:45:30.230747    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5fd0d1e4-e8d2-43a1-ac72-1e16bb0cbaef-credential-socket\") pod \"e2e-worker-w2-29ece5-openshift-default-6fcb86dbc-n2k5v\" (UID: \"5fd0d1e4-e8d2-43a1-ac72-1e16bb0cbaef\") " pod="openshift-ingress/e2e-worker-w2-29ece5-openshift-default-6fcb86dbc-n2k5v"
Oct 31 00:45:30.230866 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:45:30.230775    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5fd0d1e4-e8d2-43a1-ac72-1e16bb0cbaef-istio-data\") pod \"e2e-worker-w2-29ece5-openshift-default-6fcb86dbc-n2k5v\" (UID: \"5fd0d1e4-e8d2-43a1-ac72-1e16bb0cbaef\") " pod="openshift-ingress/e2e-worker-w2-29ece5-openshift-default-6fcb86dbc-n2k5v"
Oct 31 00:45:30.230935 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:45:30.230918    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5fd0d1e4-e8d2-43a1-ac72-1e16bb0cbaef-workload-socket\") pod \"e2e-worker-w2-29ece5-openshift-default-6fcb86dbc-n2k5v\" (UID: \"5fd0d1e4-e8d2-43a1-ac72-1e16bb0cbaef\") " pod="openshift-ingress/e2e-worker-w2-29ece5-openshift-default-6fcb86dbc-n2k5v"
Oct 31 00:45:30.231129 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:45:30.231108    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5fd0d1e4-e8d2-43a1-ac72-1e16bb0cbaef-istiod-ca-cert\") pod \"e2e-worker-w2-29ece5-openshift-default-6fcb86dbc-n2k5v\" (UID: \"5fd0d1e4-e8d2-43a1-ac72-1e16bb0cbaef\") " pod="openshift-ingress/e2e-worker-w2-29ece5-openshift-default-6fcb86dbc-n2k5v"
Oct 31 00:45:30.233073 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:45:30.233055    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5fd0d1e4-e8d2-43a1-ac72-1e16bb0cbaef-istio-envoy\") pod \"e2e-worker-w2-29ece5-openshift-default-6fcb86dbc-n2k5v\" (UID: \"5fd0d1e4-e8d2-43a1-ac72-1e16bb0cbaef\") " pod="openshift-ingress/e2e-worker-w2-29ece5-openshift-default-6fcb86dbc-n2k5v"
Oct 31 00:45:30.233396 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:45:30.233373    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5fd0d1e4-e8d2-43a1-ac72-1e16bb0cbaef-istio-podinfo\") pod \"e2e-worker-w2-29ece5-openshift-default-6fcb86dbc-n2k5v\" (UID: \"5fd0d1e4-e8d2-43a1-ac72-1e16bb0cbaef\") " pod="openshift-ingress/e2e-worker-w2-29ece5-openshift-default-6fcb86dbc-n2k5v"
Oct 31 00:45:30.289647 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:45:30.289613    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5fd0d1e4-e8d2-43a1-ac72-1e16bb0cbaef-istio-token\") pod \"e2e-worker-w2-29ece5-openshift-default-6fcb86dbc-n2k5v\" (UID: \"5fd0d1e4-e8d2-43a1-ac72-1e16bb0cbaef\") " pod="openshift-ingress/e2e-worker-w2-29ece5-openshift-default-6fcb86dbc-n2k5v"
Oct 31 00:45:30.289897 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:45:30.289874    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cpr4j\" (UniqueName: \"kubernetes.io/projected/5fd0d1e4-e8d2-43a1-ac72-1e16bb0cbaef-kube-api-access-cpr4j\") pod \"e2e-worker-w2-29ece5-openshift-default-6fcb86dbc-n2k5v\" (UID: \"5fd0d1e4-e8d2-43a1-ac72-1e16bb0cbaef\") " pod="openshift-ingress/e2e-worker-w2-29ece5-openshift-default-6fcb86dbc-n2k5v"
Oct 31 00:45:30.382617 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:45:30.382530    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-29ece5-openshift-default-6fcb86dbc-n2k5v"
Oct 31 00:45:30.801023 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:45:30.800989    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-29ece5-openshift-default-6fcb86dbc-n2k5v"]
Oct 31 00:45:30.833843 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:45:30.833817    4284 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5fd0d1e4_e8d2_43a1_ac72_1e16bb0cbaef.slice/crio-7ad40f2794f42b49c0c46dcd930e27e4ab3b856016bbd0194f89c08483dcb83f WatchSource:0}: Error finding container 7ad40f2794f42b49c0c46dcd930e27e4ab3b856016bbd0194f89c08483dcb83f: Status 404 returned error can't find the container with id 7ad40f2794f42b49c0c46dcd930e27e4ab3b856016bbd0194f89c08483dcb83f
Oct 31 00:45:30.836133 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:45:30.836103    4284 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 00:45:30.836234 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:45:30.836169    4284 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 00:45:30.836234 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:45:30.836196    4284 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 00:45:31.023197 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:45:31.023164    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-29ece5-openshift-default-6fcb86dbc-n2k5v" event={"ID":"5fd0d1e4-e8d2-43a1-ac72-1e16bb0cbaef","Type":"ContainerStarted","Data":"b09d4f9a585ccdaac072f6e7ad63aaa1349624667b785adb080e3d5788e8dc42"}
Oct 31 00:45:31.023388 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:45:31.023234    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-29ece5-openshift-default-6fcb86dbc-n2k5v" event={"ID":"5fd0d1e4-e8d2-43a1-ac72-1e16bb0cbaef","Type":"ContainerStarted","Data":"7ad40f2794f42b49c0c46dcd930e27e4ab3b856016bbd0194f89c08483dcb83f"}
Oct 31 00:45:31.141940 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:45:31.141809    4284 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w2-29ece5-openshift-default-6fcb86dbc-n2k5v" podStartSLOduration=1.141788892 podStartE2EDuration="1.141788892s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 00:45:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:45:31.139797184 +0000 UTC m=+1045.284804103" watchObservedRunningTime="2026-10-31 00:45:31.141788892 +0000 UTC m=+1045.286795805"
Oct 31 00:45:31.382707 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:45:31.382668    4284 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-29ece5-openshift-default-6fcb86dbc-n2k5v"
Oct 31 00:45:31.384652 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:45:31.384625    4284 patch_prober.go:29] interesting pod/e2e-worker-w2-29ece5-openshift-default-6fcb86dbc-n2k5v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused" start-of-body=
Oct 31 00:45:31.385134 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:45:31.384866    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-29ece5-openshift-default-6fcb86dbc-n2k5v" podUID="5fd0d1e4-e8d2-43a1-ac72-1e16bb0cbaef" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused"
Oct 31 00:45:32.383223 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:45:32.383187    4284 patch_prober.go:29] interesting pod/e2e-worker-w2-29ece5-openshift-default-6fcb86dbc-n2k5v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused" start-of-body=
Oct 31 00:45:32.383748 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:45:32.383272    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-29ece5-openshift-default-6fcb86dbc-n2k5v" podUID="5fd0d1e4-e8d2-43a1-ac72-1e16bb0cbaef" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused"
Oct 31 00:45:32.511064 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:45:32.511020    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:45:33.383825 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:45:33.383794    4284 patch_prober.go:29] interesting pod/e2e-worker-w2-29ece5-openshift-default-6fcb86dbc-n2k5v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused" start-of-body=
Oct 31 00:45:33.384343 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:45:33.383859    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-29ece5-openshift-default-6fcb86dbc-n2k5v" podUID="5fd0d1e4-e8d2-43a1-ac72-1e16bb0cbaef" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused"
Oct 31 00:45:34.383554 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:45:34.383522    4284 patch_prober.go:29] interesting pod/e2e-worker-w2-29ece5-openshift-default-6fcb86dbc-n2k5v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused" start-of-body=
Oct 31 00:45:34.383740 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:45:34.383578    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-29ece5-openshift-default-6fcb86dbc-n2k5v" podUID="5fd0d1e4-e8d2-43a1-ac72-1e16bb0cbaef" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused"
Oct 31 00:45:35.383124 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:45:35.383088    4284 patch_prober.go:29] interesting pod/e2e-worker-w2-29ece5-openshift-default-6fcb86dbc-n2k5v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused" start-of-body=
Oct 31 00:45:35.383610 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:45:35.383148    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-29ece5-openshift-default-6fcb86dbc-n2k5v" podUID="5fd0d1e4-e8d2-43a1-ac72-1e16bb0cbaef" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused"
Oct 31 00:45:36.382822 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:45:36.382785    4284 patch_prober.go:29] interesting pod/e2e-worker-w2-29ece5-openshift-default-6fcb86dbc-n2k5v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused" start-of-body=
Oct 31 00:45:36.382998 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:45:36.382848    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-29ece5-openshift-default-6fcb86dbc-n2k5v" podUID="5fd0d1e4-e8d2-43a1-ac72-1e16bb0cbaef" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused"
Oct 31 00:45:37.383370 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:45:37.383332    4284 patch_prober.go:29] interesting pod/e2e-worker-w2-29ece5-openshift-default-6fcb86dbc-n2k5v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused" start-of-body=
Oct 31 00:45:37.383821 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:45:37.383399    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-29ece5-openshift-default-6fcb86dbc-n2k5v" podUID="5fd0d1e4-e8d2-43a1-ac72-1e16bb0cbaef" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused"
Oct 31 00:45:37.509000 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:45:37.508961    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:45:38.388195 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:45:38.388163    4284 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-29ece5-openshift-default-6fcb86dbc-n2k5v"
Oct 31 00:45:38.388651 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:45:38.388446    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-29ece5-openshift-default-6fcb86dbc-n2k5v"
Oct 31 00:45:38.389159 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:45:38.389140    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-29ece5-openshift-default-6fcb86dbc-n2k5v"
Oct 31 00:45:42.312349 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:45:42.312310    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:45:47.014327 ip-10-0-132-173 kubenswrapper[4284]: E1031 00:45:47.014284    4284 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fd8sp" containerName="extractor"
Oct 31 00:45:47.158735 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:45:47.158648    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:45:49.579609 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:45:49.579568    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:45:50.999669 ip-10-0-132-173 kubenswrapper[4284]: E1031 00:45:50.999629    4284 projected.go:266] Couldn't get secret models-as-a-service/agc-cred-1a4f67076b57: secret "agc-cred-1a4f67076b57" not found
Oct 31 00:45:50.999669 ip-10-0-132-173 kubenswrapper[4284]: E1031 00:45:50.999670    4284 projected.go:196] Error preparing data for projected volume provider-credentials for pod models-as-a-service/payload-processing-external-model-56f84cd566-gzxbm: secret "agc-cred-1a4f67076b57" not found
Oct 31 00:45:51.000311 ip-10-0-132-173 kubenswrapper[4284]: E1031 00:45:50.999765    4284 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a5b686fc-3da6-40c0-9619-86357c50fd1e-provider-credentials podName:a5b686fc-3da6-40c0-9619-86357c50fd1e nodeName:}" failed. No retries permitted until 2026-10-31 00:45:51.499746444 +0000 UTC m=+1065.644753352 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "provider-credentials" (UniqueName: "kubernetes.io/projected/a5b686fc-3da6-40c0-9619-86357c50fd1e-provider-credentials") pod "payload-processing-external-model-56f84cd566-gzxbm" (UID: "a5b686fc-3da6-40c0-9619-86357c50fd1e") : secret "agc-cred-1a4f67076b57" not found
Oct 31 00:45:51.506831 ip-10-0-132-173 kubenswrapper[4284]: E1031 00:45:51.506798    4284 projected.go:266] Couldn't get secret models-as-a-service/agc-cred-1a4f67076b57: secret "agc-cred-1a4f67076b57" not found
Oct 31 00:45:51.506831 ip-10-0-132-173 kubenswrapper[4284]: E1031 00:45:51.506828    4284 projected.go:196] Error preparing data for projected volume provider-credentials for pod models-as-a-service/payload-processing-external-model-56f84cd566-gzxbm: secret "agc-cred-1a4f67076b57" not found
Oct 31 00:45:51.507040 ip-10-0-132-173 kubenswrapper[4284]: E1031 00:45:51.506889    4284 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a5b686fc-3da6-40c0-9619-86357c50fd1e-provider-credentials podName:a5b686fc-3da6-40c0-9619-86357c50fd1e nodeName:}" failed. No retries permitted until 2026-10-31 00:45:52.506873252 +0000 UTC m=+1066.651880152 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "provider-credentials" (UniqueName: "kubernetes.io/projected/a5b686fc-3da6-40c0-9619-86357c50fd1e-provider-credentials") pod "payload-processing-external-model-56f84cd566-gzxbm" (UID: "a5b686fc-3da6-40c0-9619-86357c50fd1e") : secret "agc-cred-1a4f67076b57" not found
Oct 31 00:45:51.893265 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:45:51.893148    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:45:52.517304 ip-10-0-132-173 kubenswrapper[4284]: E1031 00:45:52.517272    4284 projected.go:266] Couldn't get secret models-as-a-service/agc-cred-1a4f67076b57: secret "agc-cred-1a4f67076b57" not found
Oct 31 00:45:52.517304 ip-10-0-132-173 kubenswrapper[4284]: E1031 00:45:52.517304    4284 projected.go:196] Error preparing data for projected volume provider-credentials for pod models-as-a-service/payload-processing-external-model-56f84cd566-gzxbm: secret "agc-cred-1a4f67076b57" not found
Oct 31 00:45:52.517756 ip-10-0-132-173 kubenswrapper[4284]: E1031 00:45:52.517366    4284 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a5b686fc-3da6-40c0-9619-86357c50fd1e-provider-credentials podName:a5b686fc-3da6-40c0-9619-86357c50fd1e nodeName:}" failed. No retries permitted until 2026-10-31 00:45:54.517347053 +0000 UTC m=+1068.662353950 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "provider-credentials" (UniqueName: "kubernetes.io/projected/a5b686fc-3da6-40c0-9619-86357c50fd1e-provider-credentials") pod "payload-processing-external-model-56f84cd566-gzxbm" (UID: "a5b686fc-3da6-40c0-9619-86357c50fd1e") : secret "agc-cred-1a4f67076b57" not found
Oct 31 00:45:54.559065 ip-10-0-132-173 kubenswrapper[4284]: E1031 00:45:54.559032    4284 projected.go:266] Couldn't get secret models-as-a-service/agc-cred-bf4919d00529: secret "agc-cred-bf4919d00529" not found
Oct 31 00:45:54.559065 ip-10-0-132-173 kubenswrapper[4284]: E1031 00:45:54.559059    4284 projected.go:196] Error preparing data for projected volume provider-credentials for pod models-as-a-service/payload-processing-external-model-56f84cd566-gzxbm: secret "agc-cred-bf4919d00529" not found
Oct 31 00:45:54.559522 ip-10-0-132-173 kubenswrapper[4284]: E1031 00:45:54.559113    4284 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a5b686fc-3da6-40c0-9619-86357c50fd1e-provider-credentials podName:a5b686fc-3da6-40c0-9619-86357c50fd1e nodeName:}" failed. No retries permitted until 2026-10-31 00:45:58.559096971 +0000 UTC m=+1072.704103883 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "provider-credentials" (UniqueName: "kubernetes.io/projected/a5b686fc-3da6-40c0-9619-86357c50fd1e-provider-credentials") pod "payload-processing-external-model-56f84cd566-gzxbm" (UID: "a5b686fc-3da6-40c0-9619-86357c50fd1e") : secret "agc-cred-bf4919d00529" not found
Oct 31 00:45:54.675726 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:45:54.675685    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:45:56.698141 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:45:56.698097    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:45:57.255221 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:45:57.255184    4284 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["models-as-a-service/payload-processing-external-model-5747777fbb-w572x"]
Oct 31 00:45:57.255776 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:45:57.255759    4284 cpu_manager.go:414] "containerMap: removing container" podUID="32a7e4d5-1e58-4073-81f1-57a9b7b71999" containerName="cleanup"
Oct 31 00:45:57.255829 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:45:57.255779    4284 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="32a7e4d5-1e58-4073-81f1-57a9b7b71999" containerName="cleanup"
Oct 31 00:45:57.259109 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:45:57.259089    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="models-as-a-service/payload-processing-external-model-5747777fbb-w572x"
Oct 31 00:45:57.309417 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:45:57.309387    4284 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"models-as-a-service\"/\"payload-processing-external-model-dockercfg-zh9bc\""
Oct 31 00:45:57.361163 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:45:57.361123    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["models-as-a-service/payload-processing-external-model-5747777fbb-w572x"]
Oct 31 00:45:57.395018 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:45:57.394986    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/863ee1ca-55df-442d-b754-124eb102c5b6-plugins-config-volume\") pod \"payload-processing-external-model-5747777fbb-w572x\" (UID: \"863ee1ca-55df-442d-b754-124eb102c5b6\") " pod="models-as-a-service/payload-processing-external-model-5747777fbb-w572x"
Oct 31 00:45:57.395246 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:45:57.395097    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/863ee1ca-55df-442d-b754-124eb102c5b6-routing-overlay-volume\") pod \"payload-processing-external-model-5747777fbb-w572x\" (UID: \"863ee1ca-55df-442d-b754-124eb102c5b6\") " pod="models-as-a-service/payload-processing-external-model-5747777fbb-w572x"
Oct 31 00:45:57.395246 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:45:57.395145    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bxrnd\" (UniqueName: \"kubernetes.io/projected/863ee1ca-55df-442d-b754-124eb102c5b6-kube-api-access-bxrnd\") pod \"payload-processing-external-model-5747777fbb-w572x\" (UID: \"863ee1ca-55df-442d-b754-124eb102c5b6\") " pod="models-as-a-service/payload-processing-external-model-5747777fbb-w572x"
Oct 31 00:45:57.395383 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:45:57.395357    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/863ee1ca-55df-442d-b754-124eb102c5b6-provider-credentials\") pod \"payload-processing-external-model-5747777fbb-w572x\" (UID: \"863ee1ca-55df-442d-b754-124eb102c5b6\") " pod="models-as-a-service/payload-processing-external-model-5747777fbb-w572x"
Oct 31 00:45:57.496368 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:45:57.496326    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bxrnd\" (UniqueName: \"kubernetes.io/projected/863ee1ca-55df-442d-b754-124eb102c5b6-kube-api-access-bxrnd\") pod \"payload-processing-external-model-5747777fbb-w572x\" (UID: \"863ee1ca-55df-442d-b754-124eb102c5b6\") " pod="models-as-a-service/payload-processing-external-model-5747777fbb-w572x"
Oct 31 00:45:57.496568 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:45:57.496448    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/863ee1ca-55df-442d-b754-124eb102c5b6-provider-credentials\") pod \"payload-processing-external-model-5747777fbb-w572x\" (UID: \"863ee1ca-55df-442d-b754-124eb102c5b6\") " pod="models-as-a-service/payload-processing-external-model-5747777fbb-w572x"
Oct 31 00:45:57.496568 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:45:57.496508    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/863ee1ca-55df-442d-b754-124eb102c5b6-plugins-config-volume\") pod \"payload-processing-external-model-5747777fbb-w572x\" (UID: \"863ee1ca-55df-442d-b754-124eb102c5b6\") " pod="models-as-a-service/payload-processing-external-model-5747777fbb-w572x"
Oct 31 00:45:57.496694 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:45:57.496575    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/863ee1ca-55df-442d-b754-124eb102c5b6-routing-overlay-volume\") pod \"payload-processing-external-model-5747777fbb-w572x\" (UID: \"863ee1ca-55df-442d-b754-124eb102c5b6\") " pod="models-as-a-service/payload-processing-external-model-5747777fbb-w572x"
Oct 31 00:45:57.497440 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:45:57.497343    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/863ee1ca-55df-442d-b754-124eb102c5b6-routing-overlay-volume\") pod \"payload-processing-external-model-5747777fbb-w572x\" (UID: \"863ee1ca-55df-442d-b754-124eb102c5b6\") " pod="models-as-a-service/payload-processing-external-model-5747777fbb-w572x"
Oct 31 00:45:57.497440 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:45:57.497379    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/863ee1ca-55df-442d-b754-124eb102c5b6-plugins-config-volume\") pod \"payload-processing-external-model-5747777fbb-w572x\" (UID: \"863ee1ca-55df-442d-b754-124eb102c5b6\") " pod="models-as-a-service/payload-processing-external-model-5747777fbb-w572x"
Oct 31 00:45:57.499282 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:45:57.499259    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/863ee1ca-55df-442d-b754-124eb102c5b6-provider-credentials\") pod \"payload-processing-external-model-5747777fbb-w572x\" (UID: \"863ee1ca-55df-442d-b754-124eb102c5b6\") " pod="models-as-a-service/payload-processing-external-model-5747777fbb-w572x"
Oct 31 00:45:57.555330 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:45:57.555261    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bxrnd\" (UniqueName: \"kubernetes.io/projected/863ee1ca-55df-442d-b754-124eb102c5b6-kube-api-access-bxrnd\") pod \"payload-processing-external-model-5747777fbb-w572x\" (UID: \"863ee1ca-55df-442d-b754-124eb102c5b6\") " pod="models-as-a-service/payload-processing-external-model-5747777fbb-w572x"
Oct 31 00:45:57.575139 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:45:57.575112    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="models-as-a-service/payload-processing-external-model-5747777fbb-w572x"
Oct 31 00:45:57.946035 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:45:57.946002    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["models-as-a-service/payload-processing-external-model-5747777fbb-w572x"]
Oct 31 00:45:57.947017 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:45:57.946985    4284 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod863ee1ca_55df_442d_b754_124eb102c5b6.slice/crio-b95923d75b631f49f188c57971a48b77e1779b244a171687684537156e7fe277 WatchSource:0}: Error finding container b95923d75b631f49f188c57971a48b77e1779b244a171687684537156e7fe277: Status 404 returned error can't find the container with id b95923d75b631f49f188c57971a48b77e1779b244a171687684537156e7fe277
Oct 31 00:45:58.203678 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:45:58.203594    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="models-as-a-service/payload-processing-external-model-5747777fbb-w572x" event={"ID":"863ee1ca-55df-442d-b754-124eb102c5b6","Type":"ContainerStarted","Data":"9cb4b2c884af337af9d47ab3ed0974eeb51c9e23803acde5ea5d90e0e580eccb"}
Oct 31 00:45:58.203678 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:45:58.203646    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="models-as-a-service/payload-processing-external-model-5747777fbb-w572x" event={"ID":"863ee1ca-55df-442d-b754-124eb102c5b6","Type":"ContainerStarted","Data":"b95923d75b631f49f188c57971a48b77e1779b244a171687684537156e7fe277"}
Oct 31 00:45:58.203873 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:45:58.203711    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="models-as-a-service/payload-processing-external-model-5747777fbb-w572x"
Oct 31 00:45:58.317778 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:45:58.317730    4284 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="models-as-a-service/payload-processing-external-model-5747777fbb-w572x" podStartSLOduration=1.317716045 podStartE2EDuration="1.317716045s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 00:45:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:45:58.314993916 +0000 UTC m=+1072.460000832" watchObservedRunningTime="2026-10-31 00:45:58.317716045 +0000 UTC m=+1072.462722989"
Oct 31 00:45:58.611686 ip-10-0-132-173 kubenswrapper[4284]: E1031 00:45:58.611609    4284 projected.go:266] Couldn't get secret models-as-a-service/agc-cred-bf4919d00529: secret "agc-cred-bf4919d00529" not found
Oct 31 00:45:58.611686 ip-10-0-132-173 kubenswrapper[4284]: E1031 00:45:58.611635    4284 projected.go:196] Error preparing data for projected volume provider-credentials for pod models-as-a-service/payload-processing-external-model-56f84cd566-gzxbm: secret "agc-cred-bf4919d00529" not found
Oct 31 00:45:58.611871 ip-10-0-132-173 kubenswrapper[4284]: E1031 00:45:58.611696    4284 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a5b686fc-3da6-40c0-9619-86357c50fd1e-provider-credentials podName:a5b686fc-3da6-40c0-9619-86357c50fd1e nodeName:}" failed. No retries permitted until 2026-10-31 00:46:06.611680299 +0000 UTC m=+1080.756687197 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "provider-credentials" (UniqueName: "kubernetes.io/projected/a5b686fc-3da6-40c0-9619-86357c50fd1e-provider-credentials") pod "payload-processing-external-model-56f84cd566-gzxbm" (UID: "a5b686fc-3da6-40c0-9619-86357c50fd1e") : secret "agc-cred-bf4919d00529" not found
Oct 31 00:46:01.558096 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:46:01.558052    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:46:04.618241 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:46:04.617585    4284 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6cc458d887-rr5k6 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.39:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 31 00:46:04.618241 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:46:04.617664    4284 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-rr5k6" podUID="27f8cf21-0f47-4397-831a-bf716741dc67" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.39:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:46:05.699573 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:46:05.699535    4284 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-k89gx"]
Oct 31 00:46:05.704729 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:46:05.704705    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-k89gx"
Oct 31 00:46:05.806499 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:46:05.806469    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-k89gx"]
Oct 31 00:46:05.841819 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:46:05.841784    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/41a8f8d6-dbbb-48db-a957-5a3bfe8aa418-istio-podinfo\") pod \"e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-k89gx\" (UID: \"41a8f8d6-dbbb-48db-a957-5a3bfe8aa418\") " pod="openshift-ingress/e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-k89gx"
Oct 31 00:46:05.841993 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:46:05.841903    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/41a8f8d6-dbbb-48db-a957-5a3bfe8aa418-istio-token\") pod \"e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-k89gx\" (UID: \"41a8f8d6-dbbb-48db-a957-5a3bfe8aa418\") " pod="openshift-ingress/e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-k89gx"
Oct 31 00:46:05.841993 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:46:05.841960    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/41a8f8d6-dbbb-48db-a957-5a3bfe8aa418-workload-certs\") pod \"e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-k89gx\" (UID: \"41a8f8d6-dbbb-48db-a957-5a3bfe8aa418\") " pod="openshift-ingress/e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-k89gx"
Oct 31 00:46:05.842098 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:46:05.841993    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/41a8f8d6-dbbb-48db-a957-5a3bfe8aa418-istio-data\") pod \"e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-k89gx\" (UID: \"41a8f8d6-dbbb-48db-a957-5a3bfe8aa418\") " pod="openshift-ingress/e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-k89gx"
Oct 31 00:46:05.842146 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:46:05.842114    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/41a8f8d6-dbbb-48db-a957-5a3bfe8aa418-istiod-ca-cert\") pod \"e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-k89gx\" (UID: \"41a8f8d6-dbbb-48db-a957-5a3bfe8aa418\") " pod="openshift-ingress/e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-k89gx"
Oct 31 00:46:05.842146 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:46:05.842137    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/41a8f8d6-dbbb-48db-a957-5a3bfe8aa418-istio-envoy\") pod \"e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-k89gx\" (UID: \"41a8f8d6-dbbb-48db-a957-5a3bfe8aa418\") " pod="openshift-ingress/e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-k89gx"
Oct 31 00:46:05.842247 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:46:05.842158    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/41a8f8d6-dbbb-48db-a957-5a3bfe8aa418-workload-socket\") pod \"e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-k89gx\" (UID: \"41a8f8d6-dbbb-48db-a957-5a3bfe8aa418\") " pod="openshift-ingress/e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-k89gx"
Oct 31 00:46:05.842247 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:46:05.842182    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/41a8f8d6-dbbb-48db-a957-5a3bfe8aa418-credential-socket\") pod \"e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-k89gx\" (UID: \"41a8f8d6-dbbb-48db-a957-5a3bfe8aa418\") " pod="openshift-ingress/e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-k89gx"
Oct 31 00:46:05.842311 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:46:05.842253    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mh69w\" (UniqueName: \"kubernetes.io/projected/41a8f8d6-dbbb-48db-a957-5a3bfe8aa418-kube-api-access-mh69w\") pod \"e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-k89gx\" (UID: \"41a8f8d6-dbbb-48db-a957-5a3bfe8aa418\") " pod="openshift-ingress/e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-k89gx"
Oct 31 00:46:05.942789 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:46:05.942753    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/41a8f8d6-dbbb-48db-a957-5a3bfe8aa418-istiod-ca-cert\") pod \"e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-k89gx\" (UID: \"41a8f8d6-dbbb-48db-a957-5a3bfe8aa418\") " pod="openshift-ingress/e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-k89gx"
Oct 31 00:46:05.942789 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:46:05.942794    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/41a8f8d6-dbbb-48db-a957-5a3bfe8aa418-istio-envoy\") pod \"e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-k89gx\" (UID: \"41a8f8d6-dbbb-48db-a957-5a3bfe8aa418\") " pod="openshift-ingress/e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-k89gx"
Oct 31 00:46:05.943031 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:46:05.942824    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/41a8f8d6-dbbb-48db-a957-5a3bfe8aa418-workload-socket\") pod \"e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-k89gx\" (UID: \"41a8f8d6-dbbb-48db-a957-5a3bfe8aa418\") " pod="openshift-ingress/e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-k89gx"
Oct 31 00:46:05.943031 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:46:05.942855    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/41a8f8d6-dbbb-48db-a957-5a3bfe8aa418-credential-socket\") pod \"e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-k89gx\" (UID: \"41a8f8d6-dbbb-48db-a957-5a3bfe8aa418\") " pod="openshift-ingress/e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-k89gx"
Oct 31 00:46:05.943031 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:46:05.942887    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mh69w\" (UniqueName: \"kubernetes.io/projected/41a8f8d6-dbbb-48db-a957-5a3bfe8aa418-kube-api-access-mh69w\") pod \"e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-k89gx\" (UID: \"41a8f8d6-dbbb-48db-a957-5a3bfe8aa418\") " pod="openshift-ingress/e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-k89gx"
Oct 31 00:46:05.943031 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:46:05.942983    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/41a8f8d6-dbbb-48db-a957-5a3bfe8aa418-istio-podinfo\") pod \"e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-k89gx\" (UID: \"41a8f8d6-dbbb-48db-a957-5a3bfe8aa418\") " pod="openshift-ingress/e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-k89gx"
Oct 31 00:46:05.943244 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:46:05.943063    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/41a8f8d6-dbbb-48db-a957-5a3bfe8aa418-istio-token\") pod \"e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-k89gx\" (UID: \"41a8f8d6-dbbb-48db-a957-5a3bfe8aa418\") " pod="openshift-ingress/e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-k89gx"
Oct 31 00:46:05.943244 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:46:05.943112    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/41a8f8d6-dbbb-48db-a957-5a3bfe8aa418-workload-certs\") pod \"e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-k89gx\" (UID: \"41a8f8d6-dbbb-48db-a957-5a3bfe8aa418\") " pod="openshift-ingress/e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-k89gx"
Oct 31 00:46:05.943244 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:46:05.943158    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/41a8f8d6-dbbb-48db-a957-5a3bfe8aa418-istio-data\") pod \"e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-k89gx\" (UID: \"41a8f8d6-dbbb-48db-a957-5a3bfe8aa418\") " pod="openshift-ingress/e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-k89gx"
Oct 31 00:46:05.943423 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:46:05.943381    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/41a8f8d6-dbbb-48db-a957-5a3bfe8aa418-credential-socket\") pod \"e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-k89gx\" (UID: \"41a8f8d6-dbbb-48db-a957-5a3bfe8aa418\") " pod="openshift-ingress/e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-k89gx"
Oct 31 00:46:05.943556 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:46:05.943422    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/41a8f8d6-dbbb-48db-a957-5a3bfe8aa418-workload-socket\") pod \"e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-k89gx\" (UID: \"41a8f8d6-dbbb-48db-a957-5a3bfe8aa418\") " pod="openshift-ingress/e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-k89gx"
Oct 31 00:46:05.943556 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:46:05.943523    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/41a8f8d6-dbbb-48db-a957-5a3bfe8aa418-istio-data\") pod \"e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-k89gx\" (UID: \"41a8f8d6-dbbb-48db-a957-5a3bfe8aa418\") " pod="openshift-ingress/e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-k89gx"
Oct 31 00:46:05.943668 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:46:05.943602    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/41a8f8d6-dbbb-48db-a957-5a3bfe8aa418-workload-certs\") pod \"e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-k89gx\" (UID: \"41a8f8d6-dbbb-48db-a957-5a3bfe8aa418\") " pod="openshift-ingress/e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-k89gx"
Oct 31 00:46:05.943668 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:46:05.943605    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/41a8f8d6-dbbb-48db-a957-5a3bfe8aa418-istiod-ca-cert\") pod \"e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-k89gx\" (UID: \"41a8f8d6-dbbb-48db-a957-5a3bfe8aa418\") " pod="openshift-ingress/e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-k89gx"
Oct 31 00:46:05.945534 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:46:05.945515    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/41a8f8d6-dbbb-48db-a957-5a3bfe8aa418-istio-envoy\") pod \"e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-k89gx\" (UID: \"41a8f8d6-dbbb-48db-a957-5a3bfe8aa418\") " pod="openshift-ingress/e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-k89gx"
Oct 31 00:46:05.945738 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:46:05.945720    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/41a8f8d6-dbbb-48db-a957-5a3bfe8aa418-istio-podinfo\") pod \"e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-k89gx\" (UID: \"41a8f8d6-dbbb-48db-a957-5a3bfe8aa418\") " pod="openshift-ingress/e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-k89gx"
Oct 31 00:46:06.001523 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:46:06.001449    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/41a8f8d6-dbbb-48db-a957-5a3bfe8aa418-istio-token\") pod \"e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-k89gx\" (UID: \"41a8f8d6-dbbb-48db-a957-5a3bfe8aa418\") " pod="openshift-ingress/e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-k89gx"
Oct 31 00:46:06.001645 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:46:06.001582    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mh69w\" (UniqueName: \"kubernetes.io/projected/41a8f8d6-dbbb-48db-a957-5a3bfe8aa418-kube-api-access-mh69w\") pod \"e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-k89gx\" (UID: \"41a8f8d6-dbbb-48db-a957-5a3bfe8aa418\") " pod="openshift-ingress/e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-k89gx"
Oct 31 00:46:06.018824 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:46:06.018799    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-k89gx"
Oct 31 00:46:06.442468 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:46:06.442443    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-k89gx"]
Oct 31 00:46:06.443415 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:46:06.443391    4284 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod41a8f8d6_dbbb_48db_a957_5a3bfe8aa418.slice/crio-f83e5d6047d314c4b0dbaba0af53c69e761d0e8b62c1558cc48a098a4910813b WatchSource:0}: Error finding container f83e5d6047d314c4b0dbaba0af53c69e761d0e8b62c1558cc48a098a4910813b: Status 404 returned error can't find the container with id f83e5d6047d314c4b0dbaba0af53c69e761d0e8b62c1558cc48a098a4910813b
Oct 31 00:46:06.445797 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:46:06.445764    4284 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 00:46:06.445878 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:46:06.445828    4284 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 00:46:06.445878 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:46:06.445865    4284 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 00:46:06.652447 ip-10-0-132-173 kubenswrapper[4284]: E1031 00:46:06.652408    4284 projected.go:266] Couldn't get secret models-as-a-service/agc-cred-bf4919d00529: secret "agc-cred-bf4919d00529" not found
Oct 31 00:46:06.652447 ip-10-0-132-173 kubenswrapper[4284]: E1031 00:46:06.652432    4284 projected.go:196] Error preparing data for projected volume provider-credentials for pod models-as-a-service/payload-processing-external-model-56f84cd566-gzxbm: secret "agc-cred-bf4919d00529" not found
Oct 31 00:46:06.652701 ip-10-0-132-173 kubenswrapper[4284]: E1031 00:46:06.652493    4284 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a5b686fc-3da6-40c0-9619-86357c50fd1e-provider-credentials podName:a5b686fc-3da6-40c0-9619-86357c50fd1e nodeName:}" failed. No retries permitted until 2026-10-31 00:46:22.652478012 +0000 UTC m=+1096.797484906 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "provider-credentials" (UniqueName: "kubernetes.io/projected/a5b686fc-3da6-40c0-9619-86357c50fd1e-provider-credentials") pod "payload-processing-external-model-56f84cd566-gzxbm" (UID: "a5b686fc-3da6-40c0-9619-86357c50fd1e") : secret "agc-cred-bf4919d00529" not found
Oct 31 00:46:06.726119 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:46:06.726038    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:46:07.282161 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:46:07.282128    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-k89gx" event={"ID":"41a8f8d6-dbbb-48db-a957-5a3bfe8aa418","Type":"ContainerStarted","Data":"06ab3e269d09bcaeed61b677bdc73a2b6b116cd3b10be0c1c7721ae11d7127d0"}
Oct 31 00:46:07.282354 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:46:07.282180    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-k89gx" event={"ID":"41a8f8d6-dbbb-48db-a957-5a3bfe8aa418","Type":"ContainerStarted","Data":"f83e5d6047d314c4b0dbaba0af53c69e761d0e8b62c1558cc48a098a4910813b"}
Oct 31 00:46:07.403690 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:46:07.403627    4284 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-k89gx" podStartSLOduration=2.403608203 podStartE2EDuration="2.403608203s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 00:46:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:46:07.401926558 +0000 UTC m=+1081.546933503" watchObservedRunningTime="2026-10-31 00:46:07.403608203 +0000 UTC m=+1081.548615120"
Oct 31 00:46:08.019014 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:46:08.018981    4284 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-k89gx"
Oct 31 00:46:08.020877 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:46:08.020842    4284 patch_prober.go:29] interesting pod/e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-k89gx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused" start-of-body=
Oct 31 00:46:08.021000 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:46:08.020909    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-k89gx" podUID="41a8f8d6-dbbb-48db-a957-5a3bfe8aa418" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused"
Oct 31 00:46:09.019343 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:46:09.019282    4284 patch_prober.go:29] interesting pod/e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-k89gx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused" start-of-body=
Oct 31 00:46:09.019745 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:46:09.019361    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-k89gx" podUID="41a8f8d6-dbbb-48db-a957-5a3bfe8aa418" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused"
Oct 31 00:46:09.212706 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:46:09.212657    4284 prober.go:122] "Probe failed" probeType="Readiness" pod="models-as-a-service/payload-processing-external-model-5747777fbb-w572x" podUID="863ee1ca-55df-442d-b754-124eb102c5b6" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 31 00:46:10.019956 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:46:10.019916    4284 patch_prober.go:29] interesting pod/e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-k89gx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused" start-of-body=
Oct 31 00:46:10.020440 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:46:10.019997    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-k89gx" podUID="41a8f8d6-dbbb-48db-a957-5a3bfe8aa418" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused"
Oct 31 00:46:11.019756 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:46:11.019723    4284 patch_prober.go:29] interesting pod/e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-k89gx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused" start-of-body=
Oct 31 00:46:11.019933 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:46:11.019782    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-k89gx" podUID="41a8f8d6-dbbb-48db-a957-5a3bfe8aa418" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused"
Oct 31 00:46:11.535190 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:46:11.535153    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:46:12.019737 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:46:12.019705    4284 patch_prober.go:29] interesting pod/e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-k89gx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused" start-of-body=
Oct 31 00:46:12.019919 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:46:12.019764    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-k89gx" podUID="41a8f8d6-dbbb-48db-a957-5a3bfe8aa418" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused"
Oct 31 00:46:13.019786 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:46:13.019747    4284 patch_prober.go:29] interesting pod/e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-k89gx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused" start-of-body=
Oct 31 00:46:13.020203 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:46:13.019803    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-k89gx" podUID="41a8f8d6-dbbb-48db-a957-5a3bfe8aa418" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused"
Oct 31 00:46:14.024856 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:46:14.024827    4284 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-k89gx"
Oct 31 00:46:14.025451 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:46:14.025322    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-k89gx"
Oct 31 00:46:14.026130 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:46:14.026110    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-k89gx"
Oct 31 00:46:16.494446 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:46:16.494410    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:46:17.013985 ip-10-0-132-173 kubenswrapper[4284]: E1031 00:46:17.013953    4284 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fd8sp" containerName="extractor"
Oct 31 00:46:19.210160 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:46:19.210123    4284 prober.go:122] "Probe failed" probeType="Readiness" pod="models-as-a-service/payload-processing-external-model-5747777fbb-w572x" podUID="863ee1ca-55df-442d-b754-124eb102c5b6" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 31 00:46:21.293638 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:46:21.293598    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:46:22.667877 ip-10-0-132-173 kubenswrapper[4284]: E1031 00:46:22.667842    4284 projected.go:266] Couldn't get secret models-as-a-service/agc-cred-bf4919d00529: secret "agc-cred-bf4919d00529" not found
Oct 31 00:46:22.667877 ip-10-0-132-173 kubenswrapper[4284]: E1031 00:46:22.667872    4284 projected.go:196] Error preparing data for projected volume provider-credentials for pod models-as-a-service/payload-processing-external-model-56f84cd566-gzxbm: secret "agc-cred-bf4919d00529" not found
Oct 31 00:46:22.668349 ip-10-0-132-173 kubenswrapper[4284]: E1031 00:46:22.667938    4284 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a5b686fc-3da6-40c0-9619-86357c50fd1e-provider-credentials podName:a5b686fc-3da6-40c0-9619-86357c50fd1e nodeName:}" failed. No retries permitted until 2026-10-31 00:46:54.667922315 +0000 UTC m=+1128.812929216 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "provider-credentials" (UniqueName: "kubernetes.io/projected/a5b686fc-3da6-40c0-9619-86357c50fd1e-provider-credentials") pod "payload-processing-external-model-56f84cd566-gzxbm" (UID: "a5b686fc-3da6-40c0-9619-86357c50fd1e") : secret "agc-cred-bf4919d00529" not found
Oct 31 00:46:24.921743 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:46:24.921711    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:46:26.119813 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:46:26.119778    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:46:29.210636 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:46:29.210600    4284 prober.go:122] "Probe failed" probeType="Readiness" pod="models-as-a-service/payload-processing-external-model-5747777fbb-w572x" podUID="863ee1ca-55df-442d-b754-124eb102c5b6" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 31 00:46:29.948771 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:46:29.948727    4284 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w2-29ece5-openshift-default-6fcb86dbc-f75sz"]
Oct 31 00:46:29.952670 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:46:29.952648    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-29ece5-openshift-default-6fcb86dbc-f75sz"
Oct 31 00:46:30.043601 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:46:30.043562    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-29ece5-openshift-default-6fcb86dbc-f75sz"]
Oct 31 00:46:30.054072 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:46:30.054034    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ffd0eb46-1e6b-4bcc-95bc-eaca5254c056-workload-certs\") pod \"e2e-worker-w2-29ece5-openshift-default-6fcb86dbc-f75sz\" (UID: \"ffd0eb46-1e6b-4bcc-95bc-eaca5254c056\") " pod="openshift-ingress/e2e-worker-w2-29ece5-openshift-default-6fcb86dbc-f75sz"
Oct 31 00:46:30.054339 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:46:30.054084    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ffd0eb46-1e6b-4bcc-95bc-eaca5254c056-credential-socket\") pod \"e2e-worker-w2-29ece5-openshift-default-6fcb86dbc-f75sz\" (UID: \"ffd0eb46-1e6b-4bcc-95bc-eaca5254c056\") " pod="openshift-ingress/e2e-worker-w2-29ece5-openshift-default-6fcb86dbc-f75sz"
Oct 31 00:46:30.054339 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:46:30.054115    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ffd0eb46-1e6b-4bcc-95bc-eaca5254c056-istio-data\") pod \"e2e-worker-w2-29ece5-openshift-default-6fcb86dbc-f75sz\" (UID: \"ffd0eb46-1e6b-4bcc-95bc-eaca5254c056\") " pod="openshift-ingress/e2e-worker-w2-29ece5-openshift-default-6fcb86dbc-f75sz"
Oct 31 00:46:30.054339 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:46:30.054140    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ffd0eb46-1e6b-4bcc-95bc-eaca5254c056-istio-podinfo\") pod \"e2e-worker-w2-29ece5-openshift-default-6fcb86dbc-f75sz\" (UID: \"ffd0eb46-1e6b-4bcc-95bc-eaca5254c056\") " pod="openshift-ingress/e2e-worker-w2-29ece5-openshift-default-6fcb86dbc-f75sz"
Oct 31 00:46:30.054339 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:46:30.054207    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ffd0eb46-1e6b-4bcc-95bc-eaca5254c056-workload-socket\") pod \"e2e-worker-w2-29ece5-openshift-default-6fcb86dbc-f75sz\" (UID: \"ffd0eb46-1e6b-4bcc-95bc-eaca5254c056\") " pod="openshift-ingress/e2e-worker-w2-29ece5-openshift-default-6fcb86dbc-f75sz"
Oct 31 00:46:30.054525 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:46:30.054350    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vcml2\" (UniqueName: \"kubernetes.io/projected/ffd0eb46-1e6b-4bcc-95bc-eaca5254c056-kube-api-access-vcml2\") pod \"e2e-worker-w2-29ece5-openshift-default-6fcb86dbc-f75sz\" (UID: \"ffd0eb46-1e6b-4bcc-95bc-eaca5254c056\") " pod="openshift-ingress/e2e-worker-w2-29ece5-openshift-default-6fcb86dbc-f75sz"
Oct 31 00:46:30.054525 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:46:30.054396    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ffd0eb46-1e6b-4bcc-95bc-eaca5254c056-istio-envoy\") pod \"e2e-worker-w2-29ece5-openshift-default-6fcb86dbc-f75sz\" (UID: \"ffd0eb46-1e6b-4bcc-95bc-eaca5254c056\") " pod="openshift-ingress/e2e-worker-w2-29ece5-openshift-default-6fcb86dbc-f75sz"
Oct 31 00:46:30.054525 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:46:30.054478    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ffd0eb46-1e6b-4bcc-95bc-eaca5254c056-istio-token\") pod \"e2e-worker-w2-29ece5-openshift-default-6fcb86dbc-f75sz\" (UID: \"ffd0eb46-1e6b-4bcc-95bc-eaca5254c056\") " pod="openshift-ingress/e2e-worker-w2-29ece5-openshift-default-6fcb86dbc-f75sz"
Oct 31 00:46:30.054525 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:46:30.054504    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ffd0eb46-1e6b-4bcc-95bc-eaca5254c056-istiod-ca-cert\") pod \"e2e-worker-w2-29ece5-openshift-default-6fcb86dbc-f75sz\" (UID: \"ffd0eb46-1e6b-4bcc-95bc-eaca5254c056\") " pod="openshift-ingress/e2e-worker-w2-29ece5-openshift-default-6fcb86dbc-f75sz"
Oct 31 00:46:30.155394 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:46:30.155362    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ffd0eb46-1e6b-4bcc-95bc-eaca5254c056-workload-certs\") pod \"e2e-worker-w2-29ece5-openshift-default-6fcb86dbc-f75sz\" (UID: \"ffd0eb46-1e6b-4bcc-95bc-eaca5254c056\") " pod="openshift-ingress/e2e-worker-w2-29ece5-openshift-default-6fcb86dbc-f75sz"
Oct 31 00:46:30.155394 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:46:30.155399    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ffd0eb46-1e6b-4bcc-95bc-eaca5254c056-credential-socket\") pod \"e2e-worker-w2-29ece5-openshift-default-6fcb86dbc-f75sz\" (UID: \"ffd0eb46-1e6b-4bcc-95bc-eaca5254c056\") " pod="openshift-ingress/e2e-worker-w2-29ece5-openshift-default-6fcb86dbc-f75sz"
Oct 31 00:46:30.155679 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:46:30.155417    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ffd0eb46-1e6b-4bcc-95bc-eaca5254c056-istio-data\") pod \"e2e-worker-w2-29ece5-openshift-default-6fcb86dbc-f75sz\" (UID: \"ffd0eb46-1e6b-4bcc-95bc-eaca5254c056\") " pod="openshift-ingress/e2e-worker-w2-29ece5-openshift-default-6fcb86dbc-f75sz"
Oct 31 00:46:30.155679 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:46:30.155434    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ffd0eb46-1e6b-4bcc-95bc-eaca5254c056-istio-podinfo\") pod \"e2e-worker-w2-29ece5-openshift-default-6fcb86dbc-f75sz\" (UID: \"ffd0eb46-1e6b-4bcc-95bc-eaca5254c056\") " pod="openshift-ingress/e2e-worker-w2-29ece5-openshift-default-6fcb86dbc-f75sz"
Oct 31 00:46:30.155679 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:46:30.155483    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ffd0eb46-1e6b-4bcc-95bc-eaca5254c056-workload-socket\") pod \"e2e-worker-w2-29ece5-openshift-default-6fcb86dbc-f75sz\" (UID: \"ffd0eb46-1e6b-4bcc-95bc-eaca5254c056\") " pod="openshift-ingress/e2e-worker-w2-29ece5-openshift-default-6fcb86dbc-f75sz"
Oct 31 00:46:30.155679 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:46:30.155515    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vcml2\" (UniqueName: \"kubernetes.io/projected/ffd0eb46-1e6b-4bcc-95bc-eaca5254c056-kube-api-access-vcml2\") pod \"e2e-worker-w2-29ece5-openshift-default-6fcb86dbc-f75sz\" (UID: \"ffd0eb46-1e6b-4bcc-95bc-eaca5254c056\") " pod="openshift-ingress/e2e-worker-w2-29ece5-openshift-default-6fcb86dbc-f75sz"
Oct 31 00:46:30.155679 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:46:30.155555    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ffd0eb46-1e6b-4bcc-95bc-eaca5254c056-istio-envoy\") pod \"e2e-worker-w2-29ece5-openshift-default-6fcb86dbc-f75sz\" (UID: \"ffd0eb46-1e6b-4bcc-95bc-eaca5254c056\") " pod="openshift-ingress/e2e-worker-w2-29ece5-openshift-default-6fcb86dbc-f75sz"
Oct 31 00:46:30.155679 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:46:30.155637    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ffd0eb46-1e6b-4bcc-95bc-eaca5254c056-istio-token\") pod \"e2e-worker-w2-29ece5-openshift-default-6fcb86dbc-f75sz\" (UID: \"ffd0eb46-1e6b-4bcc-95bc-eaca5254c056\") " pod="openshift-ingress/e2e-worker-w2-29ece5-openshift-default-6fcb86dbc-f75sz"
Oct 31 00:46:30.155679 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:46:30.155676    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ffd0eb46-1e6b-4bcc-95bc-eaca5254c056-istiod-ca-cert\") pod \"e2e-worker-w2-29ece5-openshift-default-6fcb86dbc-f75sz\" (UID: \"ffd0eb46-1e6b-4bcc-95bc-eaca5254c056\") " pod="openshift-ingress/e2e-worker-w2-29ece5-openshift-default-6fcb86dbc-f75sz"
Oct 31 00:46:30.156021 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:46:30.155896    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ffd0eb46-1e6b-4bcc-95bc-eaca5254c056-workload-certs\") pod \"e2e-worker-w2-29ece5-openshift-default-6fcb86dbc-f75sz\" (UID: \"ffd0eb46-1e6b-4bcc-95bc-eaca5254c056\") " pod="openshift-ingress/e2e-worker-w2-29ece5-openshift-default-6fcb86dbc-f75sz"
Oct 31 00:46:30.156021 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:46:30.155923    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ffd0eb46-1e6b-4bcc-95bc-eaca5254c056-istio-data\") pod \"e2e-worker-w2-29ece5-openshift-default-6fcb86dbc-f75sz\" (UID: \"ffd0eb46-1e6b-4bcc-95bc-eaca5254c056\") " pod="openshift-ingress/e2e-worker-w2-29ece5-openshift-default-6fcb86dbc-f75sz"
Oct 31 00:46:30.156021 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:46:30.155973    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ffd0eb46-1e6b-4bcc-95bc-eaca5254c056-workload-socket\") pod \"e2e-worker-w2-29ece5-openshift-default-6fcb86dbc-f75sz\" (UID: \"ffd0eb46-1e6b-4bcc-95bc-eaca5254c056\") " pod="openshift-ingress/e2e-worker-w2-29ece5-openshift-default-6fcb86dbc-f75sz"
Oct 31 00:46:30.156175 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:46:30.156145    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ffd0eb46-1e6b-4bcc-95bc-eaca5254c056-credential-socket\") pod \"e2e-worker-w2-29ece5-openshift-default-6fcb86dbc-f75sz\" (UID: \"ffd0eb46-1e6b-4bcc-95bc-eaca5254c056\") " pod="openshift-ingress/e2e-worker-w2-29ece5-openshift-default-6fcb86dbc-f75sz"
Oct 31 00:46:30.156453 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:46:30.156433    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ffd0eb46-1e6b-4bcc-95bc-eaca5254c056-istiod-ca-cert\") pod \"e2e-worker-w2-29ece5-openshift-default-6fcb86dbc-f75sz\" (UID: \"ffd0eb46-1e6b-4bcc-95bc-eaca5254c056\") " pod="openshift-ingress/e2e-worker-w2-29ece5-openshift-default-6fcb86dbc-f75sz"
Oct 31 00:46:30.157993 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:46:30.157972    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ffd0eb46-1e6b-4bcc-95bc-eaca5254c056-istio-envoy\") pod \"e2e-worker-w2-29ece5-openshift-default-6fcb86dbc-f75sz\" (UID: \"ffd0eb46-1e6b-4bcc-95bc-eaca5254c056\") " pod="openshift-ingress/e2e-worker-w2-29ece5-openshift-default-6fcb86dbc-f75sz"
Oct 31 00:46:30.158080 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:46:30.158001    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ffd0eb46-1e6b-4bcc-95bc-eaca5254c056-istio-podinfo\") pod \"e2e-worker-w2-29ece5-openshift-default-6fcb86dbc-f75sz\" (UID: \"ffd0eb46-1e6b-4bcc-95bc-eaca5254c056\") " pod="openshift-ingress/e2e-worker-w2-29ece5-openshift-default-6fcb86dbc-f75sz"
Oct 31 00:46:30.222243 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:46:30.222141    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ffd0eb46-1e6b-4bcc-95bc-eaca5254c056-istio-token\") pod \"e2e-worker-w2-29ece5-openshift-default-6fcb86dbc-f75sz\" (UID: \"ffd0eb46-1e6b-4bcc-95bc-eaca5254c056\") " pod="openshift-ingress/e2e-worker-w2-29ece5-openshift-default-6fcb86dbc-f75sz"
Oct 31 00:46:30.222243 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:46:30.222178    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vcml2\" (UniqueName: \"kubernetes.io/projected/ffd0eb46-1e6b-4bcc-95bc-eaca5254c056-kube-api-access-vcml2\") pod \"e2e-worker-w2-29ece5-openshift-default-6fcb86dbc-f75sz\" (UID: \"ffd0eb46-1e6b-4bcc-95bc-eaca5254c056\") " pod="openshift-ingress/e2e-worker-w2-29ece5-openshift-default-6fcb86dbc-f75sz"
Oct 31 00:46:30.266421 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:46:30.266386    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-29ece5-openshift-default-6fcb86dbc-f75sz"
Oct 31 00:46:30.697186 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:46:30.697153    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-29ece5-openshift-default-6fcb86dbc-f75sz"]
Oct 31 00:46:30.697518 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:46:30.697498    4284 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podffd0eb46_1e6b_4bcc_95bc_eaca5254c056.slice/crio-ee06bb1149ffaf293ae579d6309b8ea9bf16d0398bb6fa43454f20f5a729389c WatchSource:0}: Error finding container ee06bb1149ffaf293ae579d6309b8ea9bf16d0398bb6fa43454f20f5a729389c: Status 404 returned error can't find the container with id ee06bb1149ffaf293ae579d6309b8ea9bf16d0398bb6fa43454f20f5a729389c
Oct 31 00:46:30.700073 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:46:30.700044    4284 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 00:46:30.700153 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:46:30.700108    4284 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 00:46:30.700153 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:46:30.700134    4284 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 00:46:30.921453 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:46:30.921411    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:46:31.446722 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:46:31.446684    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-29ece5-openshift-default-6fcb86dbc-f75sz" event={"ID":"ffd0eb46-1e6b-4bcc-95bc-eaca5254c056","Type":"ContainerStarted","Data":"6cf3ff5d8abe6d686ff1ea7ab91616d4b561140c6ea5cf6e905eb5c37ed8b79c"}
Oct 31 00:46:31.447123 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:46:31.446743    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-29ece5-openshift-default-6fcb86dbc-f75sz" event={"ID":"ffd0eb46-1e6b-4bcc-95bc-eaca5254c056","Type":"ContainerStarted","Data":"ee06bb1149ffaf293ae579d6309b8ea9bf16d0398bb6fa43454f20f5a729389c"}
Oct 31 00:46:31.616040 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:46:31.615960    4284 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w2-29ece5-openshift-default-6fcb86dbc-f75sz" podStartSLOduration=2.615941801 podStartE2EDuration="2.615941801s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 00:46:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:46:31.614685227 +0000 UTC m=+1105.759692143" watchObservedRunningTime="2026-10-31 00:46:31.615941801 +0000 UTC m=+1105.760948718"
Oct 31 00:46:32.266916 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:46:32.266888    4284 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-29ece5-openshift-default-6fcb86dbc-f75sz"
Oct 31 00:46:32.268645 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:46:32.268618    4284 patch_prober.go:29] interesting pod/e2e-worker-w2-29ece5-openshift-default-6fcb86dbc-f75sz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused" start-of-body=
Oct 31 00:46:32.268791 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:46:32.268673    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-29ece5-openshift-default-6fcb86dbc-f75sz" podUID="ffd0eb46-1e6b-4bcc-95bc-eaca5254c056" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused"
Oct 31 00:46:33.267562 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:46:33.267530    4284 patch_prober.go:29] interesting pod/e2e-worker-w2-29ece5-openshift-default-6fcb86dbc-f75sz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused" start-of-body=
Oct 31 00:46:33.267971 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:46:33.267581    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-29ece5-openshift-default-6fcb86dbc-f75sz" podUID="ffd0eb46-1e6b-4bcc-95bc-eaca5254c056" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused"
Oct 31 00:46:34.280050 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:46:34.280017    4284 patch_prober.go:29] interesting pod/e2e-worker-w2-29ece5-openshift-default-6fcb86dbc-f75sz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused" start-of-body=
Oct 31 00:46:34.280709 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:46:34.280668    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-29ece5-openshift-default-6fcb86dbc-f75sz" podUID="ffd0eb46-1e6b-4bcc-95bc-eaca5254c056" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused"
Oct 31 00:46:35.267246 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:46:35.267190    4284 patch_prober.go:29] interesting pod/e2e-worker-w2-29ece5-openshift-default-6fcb86dbc-f75sz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused" start-of-body=
Oct 31 00:46:35.267576 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:46:35.267527    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-29ece5-openshift-default-6fcb86dbc-f75sz" podUID="ffd0eb46-1e6b-4bcc-95bc-eaca5254c056" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused"
Oct 31 00:46:35.753175 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:46:35.753138    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:46:36.267698 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:46:36.267662    4284 patch_prober.go:29] interesting pod/e2e-worker-w2-29ece5-openshift-default-6fcb86dbc-f75sz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused" start-of-body=
Oct 31 00:46:36.267883 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:46:36.267717    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-29ece5-openshift-default-6fcb86dbc-f75sz" podUID="ffd0eb46-1e6b-4bcc-95bc-eaca5254c056" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused"
Oct 31 00:46:37.266938 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:46:37.266902    4284 patch_prober.go:29] interesting pod/e2e-worker-w2-29ece5-openshift-default-6fcb86dbc-f75sz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused" start-of-body=
Oct 31 00:46:37.267379 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:46:37.266969    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-29ece5-openshift-default-6fcb86dbc-f75sz" podUID="ffd0eb46-1e6b-4bcc-95bc-eaca5254c056" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused"
Oct 31 00:46:38.077945 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:46:38.077902    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:46:38.227743 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:46:38.227702    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:46:38.388337 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:46:38.387153    4284 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-29ece5-openshift-default-6fcb86dbc-f75sz"
Oct 31 00:46:38.388337 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:46:38.388098    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-29ece5-openshift-default-6fcb86dbc-f75sz"
Oct 31 00:46:38.389134 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:46:38.389110    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-29ece5-openshift-default-6fcb86dbc-f75sz"
Oct 31 00:46:38.950077 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:46:38.950038    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:46:39.250464 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:46:39.250349    4284 prober.go:122] "Probe failed" probeType="Readiness" pod="models-as-a-service/payload-processing-external-model-5747777fbb-w572x" podUID="863ee1ca-55df-442d-b754-124eb102c5b6" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 31 00:46:40.594601 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:46:40.594571    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:46:45.462826 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:46:45.462792    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:46:47.014573 ip-10-0-132-173 kubenswrapper[4284]: E1031 00:46:47.014497    4284 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fd8sp" containerName="extractor"
Oct 31 00:46:49.210456 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:46:49.210418    4284 prober.go:122] "Probe failed" probeType="Readiness" pod="models-as-a-service/payload-processing-external-model-5747777fbb-w572x" podUID="863ee1ca-55df-442d-b754-124eb102c5b6" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 31 00:46:50.197094 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:46:50.197051    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:46:54.739244 ip-10-0-132-173 kubenswrapper[4284]: E1031 00:46:54.736362    4284 projected.go:266] Couldn't get secret models-as-a-service/agc-cred-bf4919d00529: secret "agc-cred-bf4919d00529" not found
Oct 31 00:46:54.739244 ip-10-0-132-173 kubenswrapper[4284]: E1031 00:46:54.736402    4284 projected.go:196] Error preparing data for projected volume provider-credentials for pod models-as-a-service/payload-processing-external-model-56f84cd566-gzxbm: secret "agc-cred-bf4919d00529" not found
Oct 31 00:46:54.739244 ip-10-0-132-173 kubenswrapper[4284]: E1031 00:46:54.736503    4284 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a5b686fc-3da6-40c0-9619-86357c50fd1e-provider-credentials podName:a5b686fc-3da6-40c0-9619-86357c50fd1e nodeName:}" failed. No retries permitted until 2026-10-31 00:47:58.736482025 +0000 UTC m=+1192.881488943 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "provider-credentials" (UniqueName: "kubernetes.io/projected/a5b686fc-3da6-40c0-9619-86357c50fd1e-provider-credentials") pod "payload-processing-external-model-56f84cd566-gzxbm" (UID: "a5b686fc-3da6-40c0-9619-86357c50fd1e") : secret "agc-cred-bf4919d00529" not found
Oct 31 00:46:55.015366 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:46:55.015088    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:46:55.217236 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:46:55.215156    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:46:59.209850 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:46:59.209815    4284 prober.go:122] "Probe failed" probeType="Readiness" pod="models-as-a-service/payload-processing-external-model-5747777fbb-w572x" podUID="863ee1ca-55df-442d-b754-124eb102c5b6" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 31 00:46:59.886312 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:46:59.886270    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:47:04.705821 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:47:04.705764    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:47:07.505142 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:47:07.505103    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:47:07.649695 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:47:07.649661    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:47:09.210521 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:47:09.210482    4284 prober.go:122] "Probe failed" probeType="Readiness" pod="models-as-a-service/payload-processing-external-model-5747777fbb-w572x" podUID="863ee1ca-55df-442d-b754-124eb102c5b6" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 31 00:47:09.494191 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:47:09.494096    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:47:11.957366 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:47:11.957315    4284 prober.go:122] "Probe failed" probeType="Readiness" pod="models-as-a-service/payload-processing-external-model-5747777fbb-w572x" podUID="863ee1ca-55df-442d-b754-124eb102c5b6" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 31 00:47:14.322316 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:47:14.322272    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:47:14.957499 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:47:14.957463    4284 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-6njss"]
Oct 31 00:47:14.962952 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:47:14.962923    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-6njss"
Oct 31 00:47:15.000762 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:47:15.000730    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7b453508-9670-49c5-8a23-164a9a59e814-workload-socket\") pod \"e2e-worker-w4-ef2572-openshift-default-7649dd87-6njss\" (UID: \"7b453508-9670-49c5-8a23-164a9a59e814\") " pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-6njss"
Oct 31 00:47:15.000955 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:47:15.000789    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7b453508-9670-49c5-8a23-164a9a59e814-credential-socket\") pod \"e2e-worker-w4-ef2572-openshift-default-7649dd87-6njss\" (UID: \"7b453508-9670-49c5-8a23-164a9a59e814\") " pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-6njss"
Oct 31 00:47:15.000955 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:47:15.000882    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7b453508-9670-49c5-8a23-164a9a59e814-istio-token\") pod \"e2e-worker-w4-ef2572-openshift-default-7649dd87-6njss\" (UID: \"7b453508-9670-49c5-8a23-164a9a59e814\") " pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-6njss"
Oct 31 00:47:15.000955 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:47:15.000929    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vj94b\" (UniqueName: \"kubernetes.io/projected/7b453508-9670-49c5-8a23-164a9a59e814-kube-api-access-vj94b\") pod \"e2e-worker-w4-ef2572-openshift-default-7649dd87-6njss\" (UID: \"7b453508-9670-49c5-8a23-164a9a59e814\") " pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-6njss"
Oct 31 00:47:15.001099 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:47:15.001007    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7b453508-9670-49c5-8a23-164a9a59e814-istiod-ca-cert\") pod \"e2e-worker-w4-ef2572-openshift-default-7649dd87-6njss\" (UID: \"7b453508-9670-49c5-8a23-164a9a59e814\") " pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-6njss"
Oct 31 00:47:15.001099 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:47:15.001077    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7b453508-9670-49c5-8a23-164a9a59e814-istio-podinfo\") pod \"e2e-worker-w4-ef2572-openshift-default-7649dd87-6njss\" (UID: \"7b453508-9670-49c5-8a23-164a9a59e814\") " pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-6njss"
Oct 31 00:47:15.001180 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:47:15.001121    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7b453508-9670-49c5-8a23-164a9a59e814-workload-certs\") pod \"e2e-worker-w4-ef2572-openshift-default-7649dd87-6njss\" (UID: \"7b453508-9670-49c5-8a23-164a9a59e814\") " pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-6njss"
Oct 31 00:47:15.001180 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:47:15.001169    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7b453508-9670-49c5-8a23-164a9a59e814-istio-envoy\") pod \"e2e-worker-w4-ef2572-openshift-default-7649dd87-6njss\" (UID: \"7b453508-9670-49c5-8a23-164a9a59e814\") " pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-6njss"
Oct 31 00:47:15.001307 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:47:15.001205    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7b453508-9670-49c5-8a23-164a9a59e814-istio-data\") pod \"e2e-worker-w4-ef2572-openshift-default-7649dd87-6njss\" (UID: \"7b453508-9670-49c5-8a23-164a9a59e814\") " pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-6njss"
Oct 31 00:47:15.064001 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:47:15.063972    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-6njss"]
Oct 31 00:47:15.103019 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:47:15.102974    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7b453508-9670-49c5-8a23-164a9a59e814-istio-podinfo\") pod \"e2e-worker-w4-ef2572-openshift-default-7649dd87-6njss\" (UID: \"7b453508-9670-49c5-8a23-164a9a59e814\") " pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-6njss"
Oct 31 00:47:15.103231 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:47:15.103058    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7b453508-9670-49c5-8a23-164a9a59e814-workload-certs\") pod \"e2e-worker-w4-ef2572-openshift-default-7649dd87-6njss\" (UID: \"7b453508-9670-49c5-8a23-164a9a59e814\") " pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-6njss"
Oct 31 00:47:15.103231 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:47:15.103101    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7b453508-9670-49c5-8a23-164a9a59e814-istio-envoy\") pod \"e2e-worker-w4-ef2572-openshift-default-7649dd87-6njss\" (UID: \"7b453508-9670-49c5-8a23-164a9a59e814\") " pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-6njss"
Oct 31 00:47:15.103231 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:47:15.103140    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7b453508-9670-49c5-8a23-164a9a59e814-istio-data\") pod \"e2e-worker-w4-ef2572-openshift-default-7649dd87-6njss\" (UID: \"7b453508-9670-49c5-8a23-164a9a59e814\") " pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-6njss"
Oct 31 00:47:15.103231 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:47:15.103182    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7b453508-9670-49c5-8a23-164a9a59e814-workload-socket\") pod \"e2e-worker-w4-ef2572-openshift-default-7649dd87-6njss\" (UID: \"7b453508-9670-49c5-8a23-164a9a59e814\") " pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-6njss"
Oct 31 00:47:15.103452 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:47:15.103259    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7b453508-9670-49c5-8a23-164a9a59e814-credential-socket\") pod \"e2e-worker-w4-ef2572-openshift-default-7649dd87-6njss\" (UID: \"7b453508-9670-49c5-8a23-164a9a59e814\") " pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-6njss"
Oct 31 00:47:15.103452 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:47:15.103323    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7b453508-9670-49c5-8a23-164a9a59e814-istio-token\") pod \"e2e-worker-w4-ef2572-openshift-default-7649dd87-6njss\" (UID: \"7b453508-9670-49c5-8a23-164a9a59e814\") " pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-6njss"
Oct 31 00:47:15.103452 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:47:15.103384    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vj94b\" (UniqueName: \"kubernetes.io/projected/7b453508-9670-49c5-8a23-164a9a59e814-kube-api-access-vj94b\") pod \"e2e-worker-w4-ef2572-openshift-default-7649dd87-6njss\" (UID: \"7b453508-9670-49c5-8a23-164a9a59e814\") " pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-6njss"
Oct 31 00:47:15.103452 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:47:15.103438    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7b453508-9670-49c5-8a23-164a9a59e814-istiod-ca-cert\") pod \"e2e-worker-w4-ef2572-openshift-default-7649dd87-6njss\" (UID: \"7b453508-9670-49c5-8a23-164a9a59e814\") " pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-6njss"
Oct 31 00:47:15.103688 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:47:15.103656    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7b453508-9670-49c5-8a23-164a9a59e814-istio-data\") pod \"e2e-worker-w4-ef2572-openshift-default-7649dd87-6njss\" (UID: \"7b453508-9670-49c5-8a23-164a9a59e814\") " pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-6njss"
Oct 31 00:47:15.103758 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:47:15.103664    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7b453508-9670-49c5-8a23-164a9a59e814-credential-socket\") pod \"e2e-worker-w4-ef2572-openshift-default-7649dd87-6njss\" (UID: \"7b453508-9670-49c5-8a23-164a9a59e814\") " pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-6njss"
Oct 31 00:47:15.104136 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:47:15.104107    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7b453508-9670-49c5-8a23-164a9a59e814-workload-socket\") pod \"e2e-worker-w4-ef2572-openshift-default-7649dd87-6njss\" (UID: \"7b453508-9670-49c5-8a23-164a9a59e814\") " pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-6njss"
Oct 31 00:47:15.104269 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:47:15.104202    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7b453508-9670-49c5-8a23-164a9a59e814-istiod-ca-cert\") pod \"e2e-worker-w4-ef2572-openshift-default-7649dd87-6njss\" (UID: \"7b453508-9670-49c5-8a23-164a9a59e814\") " pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-6njss"
Oct 31 00:47:15.104377 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:47:15.104351    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7b453508-9670-49c5-8a23-164a9a59e814-workload-certs\") pod \"e2e-worker-w4-ef2572-openshift-default-7649dd87-6njss\" (UID: \"7b453508-9670-49c5-8a23-164a9a59e814\") " pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-6njss"
Oct 31 00:47:15.105755 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:47:15.105727    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7b453508-9670-49c5-8a23-164a9a59e814-istio-envoy\") pod \"e2e-worker-w4-ef2572-openshift-default-7649dd87-6njss\" (UID: \"7b453508-9670-49c5-8a23-164a9a59e814\") " pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-6njss"
Oct 31 00:47:15.106064 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:47:15.106046    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7b453508-9670-49c5-8a23-164a9a59e814-istio-podinfo\") pod \"e2e-worker-w4-ef2572-openshift-default-7649dd87-6njss\" (UID: \"7b453508-9670-49c5-8a23-164a9a59e814\") " pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-6njss"
Oct 31 00:47:15.162320 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:47:15.162281    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7b453508-9670-49c5-8a23-164a9a59e814-istio-token\") pod \"e2e-worker-w4-ef2572-openshift-default-7649dd87-6njss\" (UID: \"7b453508-9670-49c5-8a23-164a9a59e814\") " pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-6njss"
Oct 31 00:47:15.162505 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:47:15.162482    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vj94b\" (UniqueName: \"kubernetes.io/projected/7b453508-9670-49c5-8a23-164a9a59e814-kube-api-access-vj94b\") pod \"e2e-worker-w4-ef2572-openshift-default-7649dd87-6njss\" (UID: \"7b453508-9670-49c5-8a23-164a9a59e814\") " pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-6njss"
Oct 31 00:47:15.275866 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:47:15.275788    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-6njss"
Oct 31 00:47:15.701595 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:47:15.701564    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-6njss"]
Oct 31 00:47:15.702204 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:47:15.702183    4284 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7b453508_9670_49c5_8a23_164a9a59e814.slice/crio-709fee708e8366ca319a3889cdbdddc0ff909d76ffdb1fcc09b4613261d7633a WatchSource:0}: Error finding container 709fee708e8366ca319a3889cdbdddc0ff909d76ffdb1fcc09b4613261d7633a: Status 404 returned error can't find the container with id 709fee708e8366ca319a3889cdbdddc0ff909d76ffdb1fcc09b4613261d7633a
Oct 31 00:47:15.704632 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:47:15.704598    4284 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 00:47:15.704715 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:47:15.704668    4284 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 00:47:15.704715 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:47:15.704695    4284 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 00:47:15.732892 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:47:15.732857    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-6njss" event={"ID":"7b453508-9670-49c5-8a23-164a9a59e814","Type":"ContainerStarted","Data":"709fee708e8366ca319a3889cdbdddc0ff909d76ffdb1fcc09b4613261d7633a"}
Oct 31 00:47:16.510845 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:47:16.510805    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:47:16.674200 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:47:16.674164    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:47:16.741379 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:47:16.741331    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-6njss" event={"ID":"7b453508-9670-49c5-8a23-164a9a59e814","Type":"ContainerStarted","Data":"31e60692429111a3e0a59c25d6fc1c088ba778b121d3749c1b31e3dcb0dc1017"}
Oct 31 00:47:16.830440 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:47:16.830363    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:47:16.952236 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:47:16.951751    4284 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-6njss" podStartSLOduration=2.951725788 podStartE2EDuration="2.951725788s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 00:47:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:47:16.928862682 +0000 UTC m=+1151.073869599" watchObservedRunningTime="2026-10-31 00:47:16.951725788 +0000 UTC m=+1151.096732705"
Oct 31 00:47:17.165560 ip-10-0-132-173 kubenswrapper[4284]: E1031 00:47:17.165518    4284 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fd8sp" containerName="extractor"
Oct 31 00:47:17.277101 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:47:17.277074    4284 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-6njss"
Oct 31 00:47:17.278597 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:47:17.278574    4284 patch_prober.go:29] interesting pod/e2e-worker-w4-ef2572-openshift-default-7649dd87-6njss container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused" start-of-body=
Oct 31 00:47:17.278800 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:47:17.278781    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-6njss" podUID="7b453508-9670-49c5-8a23-164a9a59e814" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused"
Oct 31 00:47:18.276905 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:47:18.276875    4284 patch_prober.go:29] interesting pod/e2e-worker-w4-ef2572-openshift-default-7649dd87-6njss container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused" start-of-body=
Oct 31 00:47:18.277336 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:47:18.276923    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-6njss" podUID="7b453508-9670-49c5-8a23-164a9a59e814" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused"
Oct 31 00:47:18.637264 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:47:18.637159    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:47:19.277141 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:47:19.277103    4284 patch_prober.go:29] interesting pod/e2e-worker-w4-ef2572-openshift-default-7649dd87-6njss container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused" start-of-body=
Oct 31 00:47:19.277658 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:47:19.277155    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-6njss" podUID="7b453508-9670-49c5-8a23-164a9a59e814" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused"
Oct 31 00:47:19.534452 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:47:19.534355    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:47:20.276622 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:47:20.276576    4284 patch_prober.go:29] interesting pod/e2e-worker-w4-ef2572-openshift-default-7649dd87-6njss container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused" start-of-body=
Oct 31 00:47:20.276828 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:47:20.276636    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-6njss" podUID="7b453508-9670-49c5-8a23-164a9a59e814" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused"
Oct 31 00:47:21.015283 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:47:21.015249    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["models-as-a-service/payload-processing-external-model-5747777fbb-w572x"]
Oct 31 00:47:21.016621 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:47:21.016594    4284 prober.go:122] "Probe failed" probeType="Readiness" pod="models-as-a-service/payload-processing-external-model-5747777fbb-w572x" podUID="863ee1ca-55df-442d-b754-124eb102c5b6" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 31 00:47:21.055675 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:47:21.055645    4284 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["models-as-a-service/payload-processing-external-model-84cf56844-6pbmt"]
Oct 31 00:47:21.059691 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:47:21.059666    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="models-as-a-service/payload-processing-external-model-84cf56844-6pbmt"
Oct 31 00:47:21.083666 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:47:21.083643    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/e78a1e6c-10b0-4faf-b773-ffd07127b046-provider-credentials\") pod \"payload-processing-external-model-84cf56844-6pbmt\" (UID: \"e78a1e6c-10b0-4faf-b773-ffd07127b046\") " pod="models-as-a-service/payload-processing-external-model-84cf56844-6pbmt"
Oct 31 00:47:21.083807 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:47:21.083680    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e78a1e6c-10b0-4faf-b773-ffd07127b046-plugins-config-volume\") pod \"payload-processing-external-model-84cf56844-6pbmt\" (UID: \"e78a1e6c-10b0-4faf-b773-ffd07127b046\") " pod="models-as-a-service/payload-processing-external-model-84cf56844-6pbmt"
Oct 31 00:47:21.083876 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:47:21.083810    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tn7tk\" (UniqueName: \"kubernetes.io/projected/e78a1e6c-10b0-4faf-b773-ffd07127b046-kube-api-access-tn7tk\") pod \"payload-processing-external-model-84cf56844-6pbmt\" (UID: \"e78a1e6c-10b0-4faf-b773-ffd07127b046\") " pod="models-as-a-service/payload-processing-external-model-84cf56844-6pbmt"
Oct 31 00:47:21.083876 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:47:21.083867    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/e78a1e6c-10b0-4faf-b773-ffd07127b046-routing-overlay-volume\") pod \"payload-processing-external-model-84cf56844-6pbmt\" (UID: \"e78a1e6c-10b0-4faf-b773-ffd07127b046\") " pod="models-as-a-service/payload-processing-external-model-84cf56844-6pbmt"
Oct 31 00:47:21.110274 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:47:21.110233    4284 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"models-as-a-service\"/\"agc-cred-da8c34345bbb\""
Oct 31 00:47:21.110477 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:47:21.110315    4284 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"models-as-a-service\"/\"agc-cred-001239fd95f6\""
Oct 31 00:47:21.121756 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:47:21.121725    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["models-as-a-service/payload-processing-external-model-84cf56844-6pbmt"]
Oct 31 00:47:21.209407 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:47:21.184975    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/e78a1e6c-10b0-4faf-b773-ffd07127b046-provider-credentials\") pod \"payload-processing-external-model-84cf56844-6pbmt\" (UID: \"e78a1e6c-10b0-4faf-b773-ffd07127b046\") " pod="models-as-a-service/payload-processing-external-model-84cf56844-6pbmt"
Oct 31 00:47:21.209684 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:47:21.209662    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e78a1e6c-10b0-4faf-b773-ffd07127b046-plugins-config-volume\") pod \"payload-processing-external-model-84cf56844-6pbmt\" (UID: \"e78a1e6c-10b0-4faf-b773-ffd07127b046\") " pod="models-as-a-service/payload-processing-external-model-84cf56844-6pbmt"
Oct 31 00:47:21.209842 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:47:21.209830    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tn7tk\" (UniqueName: \"kubernetes.io/projected/e78a1e6c-10b0-4faf-b773-ffd07127b046-kube-api-access-tn7tk\") pod \"payload-processing-external-model-84cf56844-6pbmt\" (UID: \"e78a1e6c-10b0-4faf-b773-ffd07127b046\") " pod="models-as-a-service/payload-processing-external-model-84cf56844-6pbmt"
Oct 31 00:47:21.209944 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:47:21.209934    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/e78a1e6c-10b0-4faf-b773-ffd07127b046-routing-overlay-volume\") pod \"payload-processing-external-model-84cf56844-6pbmt\" (UID: \"e78a1e6c-10b0-4faf-b773-ffd07127b046\") " pod="models-as-a-service/payload-processing-external-model-84cf56844-6pbmt"
Oct 31 00:47:21.210992 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:47:21.210939    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/e78a1e6c-10b0-4faf-b773-ffd07127b046-routing-overlay-volume\") pod \"payload-processing-external-model-84cf56844-6pbmt\" (UID: \"e78a1e6c-10b0-4faf-b773-ffd07127b046\") " pod="models-as-a-service/payload-processing-external-model-84cf56844-6pbmt"
Oct 31 00:47:21.211556 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:47:21.211516    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/e78a1e6c-10b0-4faf-b773-ffd07127b046-provider-credentials\") pod \"payload-processing-external-model-84cf56844-6pbmt\" (UID: \"e78a1e6c-10b0-4faf-b773-ffd07127b046\") " pod="models-as-a-service/payload-processing-external-model-84cf56844-6pbmt"
Oct 31 00:47:21.211856 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:47:21.211842    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e78a1e6c-10b0-4faf-b773-ffd07127b046-plugins-config-volume\") pod \"payload-processing-external-model-84cf56844-6pbmt\" (UID: \"e78a1e6c-10b0-4faf-b773-ffd07127b046\") " pod="models-as-a-service/payload-processing-external-model-84cf56844-6pbmt"
Oct 31 00:47:21.268713 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:47:21.268634    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tn7tk\" (UniqueName: \"kubernetes.io/projected/e78a1e6c-10b0-4faf-b773-ffd07127b046-kube-api-access-tn7tk\") pod \"payload-processing-external-model-84cf56844-6pbmt\" (UID: \"e78a1e6c-10b0-4faf-b773-ffd07127b046\") " pod="models-as-a-service/payload-processing-external-model-84cf56844-6pbmt"
Oct 31 00:47:21.276750 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:47:21.276722    4284 patch_prober.go:29] interesting pod/e2e-worker-w4-ef2572-openshift-default-7649dd87-6njss container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused" start-of-body=
Oct 31 00:47:21.276880 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:47:21.276797    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-6njss" podUID="7b453508-9670-49c5-8a23-164a9a59e814" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused"
Oct 31 00:47:21.373020 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:47:21.372987    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="models-as-a-service/payload-processing-external-model-84cf56844-6pbmt"
Oct 31 00:47:21.771408 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:47:21.771374    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["models-as-a-service/payload-processing-external-model-84cf56844-6pbmt"]
Oct 31 00:47:21.772089 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:47:21.772064    4284 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode78a1e6c_10b0_4faf_b773_ffd07127b046.slice/crio-407f7e1786c82364d4a305cf50d797dc0560ab0f5b950d3e8d68c776808e7089 WatchSource:0}: Error finding container 407f7e1786c82364d4a305cf50d797dc0560ab0f5b950d3e8d68c776808e7089: Status 404 returned error can't find the container with id 407f7e1786c82364d4a305cf50d797dc0560ab0f5b950d3e8d68c776808e7089
Oct 31 00:47:21.825632 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:47:21.825605    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="models-as-a-service/payload-processing-external-model-84cf56844-6pbmt" event={"ID":"e78a1e6c-10b0-4faf-b773-ffd07127b046","Type":"ContainerStarted","Data":"407f7e1786c82364d4a305cf50d797dc0560ab0f5b950d3e8d68c776808e7089"}
Oct 31 00:47:22.277152 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:47:22.277123    4284 patch_prober.go:29] interesting pod/e2e-worker-w4-ef2572-openshift-default-7649dd87-6njss container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused" start-of-body=
Oct 31 00:47:22.277554 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:47:22.277172    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-6njss" podUID="7b453508-9670-49c5-8a23-164a9a59e814" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused"
Oct 31 00:47:22.835179 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:47:22.835146    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="models-as-a-service/payload-processing-external-model-84cf56844-6pbmt" event={"ID":"e78a1e6c-10b0-4faf-b773-ffd07127b046","Type":"ContainerStarted","Data":"b821160793a38fdf501f377bb267c0e485d23d9170d6cdcd1feae00f50311c5e"}
Oct 31 00:47:22.835385 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:47:22.835206    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="models-as-a-service/payload-processing-external-model-84cf56844-6pbmt"
Oct 31 00:47:22.948331 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:47:22.948277    4284 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="models-as-a-service/payload-processing-external-model-84cf56844-6pbmt" podStartSLOduration=1.9482644009999999 podStartE2EDuration="1.948264401s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 00:47:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:47:22.947320349 +0000 UTC m=+1157.092327283" watchObservedRunningTime="2026-10-31 00:47:22.948264401 +0000 UTC m=+1157.093271317"
Oct 31 00:47:23.280364 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:47:23.280330    4284 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-6njss"
Oct 31 00:47:23.280842 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:47:23.280619    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-6njss"
Oct 31 00:47:23.281312 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:47:23.281297    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-6njss"
Oct 31 00:47:24.349340 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:47:24.349311    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:47:24.496946 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:47:24.496901    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:47:26.015721 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:47:26.015679    4284 kuberuntime_container.go:898] "Killing container with a grace period" pod="models-as-a-service/payload-processing-external-model-5747777fbb-w572x" podUID="863ee1ca-55df-442d-b754-124eb102c5b6" containerName="payload-processing" containerID="cri-o://9cb4b2c884af337af9d47ab3ed0974eeb51c9e23803acde5ea5d90e0e580eccb" gracePeriod=25
Oct 31 00:47:26.279672 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:47:26.279605    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="models-as-a-service/payload-processing-external-model-5747777fbb-w572x"
Oct 31 00:47:26.388426 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:47:26.388394    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/863ee1ca-55df-442d-b754-124eb102c5b6-kube-api-access-bxrnd\" (UniqueName: \"kubernetes.io/projected/863ee1ca-55df-442d-b754-124eb102c5b6-kube-api-access-bxrnd\") pod \"863ee1ca-55df-442d-b754-124eb102c5b6\" (UID: \"863ee1ca-55df-442d-b754-124eb102c5b6\") "
Oct 31 00:47:26.388653 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:47:26.388593    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/863ee1ca-55df-442d-b754-124eb102c5b6-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/863ee1ca-55df-442d-b754-124eb102c5b6-plugins-config-volume\") pod \"863ee1ca-55df-442d-b754-124eb102c5b6\" (UID: \"863ee1ca-55df-442d-b754-124eb102c5b6\") "
Oct 31 00:47:26.388653 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:47:26.388626    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/863ee1ca-55df-442d-b754-124eb102c5b6-provider-credentials\" (UniqueName: \"kubernetes.io/projected/863ee1ca-55df-442d-b754-124eb102c5b6-provider-credentials\") pod \"863ee1ca-55df-442d-b754-124eb102c5b6\" (UID: \"863ee1ca-55df-442d-b754-124eb102c5b6\") "
Oct 31 00:47:26.388653 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:47:26.388650    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/863ee1ca-55df-442d-b754-124eb102c5b6-routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/863ee1ca-55df-442d-b754-124eb102c5b6-routing-overlay-volume\") pod \"863ee1ca-55df-442d-b754-124eb102c5b6\" (UID: \"863ee1ca-55df-442d-b754-124eb102c5b6\") "
Oct 31 00:47:26.389083 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:47:26.389054    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/863ee1ca-55df-442d-b754-124eb102c5b6-plugins-config-volume" pod "863ee1ca-55df-442d-b754-124eb102c5b6" (UID: "863ee1ca-55df-442d-b754-124eb102c5b6"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 00:47:26.389251 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:47:26.389070    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/863ee1ca-55df-442d-b754-124eb102c5b6-routing-overlay-volume" pod "863ee1ca-55df-442d-b754-124eb102c5b6" (UID: "863ee1ca-55df-442d-b754-124eb102c5b6"). InnerVolumeSpecName "routing-overlay-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 00:47:26.389251 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:47:26.389233    4284 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/863ee1ca-55df-442d-b754-124eb102c5b6-plugins-config-volume\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:47:26.389358 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:47:26.389256    4284 reconciler_common.go:299] "Volume detached for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/863ee1ca-55df-442d-b754-124eb102c5b6-routing-overlay-volume\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:47:26.391028 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:47:26.391006    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/863ee1ca-55df-442d-b754-124eb102c5b6-provider-credentials" pod "863ee1ca-55df-442d-b754-124eb102c5b6" (UID: "863ee1ca-55df-442d-b754-124eb102c5b6"). InnerVolumeSpecName "provider-credentials". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 00:47:26.391124 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:47:26.391044    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/863ee1ca-55df-442d-b754-124eb102c5b6-kube-api-access-bxrnd" pod "863ee1ca-55df-442d-b754-124eb102c5b6" (UID: "863ee1ca-55df-442d-b754-124eb102c5b6"). InnerVolumeSpecName "kube-api-access-bxrnd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 00:47:26.490954 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:47:26.490915    4284 reconciler_common.go:299] "Volume detached for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/863ee1ca-55df-442d-b754-124eb102c5b6-provider-credentials\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:47:26.490954 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:47:26.490946    4284 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bxrnd\" (UniqueName: \"kubernetes.io/projected/863ee1ca-55df-442d-b754-124eb102c5b6-kube-api-access-bxrnd\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:47:26.875472 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:47:26.875435    4284 generic.go:356] "Generic (PLEG): container finished" podID="863ee1ca-55df-442d-b754-124eb102c5b6" containerID="9cb4b2c884af337af9d47ab3ed0974eeb51c9e23803acde5ea5d90e0e580eccb" exitCode=0
Oct 31 00:47:26.876313 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:47:26.876291    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="models-as-a-service/payload-processing-external-model-5747777fbb-w572x"
Oct 31 00:47:26.878314 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:47:26.878281    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="models-as-a-service/payload-processing-external-model-5747777fbb-w572x" event={"ID":"863ee1ca-55df-442d-b754-124eb102c5b6","Type":"ContainerDied","Data":"9cb4b2c884af337af9d47ab3ed0974eeb51c9e23803acde5ea5d90e0e580eccb"}
Oct 31 00:47:26.892709 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:47:26.892648    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="models-as-a-service/payload-processing-external-model-5747777fbb-w572x" event={"ID":"863ee1ca-55df-442d-b754-124eb102c5b6","Type":"ContainerDied","Data":"b95923d75b631f49f188c57971a48b77e1779b244a171687684537156e7fe277"}
Oct 31 00:47:26.893010 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:47:26.892949    4284 scope.go:122] "RemoveContainer" containerID="9cb4b2c884af337af9d47ab3ed0974eeb51c9e23803acde5ea5d90e0e580eccb"
Oct 31 00:47:26.979651 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:47:26.979625    4284 scope.go:122] "RemoveContainer" containerID="9cb4b2c884af337af9d47ab3ed0974eeb51c9e23803acde5ea5d90e0e580eccb"
Oct 31 00:47:26.983233 ip-10-0-132-173 kubenswrapper[4284]: E1031 00:47:26.980073    4284 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9cb4b2c884af337af9d47ab3ed0974eeb51c9e23803acde5ea5d90e0e580eccb\": container with ID starting with 9cb4b2c884af337af9d47ab3ed0974eeb51c9e23803acde5ea5d90e0e580eccb not found: ID does not exist" containerID="9cb4b2c884af337af9d47ab3ed0974eeb51c9e23803acde5ea5d90e0e580eccb"
Oct 31 00:47:26.983233 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:47:26.980113    4284 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9cb4b2c884af337af9d47ab3ed0974eeb51c9e23803acde5ea5d90e0e580eccb"} err="failed to get container status \"9cb4b2c884af337af9d47ab3ed0974eeb51c9e23803acde5ea5d90e0e580eccb\": rpc error: code = NotFound desc = could not find container \"9cb4b2c884af337af9d47ab3ed0974eeb51c9e23803acde5ea5d90e0e580eccb\": container with ID starting with 9cb4b2c884af337af9d47ab3ed0974eeb51c9e23803acde5ea5d90e0e580eccb not found: ID does not exist"
Oct 31 00:47:27.089235 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:47:27.088412    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["models-as-a-service/payload-processing-external-model-5747777fbb-w572x"]
Oct 31 00:47:27.103140 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:47:27.102266    4284 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["models-as-a-service/payload-processing-external-model-5747777fbb-w572x"]
Oct 31 00:47:27.807240 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:47:27.803473    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:47:27.979237 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:47:27.978231    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:47:28.002233 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:47:27.991232    4284 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="863ee1ca-55df-442d-b754-124eb102c5b6" path="/var/lib/kubelet/pods/863ee1ca-55df-442d-b754-124eb102c5b6/volumes"
Oct 31 00:47:30.068237 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:47:30.067317    4284 patch_prober.go:29] interesting pod/e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-k89gx container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.69:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 31 00:47:30.068237 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:47:30.067387    4284 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-k89gx" podUID="41a8f8d6-dbbb-48db-a957-5a3bfe8aa418" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.69:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:47:31.470936 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:47:31.470902    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:47:33.844392 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:47:33.844364    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="models-as-a-service/payload-processing-external-model-84cf56844-6pbmt"
Oct 31 00:47:34.009673 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:47:34.009639    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["models-as-a-service/payload-processing-external-model-56f84cd566-gzxbm"]
Oct 31 00:47:36.354916 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:47:36.354876    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:47:39.010625 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:47:39.010568    4284 kuberuntime_container.go:898] "Killing container with a grace period" pod="models-as-a-service/payload-processing-external-model-56f84cd566-gzxbm" podUID="a5b686fc-3da6-40c0-9619-86357c50fd1e" containerName="payload-processing" containerID="cri-o://dbbaf7d79605dcee18a5b1c7407f8a1ad606705ea7237bff8212989856d34f11" gracePeriod=25
Oct 31 00:47:39.239762 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:47:39.239731    4284 generic.go:356] "Generic (PLEG): container finished" podID="a5b686fc-3da6-40c0-9619-86357c50fd1e" containerID="dbbaf7d79605dcee18a5b1c7407f8a1ad606705ea7237bff8212989856d34f11" exitCode=0
Oct 31 00:47:39.239917 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:47:39.239808    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="models-as-a-service/payload-processing-external-model-56f84cd566-gzxbm" event={"ID":"a5b686fc-3da6-40c0-9619-86357c50fd1e","Type":"ContainerDied","Data":"dbbaf7d79605dcee18a5b1c7407f8a1ad606705ea7237bff8212989856d34f11"}
Oct 31 00:47:39.284742 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:47:39.284711    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="models-as-a-service/payload-processing-external-model-56f84cd566-gzxbm"
Oct 31 00:47:39.408455 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:47:39.408420    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a5b686fc-3da6-40c0-9619-86357c50fd1e-kube-api-access-6bxnn\" (UniqueName: \"kubernetes.io/projected/a5b686fc-3da6-40c0-9619-86357c50fd1e-kube-api-access-6bxnn\") pod \"a5b686fc-3da6-40c0-9619-86357c50fd1e\" (UID: \"a5b686fc-3da6-40c0-9619-86357c50fd1e\") "
Oct 31 00:47:39.408636 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:47:39.408475    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a5b686fc-3da6-40c0-9619-86357c50fd1e-provider-credentials\" (UniqueName: \"kubernetes.io/projected/a5b686fc-3da6-40c0-9619-86357c50fd1e-provider-credentials\") pod \"a5b686fc-3da6-40c0-9619-86357c50fd1e\" (UID: \"a5b686fc-3da6-40c0-9619-86357c50fd1e\") "
Oct 31 00:47:39.408636 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:47:39.408603    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a5b686fc-3da6-40c0-9619-86357c50fd1e-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a5b686fc-3da6-40c0-9619-86357c50fd1e-plugins-config-volume\") pod \"a5b686fc-3da6-40c0-9619-86357c50fd1e\" (UID: \"a5b686fc-3da6-40c0-9619-86357c50fd1e\") "
Oct 31 00:47:39.408775 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:47:39.408653    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a5b686fc-3da6-40c0-9619-86357c50fd1e-routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/a5b686fc-3da6-40c0-9619-86357c50fd1e-routing-overlay-volume\") pod \"a5b686fc-3da6-40c0-9619-86357c50fd1e\" (UID: \"a5b686fc-3da6-40c0-9619-86357c50fd1e\") "
Oct 31 00:47:39.409202 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:47:39.409033    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a5b686fc-3da6-40c0-9619-86357c50fd1e-routing-overlay-volume" pod "a5b686fc-3da6-40c0-9619-86357c50fd1e" (UID: "a5b686fc-3da6-40c0-9619-86357c50fd1e"). InnerVolumeSpecName "routing-overlay-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 00:47:39.409202 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:47:39.409165    4284 reconciler_common.go:299] "Volume detached for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/a5b686fc-3da6-40c0-9619-86357c50fd1e-routing-overlay-volume\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:47:39.409793 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:47:39.409764    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a5b686fc-3da6-40c0-9619-86357c50fd1e-plugins-config-volume" pod "a5b686fc-3da6-40c0-9619-86357c50fd1e" (UID: "a5b686fc-3da6-40c0-9619-86357c50fd1e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 00:47:39.410923 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:47:39.410896    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a5b686fc-3da6-40c0-9619-86357c50fd1e-kube-api-access-6bxnn" pod "a5b686fc-3da6-40c0-9619-86357c50fd1e" (UID: "a5b686fc-3da6-40c0-9619-86357c50fd1e"). InnerVolumeSpecName "kube-api-access-6bxnn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 00:47:39.411174 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:47:39.411151    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a5b686fc-3da6-40c0-9619-86357c50fd1e-provider-credentials" pod "a5b686fc-3da6-40c0-9619-86357c50fd1e" (UID: "a5b686fc-3da6-40c0-9619-86357c50fd1e"). InnerVolumeSpecName "provider-credentials". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 00:47:39.512889 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:47:39.512853    4284 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a5b686fc-3da6-40c0-9619-86357c50fd1e-plugins-config-volume\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:47:39.512889 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:47:39.512887    4284 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6bxnn\" (UniqueName: \"kubernetes.io/projected/a5b686fc-3da6-40c0-9619-86357c50fd1e-kube-api-access-6bxnn\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:47:39.513171 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:47:39.512899    4284 reconciler_common.go:299] "Volume detached for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/a5b686fc-3da6-40c0-9619-86357c50fd1e-provider-credentials\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:47:40.247285 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:47:40.247185    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="models-as-a-service/payload-processing-external-model-56f84cd566-gzxbm"
Oct 31 00:47:40.247285 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:47:40.247235    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="models-as-a-service/payload-processing-external-model-56f84cd566-gzxbm" event={"ID":"a5b686fc-3da6-40c0-9619-86357c50fd1e","Type":"ContainerDied","Data":"344a21ab19a139d03cc2142d230960ce5c23acef51152a68e02c4f1367c1b8cd"}
Oct 31 00:47:40.247831 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:47:40.247304    4284 scope.go:122] "RemoveContainer" containerID="dbbaf7d79605dcee18a5b1c7407f8a1ad606705ea7237bff8212989856d34f11"
Oct 31 00:47:40.405545 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:47:40.405508    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["models-as-a-service/payload-processing-external-model-56f84cd566-gzxbm"]
Oct 31 00:47:40.452844 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:47:40.452809    4284 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["models-as-a-service/payload-processing-external-model-56f84cd566-gzxbm"]
Oct 31 00:47:41.077365 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:47:41.077325    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:47:41.962691 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:47:41.962658    4284 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a5b686fc-3da6-40c0-9619-86357c50fd1e" path="/var/lib/kubelet/pods/a5b686fc-3da6-40c0-9619-86357c50fd1e/volumes"
Oct 31 00:47:45.906354 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:47:45.906320    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:47:47.033342 ip-10-0-132-173 kubenswrapper[4284]: E1031 00:47:47.033299    4284 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fd8sp" containerName="extractor"
Oct 31 00:47:51.328747 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:47:51.328701    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:47:55.334129 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:47:55.334082    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:47:56.080803 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:47:56.080763    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:48:01.011724 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:01.011687    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:48:05.350570 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:05.350533    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:48:05.550360 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:05.550324    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:48:05.802524 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:05.802484    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:48:10.696147 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:10.696104    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:48:13.543935 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:13.543896    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:48:15.689819 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:15.689780    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:48:17.014475 ip-10-0-132-173 kubenswrapper[4284]: E1031 00:48:17.014390    4284 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fd8sp" containerName="extractor"
Oct 31 00:48:17.491140 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:17.491105    4284 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w4-ef2572/distinct-2-w4-ef2572-kserve-547fccbd5-dfpfx"]
Oct 31 00:48:17.491808 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:17.491784    4284 cpu_manager.go:414] "containerMap: removing container" podUID="a5b686fc-3da6-40c0-9619-86357c50fd1e" containerName="payload-processing"
Oct 31 00:48:17.491808 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:17.491807    4284 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a5b686fc-3da6-40c0-9619-86357c50fd1e" containerName="payload-processing"
Oct 31 00:48:17.491978 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:17.491837    4284 cpu_manager.go:414] "containerMap: removing container" podUID="863ee1ca-55df-442d-b754-124eb102c5b6" containerName="payload-processing"
Oct 31 00:48:17.491978 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:17.491842    4284 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="863ee1ca-55df-442d-b754-124eb102c5b6" containerName="payload-processing"
Oct 31 00:48:17.491978 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:17.491918    4284 memory_manager.go:365] "RemoveStaleState removing state" podUID="863ee1ca-55df-442d-b754-124eb102c5b6" containerName="payload-processing"
Oct 31 00:48:17.491978 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:17.491928    4284 memory_manager.go:365] "RemoveStaleState removing state" podUID="a5b686fc-3da6-40c0-9619-86357c50fd1e" containerName="payload-processing"
Oct 31 00:48:17.496686 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:17.496668    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-ef2572/distinct-2-w4-ef2572-kserve-547fccbd5-dfpfx"
Oct 31 00:48:17.549845 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:17.549814    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-ef2572/distinct-2-w4-ef2572-kserve-547fccbd5-dfpfx"]
Oct 31 00:48:17.567623 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:17.567590    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cb311a47-264c-4b33-bc31-377b94b0c59b-dshm\") pod \"distinct-2-w4-ef2572-kserve-547fccbd5-dfpfx\" (UID: \"cb311a47-264c-4b33-bc31-377b94b0c59b\") " pod="e2e-models-e2e-worker-w4-ef2572/distinct-2-w4-ef2572-kserve-547fccbd5-dfpfx"
Oct 31 00:48:17.567623 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:17.567624    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cb311a47-264c-4b33-bc31-377b94b0c59b-model-cache\") pod \"distinct-2-w4-ef2572-kserve-547fccbd5-dfpfx\" (UID: \"cb311a47-264c-4b33-bc31-377b94b0c59b\") " pod="e2e-models-e2e-worker-w4-ef2572/distinct-2-w4-ef2572-kserve-547fccbd5-dfpfx"
Oct 31 00:48:17.567832 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:17.567789    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cb311a47-264c-4b33-bc31-377b94b0c59b-tls-certs\") pod \"distinct-2-w4-ef2572-kserve-547fccbd5-dfpfx\" (UID: \"cb311a47-264c-4b33-bc31-377b94b0c59b\") " pod="e2e-models-e2e-worker-w4-ef2572/distinct-2-w4-ef2572-kserve-547fccbd5-dfpfx"
Oct 31 00:48:17.567870 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:17.567832    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cb311a47-264c-4b33-bc31-377b94b0c59b-tmp-dir\") pod \"distinct-2-w4-ef2572-kserve-547fccbd5-dfpfx\" (UID: \"cb311a47-264c-4b33-bc31-377b94b0c59b\") " pod="e2e-models-e2e-worker-w4-ef2572/distinct-2-w4-ef2572-kserve-547fccbd5-dfpfx"
Oct 31 00:48:17.567985 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:17.567967    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2qczm\" (UniqueName: \"kubernetes.io/projected/cb311a47-264c-4b33-bc31-377b94b0c59b-kube-api-access-2qczm\") pod \"distinct-2-w4-ef2572-kserve-547fccbd5-dfpfx\" (UID: \"cb311a47-264c-4b33-bc31-377b94b0c59b\") " pod="e2e-models-e2e-worker-w4-ef2572/distinct-2-w4-ef2572-kserve-547fccbd5-dfpfx"
Oct 31 00:48:17.568029 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:17.568002    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cb311a47-264c-4b33-bc31-377b94b0c59b-home\") pod \"distinct-2-w4-ef2572-kserve-547fccbd5-dfpfx\" (UID: \"cb311a47-264c-4b33-bc31-377b94b0c59b\") " pod="e2e-models-e2e-worker-w4-ef2572/distinct-2-w4-ef2572-kserve-547fccbd5-dfpfx"
Oct 31 00:48:17.594679 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:17.594639    4284 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-ef2572\"/\"distinct-2-w4-ef2572-kserve-self-signed-certs\""
Oct 31 00:48:17.594679 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:17.594644    4284 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-ef2572\"/\"default-dockercfg-59hf4\""
Oct 31 00:48:17.594913 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:17.594701    4284 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w4-ef2572\"/\"kube-root-ca.crt\""
Oct 31 00:48:17.594913 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:17.594707    4284 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w4-ef2572\"/\"openshift-service-ca.crt\""
Oct 31 00:48:17.612261 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:17.612196    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:48:17.675052 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:17.675011    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cb311a47-264c-4b33-bc31-377b94b0c59b-dshm\") pod \"distinct-2-w4-ef2572-kserve-547fccbd5-dfpfx\" (UID: \"cb311a47-264c-4b33-bc31-377b94b0c59b\") " pod="e2e-models-e2e-worker-w4-ef2572/distinct-2-w4-ef2572-kserve-547fccbd5-dfpfx"
Oct 31 00:48:17.689235 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:17.675351    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cb311a47-264c-4b33-bc31-377b94b0c59b-model-cache\") pod \"distinct-2-w4-ef2572-kserve-547fccbd5-dfpfx\" (UID: \"cb311a47-264c-4b33-bc31-377b94b0c59b\") " pod="e2e-models-e2e-worker-w4-ef2572/distinct-2-w4-ef2572-kserve-547fccbd5-dfpfx"
Oct 31 00:48:17.689635 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:17.689603    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cb311a47-264c-4b33-bc31-377b94b0c59b-tls-certs\") pod \"distinct-2-w4-ef2572-kserve-547fccbd5-dfpfx\" (UID: \"cb311a47-264c-4b33-bc31-377b94b0c59b\") " pod="e2e-models-e2e-worker-w4-ef2572/distinct-2-w4-ef2572-kserve-547fccbd5-dfpfx"
Oct 31 00:48:17.698267 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:17.698180    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cb311a47-264c-4b33-bc31-377b94b0c59b-tmp-dir\") pod \"distinct-2-w4-ef2572-kserve-547fccbd5-dfpfx\" (UID: \"cb311a47-264c-4b33-bc31-377b94b0c59b\") " pod="e2e-models-e2e-worker-w4-ef2572/distinct-2-w4-ef2572-kserve-547fccbd5-dfpfx"
Oct 31 00:48:17.698514 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:17.698487    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2qczm\" (UniqueName: \"kubernetes.io/projected/cb311a47-264c-4b33-bc31-377b94b0c59b-kube-api-access-2qczm\") pod \"distinct-2-w4-ef2572-kserve-547fccbd5-dfpfx\" (UID: \"cb311a47-264c-4b33-bc31-377b94b0c59b\") " pod="e2e-models-e2e-worker-w4-ef2572/distinct-2-w4-ef2572-kserve-547fccbd5-dfpfx"
Oct 31 00:48:17.698751 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:17.698653    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cb311a47-264c-4b33-bc31-377b94b0c59b-home\") pod \"distinct-2-w4-ef2572-kserve-547fccbd5-dfpfx\" (UID: \"cb311a47-264c-4b33-bc31-377b94b0c59b\") " pod="e2e-models-e2e-worker-w4-ef2572/distinct-2-w4-ef2572-kserve-547fccbd5-dfpfx"
Oct 31 00:48:17.698847 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:17.698564    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cb311a47-264c-4b33-bc31-377b94b0c59b-tmp-dir\") pod \"distinct-2-w4-ef2572-kserve-547fccbd5-dfpfx\" (UID: \"cb311a47-264c-4b33-bc31-377b94b0c59b\") " pod="e2e-models-e2e-worker-w4-ef2572/distinct-2-w4-ef2572-kserve-547fccbd5-dfpfx"
Oct 31 00:48:17.698912 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:17.696019    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cb311a47-264c-4b33-bc31-377b94b0c59b-tls-certs\") pod \"distinct-2-w4-ef2572-kserve-547fccbd5-dfpfx\" (UID: \"cb311a47-264c-4b33-bc31-377b94b0c59b\") " pod="e2e-models-e2e-worker-w4-ef2572/distinct-2-w4-ef2572-kserve-547fccbd5-dfpfx"
Oct 31 00:48:17.698912 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:17.691371    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cb311a47-264c-4b33-bc31-377b94b0c59b-model-cache\") pod \"distinct-2-w4-ef2572-kserve-547fccbd5-dfpfx\" (UID: \"cb311a47-264c-4b33-bc31-377b94b0c59b\") " pod="e2e-models-e2e-worker-w4-ef2572/distinct-2-w4-ef2572-kserve-547fccbd5-dfpfx"
Oct 31 00:48:17.701163 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:17.699178    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cb311a47-264c-4b33-bc31-377b94b0c59b-home\") pod \"distinct-2-w4-ef2572-kserve-547fccbd5-dfpfx\" (UID: \"cb311a47-264c-4b33-bc31-377b94b0c59b\") " pod="e2e-models-e2e-worker-w4-ef2572/distinct-2-w4-ef2572-kserve-547fccbd5-dfpfx"
Oct 31 00:48:17.701163 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:17.701068    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cb311a47-264c-4b33-bc31-377b94b0c59b-dshm\") pod \"distinct-2-w4-ef2572-kserve-547fccbd5-dfpfx\" (UID: \"cb311a47-264c-4b33-bc31-377b94b0c59b\") " pod="e2e-models-e2e-worker-w4-ef2572/distinct-2-w4-ef2572-kserve-547fccbd5-dfpfx"
Oct 31 00:48:17.757570 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:17.757446    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2qczm\" (UniqueName: \"kubernetes.io/projected/cb311a47-264c-4b33-bc31-377b94b0c59b-kube-api-access-2qczm\") pod \"distinct-2-w4-ef2572-kserve-547fccbd5-dfpfx\" (UID: \"cb311a47-264c-4b33-bc31-377b94b0c59b\") " pod="e2e-models-e2e-worker-w4-ef2572/distinct-2-w4-ef2572-kserve-547fccbd5-dfpfx"
Oct 31 00:48:17.808602 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:17.808563    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-ef2572/distinct-2-w4-ef2572-kserve-547fccbd5-dfpfx"
Oct 31 00:48:18.275034 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:18.275006    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-ef2572/distinct-2-w4-ef2572-kserve-547fccbd5-dfpfx"]
Oct 31 00:48:18.276179 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:48:18.276155    4284 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podcb311a47_264c_4b33_bc31_377b94b0c59b.slice/crio-7894e2d9db6c518058e322229d3e0fcaa4635fd510e2db3ec114a7225ed67bdb WatchSource:0}: Error finding container 7894e2d9db6c518058e322229d3e0fcaa4635fd510e2db3ec114a7225ed67bdb: Status 404 returned error can't find the container with id 7894e2d9db6c518058e322229d3e0fcaa4635fd510e2db3ec114a7225ed67bdb
Oct 31 00:48:18.615936 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:18.615830    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-ef2572/distinct-2-w4-ef2572-kserve-547fccbd5-dfpfx" event={"ID":"cb311a47-264c-4b33-bc31-377b94b0c59b","Type":"ContainerStarted","Data":"14a3d41fce71267b2a444815c61ddebf71404ef6978a96a322698da59788d0fb"}
Oct 31 00:48:18.615936 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:18.615884    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-ef2572/distinct-2-w4-ef2572-kserve-547fccbd5-dfpfx" event={"ID":"cb311a47-264c-4b33-bc31-377b94b0c59b","Type":"ContainerStarted","Data":"7894e2d9db6c518058e322229d3e0fcaa4635fd510e2db3ec114a7225ed67bdb"}
Oct 31 00:48:18.731867 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:18.731816    4284 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w4-ef2572/distinct-2-w4-ef2572-kserve-547fccbd5-dfpfx" podStartSLOduration=1.731797845 podStartE2EDuration="1.731797845s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 00:48:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:48:18.729736845 +0000 UTC m=+1212.874743761" watchObservedRunningTime="2026-10-31 00:48:18.731797845 +0000 UTC m=+1212.876804767"
Oct 31 00:48:18.809127 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:18.809091    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-ef2572/distinct-2-w4-ef2572-kserve-547fccbd5-dfpfx"
Oct 31 00:48:18.901448 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:18.901226    4284 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-fnnxh"]
Oct 31 00:48:18.907312 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:18.907283    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-fnnxh"
Oct 31 00:48:18.988795 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:18.988757    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-fnnxh"]
Oct 31 00:48:19.036832 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:19.036787    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/55d97eb9-77d5-4b56-ab6d-5f38df8d1deb-credential-socket\") pod \"e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-fnnxh\" (UID: \"55d97eb9-77d5-4b56-ab6d-5f38df8d1deb\") " pod="openshift-ingress/e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-fnnxh"
Oct 31 00:48:19.037016 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:19.036853    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/55d97eb9-77d5-4b56-ab6d-5f38df8d1deb-istio-envoy\") pod \"e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-fnnxh\" (UID: \"55d97eb9-77d5-4b56-ab6d-5f38df8d1deb\") " pod="openshift-ingress/e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-fnnxh"
Oct 31 00:48:19.037016 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:19.036987    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/55d97eb9-77d5-4b56-ab6d-5f38df8d1deb-istio-podinfo\") pod \"e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-fnnxh\" (UID: \"55d97eb9-77d5-4b56-ab6d-5f38df8d1deb\") " pod="openshift-ingress/e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-fnnxh"
Oct 31 00:48:19.037129 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:19.037028    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/55d97eb9-77d5-4b56-ab6d-5f38df8d1deb-workload-socket\") pod \"e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-fnnxh\" (UID: \"55d97eb9-77d5-4b56-ab6d-5f38df8d1deb\") " pod="openshift-ingress/e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-fnnxh"
Oct 31 00:48:19.037129 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:19.037060    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/55d97eb9-77d5-4b56-ab6d-5f38df8d1deb-workload-certs\") pod \"e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-fnnxh\" (UID: \"55d97eb9-77d5-4b56-ab6d-5f38df8d1deb\") " pod="openshift-ingress/e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-fnnxh"
Oct 31 00:48:19.037274 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:19.037188    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/55d97eb9-77d5-4b56-ab6d-5f38df8d1deb-istio-data\") pod \"e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-fnnxh\" (UID: \"55d97eb9-77d5-4b56-ab6d-5f38df8d1deb\") " pod="openshift-ingress/e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-fnnxh"
Oct 31 00:48:19.037322 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:19.037290    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tcs2z\" (UniqueName: \"kubernetes.io/projected/55d97eb9-77d5-4b56-ab6d-5f38df8d1deb-kube-api-access-tcs2z\") pod \"e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-fnnxh\" (UID: \"55d97eb9-77d5-4b56-ab6d-5f38df8d1deb\") " pod="openshift-ingress/e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-fnnxh"
Oct 31 00:48:19.037322 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:19.037313    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/55d97eb9-77d5-4b56-ab6d-5f38df8d1deb-istiod-ca-cert\") pod \"e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-fnnxh\" (UID: \"55d97eb9-77d5-4b56-ab6d-5f38df8d1deb\") " pod="openshift-ingress/e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-fnnxh"
Oct 31 00:48:19.037422 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:19.037332    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/55d97eb9-77d5-4b56-ab6d-5f38df8d1deb-istio-token\") pod \"e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-fnnxh\" (UID: \"55d97eb9-77d5-4b56-ab6d-5f38df8d1deb\") " pod="openshift-ingress/e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-fnnxh"
Oct 31 00:48:19.138604 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:19.138557    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/55d97eb9-77d5-4b56-ab6d-5f38df8d1deb-istio-data\") pod \"e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-fnnxh\" (UID: \"55d97eb9-77d5-4b56-ab6d-5f38df8d1deb\") " pod="openshift-ingress/e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-fnnxh"
Oct 31 00:48:19.138806 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:19.138634    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tcs2z\" (UniqueName: \"kubernetes.io/projected/55d97eb9-77d5-4b56-ab6d-5f38df8d1deb-kube-api-access-tcs2z\") pod \"e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-fnnxh\" (UID: \"55d97eb9-77d5-4b56-ab6d-5f38df8d1deb\") " pod="openshift-ingress/e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-fnnxh"
Oct 31 00:48:19.138806 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:19.138668    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/55d97eb9-77d5-4b56-ab6d-5f38df8d1deb-istiod-ca-cert\") pod \"e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-fnnxh\" (UID: \"55d97eb9-77d5-4b56-ab6d-5f38df8d1deb\") " pod="openshift-ingress/e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-fnnxh"
Oct 31 00:48:19.138806 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:19.138694    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/55d97eb9-77d5-4b56-ab6d-5f38df8d1deb-istio-token\") pod \"e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-fnnxh\" (UID: \"55d97eb9-77d5-4b56-ab6d-5f38df8d1deb\") " pod="openshift-ingress/e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-fnnxh"
Oct 31 00:48:19.138806 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:19.138778    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/55d97eb9-77d5-4b56-ab6d-5f38df8d1deb-credential-socket\") pod \"e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-fnnxh\" (UID: \"55d97eb9-77d5-4b56-ab6d-5f38df8d1deb\") " pod="openshift-ingress/e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-fnnxh"
Oct 31 00:48:19.139040 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:19.138826    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/55d97eb9-77d5-4b56-ab6d-5f38df8d1deb-istio-envoy\") pod \"e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-fnnxh\" (UID: \"55d97eb9-77d5-4b56-ab6d-5f38df8d1deb\") " pod="openshift-ingress/e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-fnnxh"
Oct 31 00:48:19.139040 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:19.138901    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/55d97eb9-77d5-4b56-ab6d-5f38df8d1deb-istio-podinfo\") pod \"e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-fnnxh\" (UID: \"55d97eb9-77d5-4b56-ab6d-5f38df8d1deb\") " pod="openshift-ingress/e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-fnnxh"
Oct 31 00:48:19.139040 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:19.138939    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/55d97eb9-77d5-4b56-ab6d-5f38df8d1deb-workload-socket\") pod \"e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-fnnxh\" (UID: \"55d97eb9-77d5-4b56-ab6d-5f38df8d1deb\") " pod="openshift-ingress/e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-fnnxh"
Oct 31 00:48:19.139040 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:19.138968    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/55d97eb9-77d5-4b56-ab6d-5f38df8d1deb-workload-certs\") pod \"e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-fnnxh\" (UID: \"55d97eb9-77d5-4b56-ab6d-5f38df8d1deb\") " pod="openshift-ingress/e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-fnnxh"
Oct 31 00:48:19.139265 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:19.139051    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/55d97eb9-77d5-4b56-ab6d-5f38df8d1deb-istio-data\") pod \"e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-fnnxh\" (UID: \"55d97eb9-77d5-4b56-ab6d-5f38df8d1deb\") " pod="openshift-ingress/e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-fnnxh"
Oct 31 00:48:19.139265 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:19.139070    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/55d97eb9-77d5-4b56-ab6d-5f38df8d1deb-credential-socket\") pod \"e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-fnnxh\" (UID: \"55d97eb9-77d5-4b56-ab6d-5f38df8d1deb\") " pod="openshift-ingress/e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-fnnxh"
Oct 31 00:48:19.139373 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:19.139330    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/55d97eb9-77d5-4b56-ab6d-5f38df8d1deb-workload-certs\") pod \"e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-fnnxh\" (UID: \"55d97eb9-77d5-4b56-ab6d-5f38df8d1deb\") " pod="openshift-ingress/e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-fnnxh"
Oct 31 00:48:19.139464 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:19.139438    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/55d97eb9-77d5-4b56-ab6d-5f38df8d1deb-workload-socket\") pod \"e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-fnnxh\" (UID: \"55d97eb9-77d5-4b56-ab6d-5f38df8d1deb\") " pod="openshift-ingress/e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-fnnxh"
Oct 31 00:48:19.139801 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:19.139782    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/55d97eb9-77d5-4b56-ab6d-5f38df8d1deb-istiod-ca-cert\") pod \"e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-fnnxh\" (UID: \"55d97eb9-77d5-4b56-ab6d-5f38df8d1deb\") " pod="openshift-ingress/e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-fnnxh"
Oct 31 00:48:19.141667 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:19.141639    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/55d97eb9-77d5-4b56-ab6d-5f38df8d1deb-istio-envoy\") pod \"e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-fnnxh\" (UID: \"55d97eb9-77d5-4b56-ab6d-5f38df8d1deb\") " pod="openshift-ingress/e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-fnnxh"
Oct 31 00:48:19.141843 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:19.141822    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/55d97eb9-77d5-4b56-ab6d-5f38df8d1deb-istio-podinfo\") pod \"e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-fnnxh\" (UID: \"55d97eb9-77d5-4b56-ab6d-5f38df8d1deb\") " pod="openshift-ingress/e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-fnnxh"
Oct 31 00:48:19.198183 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:19.198147    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/55d97eb9-77d5-4b56-ab6d-5f38df8d1deb-istio-token\") pod \"e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-fnnxh\" (UID: \"55d97eb9-77d5-4b56-ab6d-5f38df8d1deb\") " pod="openshift-ingress/e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-fnnxh"
Oct 31 00:48:19.198382 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:19.198287    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tcs2z\" (UniqueName: \"kubernetes.io/projected/55d97eb9-77d5-4b56-ab6d-5f38df8d1deb-kube-api-access-tcs2z\") pod \"e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-fnnxh\" (UID: \"55d97eb9-77d5-4b56-ab6d-5f38df8d1deb\") " pod="openshift-ingress/e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-fnnxh"
Oct 31 00:48:19.221912 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:19.221886    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-fnnxh"
Oct 31 00:48:19.641715 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:19.641688    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-fnnxh"]
Oct 31 00:48:19.642665 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:48:19.642636    4284 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod55d97eb9_77d5_4b56_ab6d_5f38df8d1deb.slice/crio-e0d370dec52bea0fcdae88cdf18b1c7968191a889668d4c510a58ca2469f7327 WatchSource:0}: Error finding container e0d370dec52bea0fcdae88cdf18b1c7968191a889668d4c510a58ca2469f7327: Status 404 returned error can't find the container with id e0d370dec52bea0fcdae88cdf18b1c7968191a889668d4c510a58ca2469f7327
Oct 31 00:48:19.645306 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:19.645274    4284 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 00:48:19.645384 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:19.645340    4284 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 00:48:19.645384 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:19.645367    4284 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 00:48:20.610050 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:20.610006    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:48:20.670331 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:20.670293    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-fnnxh" event={"ID":"55d97eb9-77d5-4b56-ab6d-5f38df8d1deb","Type":"ContainerStarted","Data":"6a9eaf70cfdeeb456778b08b524ea4131bb75313e5c24c3ece3e1836171e1cf1"}
Oct 31 00:48:20.670793 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:20.670383    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-fnnxh" event={"ID":"55d97eb9-77d5-4b56-ab6d-5f38df8d1deb","Type":"ContainerStarted","Data":"e0d370dec52bea0fcdae88cdf18b1c7968191a889668d4c510a58ca2469f7327"}
Oct 31 00:48:20.837716 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:20.837652    4284 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-fnnxh" podStartSLOduration=2.837634107 podStartE2EDuration="2.837634107s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 00:48:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:48:20.834988795 +0000 UTC m=+1214.979995711" watchObservedRunningTime="2026-10-31 00:48:20.837634107 +0000 UTC m=+1214.982641023"
Oct 31 00:48:21.223097 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:21.223058    4284 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-fnnxh"
Oct 31 00:48:21.225140 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:21.225112    4284 patch_prober.go:29] interesting pod/e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-fnnxh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused" start-of-body=
Oct 31 00:48:21.225283 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:21.225163    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-fnnxh" podUID="55d97eb9-77d5-4b56-ab6d-5f38df8d1deb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused"
Oct 31 00:48:22.222684 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:22.222644    4284 patch_prober.go:29] interesting pod/e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-fnnxh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused" start-of-body=
Oct 31 00:48:22.223095 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:22.222706    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-fnnxh" podUID="55d97eb9-77d5-4b56-ab6d-5f38df8d1deb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused"
Oct 31 00:48:23.222876 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:23.222841    4284 patch_prober.go:29] interesting pod/e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-fnnxh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused" start-of-body=
Oct 31 00:48:23.223316 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:23.222891    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-fnnxh" podUID="55d97eb9-77d5-4b56-ab6d-5f38df8d1deb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused"
Oct 31 00:48:24.222448 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:24.222408    4284 patch_prober.go:29] interesting pod/e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-fnnxh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused" start-of-body=
Oct 31 00:48:24.222654 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:24.222475    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-fnnxh" podUID="55d97eb9-77d5-4b56-ab6d-5f38df8d1deb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused"
Oct 31 00:48:25.223089 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:25.223051    4284 patch_prober.go:29] interesting pod/e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-fnnxh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused" start-of-body=
Oct 31 00:48:25.223560 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:25.223122    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-fnnxh" podUID="55d97eb9-77d5-4b56-ab6d-5f38df8d1deb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused"
Oct 31 00:48:25.431684 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:25.431633    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:48:26.226760 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:26.226725    4284 patch_prober.go:29] interesting pod/e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-fnnxh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused" start-of-body=
Oct 31 00:48:26.227449 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:26.227402    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-fnnxh" podUID="55d97eb9-77d5-4b56-ab6d-5f38df8d1deb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused"
Oct 31 00:48:27.223124 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:27.223091    4284 patch_prober.go:29] interesting pod/e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-fnnxh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused" start-of-body=
Oct 31 00:48:27.223485 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:27.223433    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-fnnxh" podUID="55d97eb9-77d5-4b56-ab6d-5f38df8d1deb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused"
Oct 31 00:48:27.813240 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:27.811365    4284 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w4-ef2572/distinct-2-w4-ef2572-kserve-547fccbd5-dfpfx"
Oct 31 00:48:28.132755 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:28.132650    4284 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w4-ef2572/distinct-2-w4-ef2572-kserve-547fccbd5-dfpfx"
Oct 31 00:48:28.208389 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:28.208345    4284 patch_prober.go:29] interesting pod/e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-wqns9 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.60:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 31 00:48:28.208697 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:28.208669    4284 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-wqns9" podUID="fc082395-6c5f-4a62-8d0a-1e7d5d562c12" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.60:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:48:28.865621 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:28.865581    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-ef2572/distinct-2-w4-ef2572-kserve-547fccbd5-dfpfx"
Oct 31 00:48:29.266358 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:29.266316    4284 patch_prober.go:29] interesting pod/e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-fnnxh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.74:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 31 00:48:29.266528 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:29.266375    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-fnnxh" podUID="55d97eb9-77d5-4b56-ab6d-5f38df8d1deb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.74:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:48:29.384328 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:29.384263    4284 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-fnnxh"
Oct 31 00:48:29.385457 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:29.385433    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-fnnxh"
Oct 31 00:48:29.386866 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:29.386840    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-fnnxh"
Oct 31 00:48:31.197258 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:31.197196    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:48:31.346596 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:31.346562    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:48:31.499969 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:31.499867    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:48:36.196632 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:36.196597    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:48:36.868857 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:36.868825    4284 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-2sg7k"]
Oct 31 00:48:36.872357 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:36.872338    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-2sg7k"
Oct 31 00:48:36.882824 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:36.882794    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a973c5a8-c25c-41ef-b2d7-7289fdc66b42-workload-socket\") pod \"e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-2sg7k\" (UID: \"a973c5a8-c25c-41ef-b2d7-7289fdc66b42\") " pod="openshift-ingress/e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-2sg7k"
Oct 31 00:48:36.882930 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:36.882854    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4z647\" (UniqueName: \"kubernetes.io/projected/a973c5a8-c25c-41ef-b2d7-7289fdc66b42-kube-api-access-4z647\") pod \"e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-2sg7k\" (UID: \"a973c5a8-c25c-41ef-b2d7-7289fdc66b42\") " pod="openshift-ingress/e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-2sg7k"
Oct 31 00:48:36.882991 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:36.882968    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a973c5a8-c25c-41ef-b2d7-7289fdc66b42-istio-token\") pod \"e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-2sg7k\" (UID: \"a973c5a8-c25c-41ef-b2d7-7289fdc66b42\") " pod="openshift-ingress/e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-2sg7k"
Oct 31 00:48:36.883033 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:36.883008    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a973c5a8-c25c-41ef-b2d7-7289fdc66b42-istio-podinfo\") pod \"e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-2sg7k\" (UID: \"a973c5a8-c25c-41ef-b2d7-7289fdc66b42\") " pod="openshift-ingress/e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-2sg7k"
Oct 31 00:48:36.883130 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:36.883118    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a973c5a8-c25c-41ef-b2d7-7289fdc66b42-istio-envoy\") pod \"e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-2sg7k\" (UID: \"a973c5a8-c25c-41ef-b2d7-7289fdc66b42\") " pod="openshift-ingress/e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-2sg7k"
Oct 31 00:48:36.883164 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:36.883142    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a973c5a8-c25c-41ef-b2d7-7289fdc66b42-workload-certs\") pod \"e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-2sg7k\" (UID: \"a973c5a8-c25c-41ef-b2d7-7289fdc66b42\") " pod="openshift-ingress/e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-2sg7k"
Oct 31 00:48:36.883205 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:36.883161    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a973c5a8-c25c-41ef-b2d7-7289fdc66b42-istio-data\") pod \"e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-2sg7k\" (UID: \"a973c5a8-c25c-41ef-b2d7-7289fdc66b42\") " pod="openshift-ingress/e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-2sg7k"
Oct 31 00:48:36.883205 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:36.883178    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a973c5a8-c25c-41ef-b2d7-7289fdc66b42-credential-socket\") pod \"e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-2sg7k\" (UID: \"a973c5a8-c25c-41ef-b2d7-7289fdc66b42\") " pod="openshift-ingress/e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-2sg7k"
Oct 31 00:48:36.883348 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:36.883329    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a973c5a8-c25c-41ef-b2d7-7289fdc66b42-istiod-ca-cert\") pod \"e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-2sg7k\" (UID: \"a973c5a8-c25c-41ef-b2d7-7289fdc66b42\") " pod="openshift-ingress/e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-2sg7k"
Oct 31 00:48:36.973395 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:36.973361    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-2sg7k"]
Oct 31 00:48:36.984167 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:36.984136    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a973c5a8-c25c-41ef-b2d7-7289fdc66b42-workload-socket\") pod \"e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-2sg7k\" (UID: \"a973c5a8-c25c-41ef-b2d7-7289fdc66b42\") " pod="openshift-ingress/e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-2sg7k"
Oct 31 00:48:36.984400 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:36.984180    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4z647\" (UniqueName: \"kubernetes.io/projected/a973c5a8-c25c-41ef-b2d7-7289fdc66b42-kube-api-access-4z647\") pod \"e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-2sg7k\" (UID: \"a973c5a8-c25c-41ef-b2d7-7289fdc66b42\") " pod="openshift-ingress/e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-2sg7k"
Oct 31 00:48:36.984400 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:36.984265    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a973c5a8-c25c-41ef-b2d7-7289fdc66b42-istio-token\") pod \"e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-2sg7k\" (UID: \"a973c5a8-c25c-41ef-b2d7-7289fdc66b42\") " pod="openshift-ingress/e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-2sg7k"
Oct 31 00:48:36.984400 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:36.984297    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a973c5a8-c25c-41ef-b2d7-7289fdc66b42-istio-podinfo\") pod \"e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-2sg7k\" (UID: \"a973c5a8-c25c-41ef-b2d7-7289fdc66b42\") " pod="openshift-ingress/e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-2sg7k"
Oct 31 00:48:36.984569 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:36.984437    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a973c5a8-c25c-41ef-b2d7-7289fdc66b42-istio-envoy\") pod \"e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-2sg7k\" (UID: \"a973c5a8-c25c-41ef-b2d7-7289fdc66b42\") " pod="openshift-ingress/e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-2sg7k"
Oct 31 00:48:36.984569 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:36.984473    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a973c5a8-c25c-41ef-b2d7-7289fdc66b42-workload-certs\") pod \"e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-2sg7k\" (UID: \"a973c5a8-c25c-41ef-b2d7-7289fdc66b42\") " pod="openshift-ingress/e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-2sg7k"
Oct 31 00:48:36.984569 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:36.984504    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a973c5a8-c25c-41ef-b2d7-7289fdc66b42-istio-data\") pod \"e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-2sg7k\" (UID: \"a973c5a8-c25c-41ef-b2d7-7289fdc66b42\") " pod="openshift-ingress/e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-2sg7k"
Oct 31 00:48:36.984569 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:36.984531    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a973c5a8-c25c-41ef-b2d7-7289fdc66b42-credential-socket\") pod \"e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-2sg7k\" (UID: \"a973c5a8-c25c-41ef-b2d7-7289fdc66b42\") " pod="openshift-ingress/e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-2sg7k"
Oct 31 00:48:36.984811 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:36.984594    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a973c5a8-c25c-41ef-b2d7-7289fdc66b42-istiod-ca-cert\") pod \"e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-2sg7k\" (UID: \"a973c5a8-c25c-41ef-b2d7-7289fdc66b42\") " pod="openshift-ingress/e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-2sg7k"
Oct 31 00:48:36.984811 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:36.984614    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a973c5a8-c25c-41ef-b2d7-7289fdc66b42-workload-socket\") pod \"e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-2sg7k\" (UID: \"a973c5a8-c25c-41ef-b2d7-7289fdc66b42\") " pod="openshift-ingress/e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-2sg7k"
Oct 31 00:48:36.985331 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:36.985302    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a973c5a8-c25c-41ef-b2d7-7289fdc66b42-istiod-ca-cert\") pod \"e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-2sg7k\" (UID: \"a973c5a8-c25c-41ef-b2d7-7289fdc66b42\") " pod="openshift-ingress/e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-2sg7k"
Oct 31 00:48:36.985519 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:36.985310    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a973c5a8-c25c-41ef-b2d7-7289fdc66b42-istio-data\") pod \"e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-2sg7k\" (UID: \"a973c5a8-c25c-41ef-b2d7-7289fdc66b42\") " pod="openshift-ingress/e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-2sg7k"
Oct 31 00:48:36.985519 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:36.985407    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a973c5a8-c25c-41ef-b2d7-7289fdc66b42-workload-certs\") pod \"e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-2sg7k\" (UID: \"a973c5a8-c25c-41ef-b2d7-7289fdc66b42\") " pod="openshift-ingress/e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-2sg7k"
Oct 31 00:48:36.985662 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:36.985529    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a973c5a8-c25c-41ef-b2d7-7289fdc66b42-credential-socket\") pod \"e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-2sg7k\" (UID: \"a973c5a8-c25c-41ef-b2d7-7289fdc66b42\") " pod="openshift-ingress/e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-2sg7k"
Oct 31 00:48:36.988531 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:36.988502    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a973c5a8-c25c-41ef-b2d7-7289fdc66b42-istio-envoy\") pod \"e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-2sg7k\" (UID: \"a973c5a8-c25c-41ef-b2d7-7289fdc66b42\") " pod="openshift-ingress/e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-2sg7k"
Oct 31 00:48:36.991609 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:36.991586    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a973c5a8-c25c-41ef-b2d7-7289fdc66b42-istio-podinfo\") pod \"e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-2sg7k\" (UID: \"a973c5a8-c25c-41ef-b2d7-7289fdc66b42\") " pod="openshift-ingress/e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-2sg7k"
Oct 31 00:48:37.043556 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:37.043522    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a973c5a8-c25c-41ef-b2d7-7289fdc66b42-istio-token\") pod \"e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-2sg7k\" (UID: \"a973c5a8-c25c-41ef-b2d7-7289fdc66b42\") " pod="openshift-ingress/e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-2sg7k"
Oct 31 00:48:37.043813 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:37.043787    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4z647\" (UniqueName: \"kubernetes.io/projected/a973c5a8-c25c-41ef-b2d7-7289fdc66b42-kube-api-access-4z647\") pod \"e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-2sg7k\" (UID: \"a973c5a8-c25c-41ef-b2d7-7289fdc66b42\") " pod="openshift-ingress/e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-2sg7k"
Oct 31 00:48:37.186081 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:37.186044    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-2sg7k"
Oct 31 00:48:37.615108 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:37.615078    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-2sg7k"]
Oct 31 00:48:37.617517 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:48:37.617487    4284 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda973c5a8_c25c_41ef_b2d7_7289fdc66b42.slice/crio-d4c34fdfa4a7e52d2b3d4fc2513b7a8dfb863fe6a9c31ae86c9361458c0ae369 WatchSource:0}: Error finding container d4c34fdfa4a7e52d2b3d4fc2513b7a8dfb863fe6a9c31ae86c9361458c0ae369: Status 404 returned error can't find the container with id d4c34fdfa4a7e52d2b3d4fc2513b7a8dfb863fe6a9c31ae86c9361458c0ae369
Oct 31 00:48:37.619746 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:37.619714    4284 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 00:48:37.619814 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:37.619785    4284 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 00:48:37.619853 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:37.619814    4284 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 00:48:38.000395 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:38.000354    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-2sg7k" event={"ID":"a973c5a8-c25c-41ef-b2d7-7289fdc66b42","Type":"ContainerStarted","Data":"e9a2438f11440d23d8677436ffa1b3b02d67819e6942a1ea71860848383d9b95"}
Oct 31 00:48:38.000566 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:38.000414    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-2sg7k" event={"ID":"a973c5a8-c25c-41ef-b2d7-7289fdc66b42","Type":"ContainerStarted","Data":"d4c34fdfa4a7e52d2b3d4fc2513b7a8dfb863fe6a9c31ae86c9361458c0ae369"}
Oct 31 00:48:38.185609 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:38.185543    4284 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-2sg7k" podStartSLOduration=2.185520824 podStartE2EDuration="2.185520824s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 00:48:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:48:38.163690702 +0000 UTC m=+1232.308697632" watchObservedRunningTime="2026-10-31 00:48:38.185520824 +0000 UTC m=+1232.330527741"
Oct 31 00:48:38.192430 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:38.186377    4284 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-2sg7k"
Oct 31 00:48:38.192735 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:38.188972    4284 patch_prober.go:29] interesting pod/e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-2sg7k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused" start-of-body=
Oct 31 00:48:38.192895 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:38.192872    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-2sg7k" podUID="a973c5a8-c25c-41ef-b2d7-7289fdc66b42" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused"
Oct 31 00:48:39.195247 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:39.194632    4284 patch_prober.go:29] interesting pod/e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-2sg7k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused" start-of-body=
Oct 31 00:48:39.195247 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:39.194692    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-2sg7k" podUID="a973c5a8-c25c-41ef-b2d7-7289fdc66b42" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused"
Oct 31 00:48:40.206240 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:40.198632    4284 patch_prober.go:29] interesting pod/e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-2sg7k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused" start-of-body=
Oct 31 00:48:40.206240 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:40.198695    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-2sg7k" podUID="a973c5a8-c25c-41ef-b2d7-7289fdc66b42" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused"
Oct 31 00:48:41.041387 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:41.041346    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:48:41.191010 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:41.190972    4284 patch_prober.go:29] interesting pod/e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-2sg7k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused" start-of-body=
Oct 31 00:48:41.191321 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:41.191289    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-2sg7k" podUID="a973c5a8-c25c-41ef-b2d7-7289fdc66b42" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused"
Oct 31 00:48:42.203239 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:42.200662    4284 patch_prober.go:29] interesting pod/e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-2sg7k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused" start-of-body=
Oct 31 00:48:42.203239 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:42.200765    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-2sg7k" podUID="a973c5a8-c25c-41ef-b2d7-7289fdc66b42" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused"
Oct 31 00:48:43.066092 ip-10-0-132-173 kubenswrapper[4284]: E1031 00:48:43.066056    4284 projected.go:266] Couldn't get secret models-as-a-service/agc-cred-001239fd95f6: secret "agc-cred-001239fd95f6" not found
Oct 31 00:48:43.066325 ip-10-0-132-173 kubenswrapper[4284]: E1031 00:48:43.066283    4284 projected.go:266] Couldn't get secret models-as-a-service/agc-cred-da8c34345bbb: secret "agc-cred-da8c34345bbb" not found
Oct 31 00:48:43.066325 ip-10-0-132-173 kubenswrapper[4284]: E1031 00:48:43.066304    4284 projected.go:196] Error preparing data for projected volume provider-credentials for pod models-as-a-service/payload-processing-external-model-84cf56844-6pbmt: [secret "agc-cred-001239fd95f6" not found, secret "agc-cred-da8c34345bbb" not found]
Oct 31 00:48:43.066453 ip-10-0-132-173 kubenswrapper[4284]: E1031 00:48:43.066369    4284 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e78a1e6c-10b0-4faf-b773-ffd07127b046-provider-credentials podName:e78a1e6c-10b0-4faf-b773-ffd07127b046 nodeName:}" failed. No retries permitted until 2026-10-31 00:48:43.566349329 +0000 UTC m=+1237.711356223 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "provider-credentials" (UniqueName: "kubernetes.io/projected/e78a1e6c-10b0-4faf-b773-ffd07127b046-provider-credentials") pod "payload-processing-external-model-84cf56844-6pbmt" (UID: "e78a1e6c-10b0-4faf-b773-ffd07127b046") : [secret "agc-cred-001239fd95f6" not found, secret "agc-cred-da8c34345bbb" not found]
Oct 31 00:48:43.201238 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:43.190614    4284 patch_prober.go:29] interesting pod/e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-2sg7k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused" start-of-body=
Oct 31 00:48:43.201238 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:43.190691    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-2sg7k" podUID="a973c5a8-c25c-41ef-b2d7-7289fdc66b42" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused"
Oct 31 00:48:43.572882 ip-10-0-132-173 kubenswrapper[4284]: E1031 00:48:43.572840    4284 projected.go:266] Couldn't get secret models-as-a-service/agc-cred-001239fd95f6: secret "agc-cred-001239fd95f6" not found
Oct 31 00:48:43.572882 ip-10-0-132-173 kubenswrapper[4284]: E1031 00:48:43.572877    4284 projected.go:266] Couldn't get secret models-as-a-service/agc-cred-da8c34345bbb: secret "agc-cred-da8c34345bbb" not found
Oct 31 00:48:43.572882 ip-10-0-132-173 kubenswrapper[4284]: E1031 00:48:43.572886    4284 projected.go:196] Error preparing data for projected volume provider-credentials for pod models-as-a-service/payload-processing-external-model-84cf56844-6pbmt: [secret "agc-cred-001239fd95f6" not found, secret "agc-cred-da8c34345bbb" not found]
Oct 31 00:48:43.573386 ip-10-0-132-173 kubenswrapper[4284]: E1031 00:48:43.572942    4284 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e78a1e6c-10b0-4faf-b773-ffd07127b046-provider-credentials podName:e78a1e6c-10b0-4faf-b773-ffd07127b046 nodeName:}" failed. No retries permitted until 2026-10-31 00:48:44.572927645 +0000 UTC m=+1238.717934538 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "provider-credentials" (UniqueName: "kubernetes.io/projected/e78a1e6c-10b0-4faf-b773-ffd07127b046-provider-credentials") pod "payload-processing-external-model-84cf56844-6pbmt" (UID: "e78a1e6c-10b0-4faf-b773-ffd07127b046") : [secret "agc-cred-001239fd95f6" not found, secret "agc-cred-da8c34345bbb" not found]
Oct 31 00:48:44.186448 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:44.186407    4284 patch_prober.go:29] interesting pod/e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-2sg7k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused" start-of-body=
Oct 31 00:48:44.186636 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:44.186480    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-2sg7k" podUID="a973c5a8-c25c-41ef-b2d7-7289fdc66b42" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused"
Oct 31 00:48:44.586085 ip-10-0-132-173 kubenswrapper[4284]: E1031 00:48:44.585978    4284 projected.go:266] Couldn't get secret models-as-a-service/agc-cred-001239fd95f6: secret "agc-cred-001239fd95f6" not found
Oct 31 00:48:44.586085 ip-10-0-132-173 kubenswrapper[4284]: E1031 00:48:44.586012    4284 projected.go:266] Couldn't get secret models-as-a-service/agc-cred-da8c34345bbb: secret "agc-cred-da8c34345bbb" not found
Oct 31 00:48:44.586085 ip-10-0-132-173 kubenswrapper[4284]: E1031 00:48:44.586022    4284 projected.go:196] Error preparing data for projected volume provider-credentials for pod models-as-a-service/payload-processing-external-model-84cf56844-6pbmt: [secret "agc-cred-001239fd95f6" not found, secret "agc-cred-da8c34345bbb" not found]
Oct 31 00:48:44.586085 ip-10-0-132-173 kubenswrapper[4284]: E1031 00:48:44.586088    4284 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e78a1e6c-10b0-4faf-b773-ffd07127b046-provider-credentials podName:e78a1e6c-10b0-4faf-b773-ffd07127b046 nodeName:}" failed. No retries permitted until 2026-10-31 00:48:46.586072378 +0000 UTC m=+1240.731079279 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "provider-credentials" (UniqueName: "kubernetes.io/projected/e78a1e6c-10b0-4faf-b773-ffd07127b046-provider-credentials") pod "payload-processing-external-model-84cf56844-6pbmt" (UID: "e78a1e6c-10b0-4faf-b773-ffd07127b046") : [secret "agc-cred-001239fd95f6" not found, secret "agc-cred-da8c34345bbb" not found]
Oct 31 00:48:45.026310 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:45.012888    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:48:45.036604 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:45.036555    4284 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-lc4wb"]
Oct 31 00:48:45.073834 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:45.072848    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-lc4wb"
Oct 31 00:48:45.144695 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:45.142553    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-lc4wb"]
Oct 31 00:48:45.187371 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:45.184198    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:48:45.220004 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:45.219973    4284 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-2sg7k"
Oct 31 00:48:45.221670 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:45.221644    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-2sg7k"
Oct 31 00:48:45.222860 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:45.222843    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-2sg7k"
Oct 31 00:48:45.262809 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:45.262768    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4r9sf\" (UniqueName: \"kubernetes.io/projected/cdded00d-879b-491a-aa0b-8736512b0907-kube-api-access-4r9sf\") pod \"e2e-worker-w4-ef2572-openshift-default-7649dd87-lc4wb\" (UID: \"cdded00d-879b-491a-aa0b-8736512b0907\") " pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-lc4wb"
Oct 31 00:48:45.263121 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:45.263089    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cdded00d-879b-491a-aa0b-8736512b0907-istio-envoy\") pod \"e2e-worker-w4-ef2572-openshift-default-7649dd87-lc4wb\" (UID: \"cdded00d-879b-491a-aa0b-8736512b0907\") " pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-lc4wb"
Oct 31 00:48:45.263292 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:45.263138    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cdded00d-879b-491a-aa0b-8736512b0907-credential-socket\") pod \"e2e-worker-w4-ef2572-openshift-default-7649dd87-lc4wb\" (UID: \"cdded00d-879b-491a-aa0b-8736512b0907\") " pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-lc4wb"
Oct 31 00:48:45.263292 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:45.263193    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cdded00d-879b-491a-aa0b-8736512b0907-istio-data\") pod \"e2e-worker-w4-ef2572-openshift-default-7649dd87-lc4wb\" (UID: \"cdded00d-879b-491a-aa0b-8736512b0907\") " pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-lc4wb"
Oct 31 00:48:45.263292 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:45.263235    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cdded00d-879b-491a-aa0b-8736512b0907-istio-podinfo\") pod \"e2e-worker-w4-ef2572-openshift-default-7649dd87-lc4wb\" (UID: \"cdded00d-879b-491a-aa0b-8736512b0907\") " pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-lc4wb"
Oct 31 00:48:45.263455 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:45.263339    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cdded00d-879b-491a-aa0b-8736512b0907-istio-token\") pod \"e2e-worker-w4-ef2572-openshift-default-7649dd87-lc4wb\" (UID: \"cdded00d-879b-491a-aa0b-8736512b0907\") " pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-lc4wb"
Oct 31 00:48:45.263517 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:45.263499    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cdded00d-879b-491a-aa0b-8736512b0907-workload-socket\") pod \"e2e-worker-w4-ef2572-openshift-default-7649dd87-lc4wb\" (UID: \"cdded00d-879b-491a-aa0b-8736512b0907\") " pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-lc4wb"
Oct 31 00:48:45.263577 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:45.263542    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cdded00d-879b-491a-aa0b-8736512b0907-workload-certs\") pod \"e2e-worker-w4-ef2572-openshift-default-7649dd87-lc4wb\" (UID: \"cdded00d-879b-491a-aa0b-8736512b0907\") " pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-lc4wb"
Oct 31 00:48:45.263577 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:45.263569    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cdded00d-879b-491a-aa0b-8736512b0907-istiod-ca-cert\") pod \"e2e-worker-w4-ef2572-openshift-default-7649dd87-lc4wb\" (UID: \"cdded00d-879b-491a-aa0b-8736512b0907\") " pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-lc4wb"
Oct 31 00:48:45.365384 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:45.365281    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cdded00d-879b-491a-aa0b-8736512b0907-istio-data\") pod \"e2e-worker-w4-ef2572-openshift-default-7649dd87-lc4wb\" (UID: \"cdded00d-879b-491a-aa0b-8736512b0907\") " pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-lc4wb"
Oct 31 00:48:45.365384 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:45.365325    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cdded00d-879b-491a-aa0b-8736512b0907-istio-podinfo\") pod \"e2e-worker-w4-ef2572-openshift-default-7649dd87-lc4wb\" (UID: \"cdded00d-879b-491a-aa0b-8736512b0907\") " pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-lc4wb"
Oct 31 00:48:45.365635 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:45.365425    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cdded00d-879b-491a-aa0b-8736512b0907-istio-token\") pod \"e2e-worker-w4-ef2572-openshift-default-7649dd87-lc4wb\" (UID: \"cdded00d-879b-491a-aa0b-8736512b0907\") " pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-lc4wb"
Oct 31 00:48:45.365635 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:45.365555    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cdded00d-879b-491a-aa0b-8736512b0907-workload-socket\") pod \"e2e-worker-w4-ef2572-openshift-default-7649dd87-lc4wb\" (UID: \"cdded00d-879b-491a-aa0b-8736512b0907\") " pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-lc4wb"
Oct 31 00:48:45.365635 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:45.365591    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cdded00d-879b-491a-aa0b-8736512b0907-workload-certs\") pod \"e2e-worker-w4-ef2572-openshift-default-7649dd87-lc4wb\" (UID: \"cdded00d-879b-491a-aa0b-8736512b0907\") " pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-lc4wb"
Oct 31 00:48:45.365635 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:45.365615    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cdded00d-879b-491a-aa0b-8736512b0907-istiod-ca-cert\") pod \"e2e-worker-w4-ef2572-openshift-default-7649dd87-lc4wb\" (UID: \"cdded00d-879b-491a-aa0b-8736512b0907\") " pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-lc4wb"
Oct 31 00:48:45.365862 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:45.365732    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4r9sf\" (UniqueName: \"kubernetes.io/projected/cdded00d-879b-491a-aa0b-8736512b0907-kube-api-access-4r9sf\") pod \"e2e-worker-w4-ef2572-openshift-default-7649dd87-lc4wb\" (UID: \"cdded00d-879b-491a-aa0b-8736512b0907\") " pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-lc4wb"
Oct 31 00:48:45.365862 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:45.365730    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cdded00d-879b-491a-aa0b-8736512b0907-istio-data\") pod \"e2e-worker-w4-ef2572-openshift-default-7649dd87-lc4wb\" (UID: \"cdded00d-879b-491a-aa0b-8736512b0907\") " pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-lc4wb"
Oct 31 00:48:45.366035 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:45.366017    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cdded00d-879b-491a-aa0b-8736512b0907-workload-socket\") pod \"e2e-worker-w4-ef2572-openshift-default-7649dd87-lc4wb\" (UID: \"cdded00d-879b-491a-aa0b-8736512b0907\") " pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-lc4wb"
Oct 31 00:48:45.366207 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:45.366178    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cdded00d-879b-491a-aa0b-8736512b0907-istio-envoy\") pod \"e2e-worker-w4-ef2572-openshift-default-7649dd87-lc4wb\" (UID: \"cdded00d-879b-491a-aa0b-8736512b0907\") " pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-lc4wb"
Oct 31 00:48:45.366207 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:45.366243    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cdded00d-879b-491a-aa0b-8736512b0907-credential-socket\") pod \"e2e-worker-w4-ef2572-openshift-default-7649dd87-lc4wb\" (UID: \"cdded00d-879b-491a-aa0b-8736512b0907\") " pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-lc4wb"
Oct 31 00:48:45.366422 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:45.366279    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cdded00d-879b-491a-aa0b-8736512b0907-workload-certs\") pod \"e2e-worker-w4-ef2572-openshift-default-7649dd87-lc4wb\" (UID: \"cdded00d-879b-491a-aa0b-8736512b0907\") " pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-lc4wb"
Oct 31 00:48:45.366641 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:45.366579    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cdded00d-879b-491a-aa0b-8736512b0907-credential-socket\") pod \"e2e-worker-w4-ef2572-openshift-default-7649dd87-lc4wb\" (UID: \"cdded00d-879b-491a-aa0b-8736512b0907\") " pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-lc4wb"
Oct 31 00:48:45.366874 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:45.366850    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cdded00d-879b-491a-aa0b-8736512b0907-istiod-ca-cert\") pod \"e2e-worker-w4-ef2572-openshift-default-7649dd87-lc4wb\" (UID: \"cdded00d-879b-491a-aa0b-8736512b0907\") " pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-lc4wb"
Oct 31 00:48:45.368664 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:45.368644    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cdded00d-879b-491a-aa0b-8736512b0907-istio-podinfo\") pod \"e2e-worker-w4-ef2572-openshift-default-7649dd87-lc4wb\" (UID: \"cdded00d-879b-491a-aa0b-8736512b0907\") " pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-lc4wb"
Oct 31 00:48:45.368777 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:45.368748    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cdded00d-879b-491a-aa0b-8736512b0907-istio-envoy\") pod \"e2e-worker-w4-ef2572-openshift-default-7649dd87-lc4wb\" (UID: \"cdded00d-879b-491a-aa0b-8736512b0907\") " pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-lc4wb"
Oct 31 00:48:45.430201 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:45.430138    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cdded00d-879b-491a-aa0b-8736512b0907-istio-token\") pod \"e2e-worker-w4-ef2572-openshift-default-7649dd87-lc4wb\" (UID: \"cdded00d-879b-491a-aa0b-8736512b0907\") " pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-lc4wb"
Oct 31 00:48:45.431734 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:45.431306    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4r9sf\" (UniqueName: \"kubernetes.io/projected/cdded00d-879b-491a-aa0b-8736512b0907-kube-api-access-4r9sf\") pod \"e2e-worker-w4-ef2572-openshift-default-7649dd87-lc4wb\" (UID: \"cdded00d-879b-491a-aa0b-8736512b0907\") " pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-lc4wb"
Oct 31 00:48:45.722416 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:45.722382    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-lc4wb"
Oct 31 00:48:46.153429 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:46.153337    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:48:46.181205 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:46.181142    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-lc4wb"]
Oct 31 00:48:46.212997 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:46.212882    4284 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 00:48:46.212997 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:46.212970    4284 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 00:48:46.213247 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:46.213021    4284 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 00:48:46.441921 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:46.441875    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["models-as-a-service/payload-processing-external-model-84cf56844-6pbmt"]
Oct 31 00:48:46.598574 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:46.598537    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:48:46.685399 ip-10-0-132-173 kubenswrapper[4284]: E1031 00:48:46.684613    4284 projected.go:266] Couldn't get secret models-as-a-service/agc-cred-001239fd95f6: secret "agc-cred-001239fd95f6" not found
Oct 31 00:48:46.685399 ip-10-0-132-173 kubenswrapper[4284]: E1031 00:48:46.684646    4284 projected.go:266] Couldn't get secret models-as-a-service/agc-cred-1a4f67076b57: secret "agc-cred-1a4f67076b57" not found
Oct 31 00:48:46.685399 ip-10-0-132-173 kubenswrapper[4284]: E1031 00:48:46.684655    4284 projected.go:266] Couldn't get secret models-as-a-service/agc-cred-da8c34345bbb: secret "agc-cred-da8c34345bbb" not found
Oct 31 00:48:46.685399 ip-10-0-132-173 kubenswrapper[4284]: E1031 00:48:46.684668    4284 projected.go:196] Error preparing data for projected volume provider-credentials for pod models-as-a-service/payload-processing-external-model-84cf56844-6pbmt: [secret "agc-cred-001239fd95f6" not found, secret "agc-cred-1a4f67076b57" not found, secret "agc-cred-da8c34345bbb" not found]
Oct 31 00:48:46.685399 ip-10-0-132-173 kubenswrapper[4284]: E1031 00:48:46.684734    4284 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e78a1e6c-10b0-4faf-b773-ffd07127b046-provider-credentials podName:e78a1e6c-10b0-4faf-b773-ffd07127b046 nodeName:}" failed. No retries permitted until 2026-10-31 00:48:50.684714827 +0000 UTC m=+1244.829721744 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "provider-credentials" (UniqueName: "kubernetes.io/projected/e78a1e6c-10b0-4faf-b773-ffd07127b046-provider-credentials") pod "payload-processing-external-model-84cf56844-6pbmt" (UID: "e78a1e6c-10b0-4faf-b773-ffd07127b046") : [secret "agc-cred-001239fd95f6" not found, secret "agc-cred-1a4f67076b57" not found, secret "agc-cred-da8c34345bbb" not found]
Oct 31 00:48:47.123014 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:47.122978    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-lc4wb" event={"ID":"cdded00d-879b-491a-aa0b-8736512b0907","Type":"ContainerStarted","Data":"efc8d05c1157193d3ce97f031f5d0169b18d48d7b4d96cc0c774f418ec241af4"}
Oct 31 00:48:47.123832 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:47.123804    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-lc4wb" event={"ID":"cdded00d-879b-491a-aa0b-8736512b0907","Type":"ContainerStarted","Data":"40ba7090893d3aab137c1b4c4cff6ae61f6c2c256ee08ac72f0a590d307ab481"}
Oct 31 00:48:47.176933 ip-10-0-132-173 kubenswrapper[4284]: E1031 00:48:47.176607    4284 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fd8sp" containerName="extractor"
Oct 31 00:48:47.248239 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:47.247958    4284 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-lc4wb" podStartSLOduration=3.247939228 podStartE2EDuration="3.247939228s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 00:48:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:48:47.245794707 +0000 UTC m=+1241.390801623" watchObservedRunningTime="2026-10-31 00:48:47.247939228 +0000 UTC m=+1241.392946146"
Oct 31 00:48:47.722597 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:47.722563    4284 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-lc4wb"
Oct 31 00:48:47.723946 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:47.723921    4284 patch_prober.go:29] interesting pod/e2e-worker-w4-ef2572-openshift-default-7649dd87-lc4wb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused" start-of-body=
Oct 31 00:48:47.724079 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:47.723989    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-lc4wb" podUID="cdded00d-879b-491a-aa0b-8736512b0907" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused"
Oct 31 00:48:48.722983 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:48.722943    4284 patch_prober.go:29] interesting pod/e2e-worker-w4-ef2572-openshift-default-7649dd87-lc4wb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused" start-of-body=
Oct 31 00:48:48.723442 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:48.722999    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-lc4wb" podUID="cdded00d-879b-491a-aa0b-8736512b0907" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused"
Oct 31 00:48:49.123061 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:49.122958    4284 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-7n64p"]
Oct 31 00:48:49.127377 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:49.127345    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-7n64p"
Oct 31 00:48:49.144567 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:49.144536    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b4556139-5a27-42e5-ae33-a85c75ea1209-istio-data\") pod \"e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-7n64p\" (UID: \"b4556139-5a27-42e5-ae33-a85c75ea1209\") " pod="openshift-ingress/e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-7n64p"
Oct 31 00:48:49.144738 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:49.144623    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b4556139-5a27-42e5-ae33-a85c75ea1209-workload-socket\") pod \"e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-7n64p\" (UID: \"b4556139-5a27-42e5-ae33-a85c75ea1209\") " pod="openshift-ingress/e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-7n64p"
Oct 31 00:48:49.144738 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:49.144667    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8fm2w\" (UniqueName: \"kubernetes.io/projected/b4556139-5a27-42e5-ae33-a85c75ea1209-kube-api-access-8fm2w\") pod \"e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-7n64p\" (UID: \"b4556139-5a27-42e5-ae33-a85c75ea1209\") " pod="openshift-ingress/e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-7n64p"
Oct 31 00:48:49.144908 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:49.144739    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b4556139-5a27-42e5-ae33-a85c75ea1209-istio-envoy\") pod \"e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-7n64p\" (UID: \"b4556139-5a27-42e5-ae33-a85c75ea1209\") " pod="openshift-ingress/e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-7n64p"
Oct 31 00:48:49.144908 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:49.144840    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b4556139-5a27-42e5-ae33-a85c75ea1209-istio-podinfo\") pod \"e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-7n64p\" (UID: \"b4556139-5a27-42e5-ae33-a85c75ea1209\") " pod="openshift-ingress/e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-7n64p"
Oct 31 00:48:49.144908 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:49.144886    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b4556139-5a27-42e5-ae33-a85c75ea1209-istiod-ca-cert\") pod \"e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-7n64p\" (UID: \"b4556139-5a27-42e5-ae33-a85c75ea1209\") " pod="openshift-ingress/e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-7n64p"
Oct 31 00:48:49.145075 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:49.144922    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b4556139-5a27-42e5-ae33-a85c75ea1209-credential-socket\") pod \"e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-7n64p\" (UID: \"b4556139-5a27-42e5-ae33-a85c75ea1209\") " pod="openshift-ingress/e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-7n64p"
Oct 31 00:48:49.145075 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:49.144952    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b4556139-5a27-42e5-ae33-a85c75ea1209-workload-certs\") pod \"e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-7n64p\" (UID: \"b4556139-5a27-42e5-ae33-a85c75ea1209\") " pod="openshift-ingress/e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-7n64p"
Oct 31 00:48:49.145248 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:49.145200    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b4556139-5a27-42e5-ae33-a85c75ea1209-istio-token\") pod \"e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-7n64p\" (UID: \"b4556139-5a27-42e5-ae33-a85c75ea1209\") " pod="openshift-ingress/e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-7n64p"
Oct 31 00:48:49.228607 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:49.228572    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-7n64p"]
Oct 31 00:48:49.246115 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:49.246084    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b4556139-5a27-42e5-ae33-a85c75ea1209-istio-token\") pod \"e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-7n64p\" (UID: \"b4556139-5a27-42e5-ae33-a85c75ea1209\") " pod="openshift-ingress/e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-7n64p"
Oct 31 00:48:49.246368 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:49.246350    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b4556139-5a27-42e5-ae33-a85c75ea1209-istio-data\") pod \"e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-7n64p\" (UID: \"b4556139-5a27-42e5-ae33-a85c75ea1209\") " pod="openshift-ingress/e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-7n64p"
Oct 31 00:48:49.246496 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:49.246420    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b4556139-5a27-42e5-ae33-a85c75ea1209-workload-socket\") pod \"e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-7n64p\" (UID: \"b4556139-5a27-42e5-ae33-a85c75ea1209\") " pod="openshift-ingress/e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-7n64p"
Oct 31 00:48:49.246496 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:49.246447    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8fm2w\" (UniqueName: \"kubernetes.io/projected/b4556139-5a27-42e5-ae33-a85c75ea1209-kube-api-access-8fm2w\") pod \"e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-7n64p\" (UID: \"b4556139-5a27-42e5-ae33-a85c75ea1209\") " pod="openshift-ingress/e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-7n64p"
Oct 31 00:48:49.246624 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:49.246524    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b4556139-5a27-42e5-ae33-a85c75ea1209-istio-envoy\") pod \"e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-7n64p\" (UID: \"b4556139-5a27-42e5-ae33-a85c75ea1209\") " pod="openshift-ingress/e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-7n64p"
Oct 31 00:48:49.246624 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:49.246576    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b4556139-5a27-42e5-ae33-a85c75ea1209-istio-podinfo\") pod \"e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-7n64p\" (UID: \"b4556139-5a27-42e5-ae33-a85c75ea1209\") " pod="openshift-ingress/e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-7n64p"
Oct 31 00:48:49.246730 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:49.246621    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b4556139-5a27-42e5-ae33-a85c75ea1209-istiod-ca-cert\") pod \"e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-7n64p\" (UID: \"b4556139-5a27-42e5-ae33-a85c75ea1209\") " pod="openshift-ingress/e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-7n64p"
Oct 31 00:48:49.246730 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:49.246653    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b4556139-5a27-42e5-ae33-a85c75ea1209-credential-socket\") pod \"e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-7n64p\" (UID: \"b4556139-5a27-42e5-ae33-a85c75ea1209\") " pod="openshift-ingress/e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-7n64p"
Oct 31 00:48:49.246730 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:49.246678    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b4556139-5a27-42e5-ae33-a85c75ea1209-workload-certs\") pod \"e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-7n64p\" (UID: \"b4556139-5a27-42e5-ae33-a85c75ea1209\") " pod="openshift-ingress/e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-7n64p"
Oct 31 00:48:49.246875 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:49.246842    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b4556139-5a27-42e5-ae33-a85c75ea1209-workload-socket\") pod \"e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-7n64p\" (UID: \"b4556139-5a27-42e5-ae33-a85c75ea1209\") " pod="openshift-ingress/e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-7n64p"
Oct 31 00:48:49.246932 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:49.246891    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b4556139-5a27-42e5-ae33-a85c75ea1209-istio-data\") pod \"e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-7n64p\" (UID: \"b4556139-5a27-42e5-ae33-a85c75ea1209\") " pod="openshift-ingress/e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-7n64p"
Oct 31 00:48:49.247098 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:49.247076    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b4556139-5a27-42e5-ae33-a85c75ea1209-workload-certs\") pod \"e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-7n64p\" (UID: \"b4556139-5a27-42e5-ae33-a85c75ea1209\") " pod="openshift-ingress/e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-7n64p"
Oct 31 00:48:49.247429 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:49.247398    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b4556139-5a27-42e5-ae33-a85c75ea1209-credential-socket\") pod \"e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-7n64p\" (UID: \"b4556139-5a27-42e5-ae33-a85c75ea1209\") " pod="openshift-ingress/e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-7n64p"
Oct 31 00:48:49.247667 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:49.247644    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b4556139-5a27-42e5-ae33-a85c75ea1209-istiod-ca-cert\") pod \"e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-7n64p\" (UID: \"b4556139-5a27-42e5-ae33-a85c75ea1209\") " pod="openshift-ingress/e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-7n64p"
Oct 31 00:48:49.248969 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:49.248950    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b4556139-5a27-42e5-ae33-a85c75ea1209-istio-envoy\") pod \"e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-7n64p\" (UID: \"b4556139-5a27-42e5-ae33-a85c75ea1209\") " pod="openshift-ingress/e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-7n64p"
Oct 31 00:48:49.249459 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:49.249442    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b4556139-5a27-42e5-ae33-a85c75ea1209-istio-podinfo\") pod \"e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-7n64p\" (UID: \"b4556139-5a27-42e5-ae33-a85c75ea1209\") " pod="openshift-ingress/e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-7n64p"
Oct 31 00:48:49.315262 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:49.315224    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b4556139-5a27-42e5-ae33-a85c75ea1209-istio-token\") pod \"e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-7n64p\" (UID: \"b4556139-5a27-42e5-ae33-a85c75ea1209\") " pod="openshift-ingress/e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-7n64p"
Oct 31 00:48:49.317825 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:49.317789    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8fm2w\" (UniqueName: \"kubernetes.io/projected/b4556139-5a27-42e5-ae33-a85c75ea1209-kube-api-access-8fm2w\") pod \"e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-7n64p\" (UID: \"b4556139-5a27-42e5-ae33-a85c75ea1209\") " pod="openshift-ingress/e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-7n64p"
Oct 31 00:48:49.450163 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:49.450128    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-7n64p"
Oct 31 00:48:49.722782 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:49.722699    4284 patch_prober.go:29] interesting pod/e2e-worker-w4-ef2572-openshift-default-7649dd87-lc4wb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused" start-of-body=
Oct 31 00:48:49.722782 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:49.722751    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-lc4wb" podUID="cdded00d-879b-491a-aa0b-8736512b0907" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused"
Oct 31 00:48:49.933242 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:49.932385    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-7n64p"]
Oct 31 00:48:50.023644 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:50.008438    4284 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 00:48:50.023644 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:50.008524    4284 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 00:48:50.023644 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:50.008595    4284 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 00:48:50.196337 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:50.193924    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-7n64p" event={"ID":"b4556139-5a27-42e5-ae33-a85c75ea1209","Type":"ContainerStarted","Data":"64ff255d7e887ceeba407a9c89a95a7bee9944b5b1622792f6b24cbeb2cdc87f"}
Oct 31 00:48:50.711535 ip-10-0-132-173 kubenswrapper[4284]: E1031 00:48:50.711485    4284 projected.go:266] Couldn't get secret models-as-a-service/agc-cred-001239fd95f6: secret "agc-cred-001239fd95f6" not found
Oct 31 00:48:50.711772 ip-10-0-132-173 kubenswrapper[4284]: E1031 00:48:50.711757    4284 projected.go:266] Couldn't get secret models-as-a-service/agc-cred-1a4f67076b57: secret "agc-cred-1a4f67076b57" not found
Oct 31 00:48:50.711895 ip-10-0-132-173 kubenswrapper[4284]: E1031 00:48:50.711885    4284 projected.go:266] Couldn't get secret models-as-a-service/agc-cred-da8c34345bbb: secret "agc-cred-da8c34345bbb" not found
Oct 31 00:48:50.711989 ip-10-0-132-173 kubenswrapper[4284]: E1031 00:48:50.711978    4284 projected.go:196] Error preparing data for projected volume provider-credentials for pod models-as-a-service/payload-processing-external-model-84cf56844-6pbmt: [secret "agc-cred-001239fd95f6" not found, secret "agc-cred-1a4f67076b57" not found, secret "agc-cred-da8c34345bbb" not found]
Oct 31 00:48:50.712147 ip-10-0-132-173 kubenswrapper[4284]: E1031 00:48:50.712129    4284 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e78a1e6c-10b0-4faf-b773-ffd07127b046-provider-credentials podName:e78a1e6c-10b0-4faf-b773-ffd07127b046 nodeName:}" failed. No retries permitted until 2026-10-31 00:48:58.712105598 +0000 UTC m=+1252.857112509 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "provider-credentials" (UniqueName: "kubernetes.io/projected/e78a1e6c-10b0-4faf-b773-ffd07127b046-provider-credentials") pod "payload-processing-external-model-84cf56844-6pbmt" (UID: "e78a1e6c-10b0-4faf-b773-ffd07127b046") : [secret "agc-cred-001239fd95f6" not found, secret "agc-cred-1a4f67076b57" not found, secret "agc-cred-da8c34345bbb" not found]
Oct 31 00:48:50.833267 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:50.789456    4284 patch_prober.go:29] interesting pod/e2e-worker-w4-ef2572-openshift-default-7649dd87-lc4wb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused" start-of-body=
Oct 31 00:48:50.890374 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:50.890322    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-lc4wb" podUID="cdded00d-879b-491a-aa0b-8736512b0907" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused"
Oct 31 00:48:51.302422 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:51.302368    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-7n64p" event={"ID":"b4556139-5a27-42e5-ae33-a85c75ea1209","Type":"ContainerStarted","Data":"a9db7c1234f6aab814b5b9e8b16f3cce82a0afe028a1a1bc819e2eaa9ac6475d"}
Oct 31 00:48:51.452410 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:51.452204    4284 kuberuntime_container.go:898] "Killing container with a grace period" pod="models-as-a-service/payload-processing-external-model-84cf56844-6pbmt" podUID="e78a1e6c-10b0-4faf-b773-ffd07127b046" containerName="payload-processing" containerID="cri-o://b821160793a38fdf501f377bb267c0e485d23d9170d6cdcd1feae00f50311c5e" gracePeriod=25
Oct 31 00:48:51.454828 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:51.454790    4284 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-7n64p"
Oct 31 00:48:51.486249 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:51.482727    4284 patch_prober.go:29] interesting pod/e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-7n64p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused" start-of-body=
Oct 31 00:48:51.486249 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:51.482810    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-7n64p" podUID="b4556139-5a27-42e5-ae33-a85c75ea1209" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused"
Oct 31 00:48:51.486249 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:51.484771    4284 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-7n64p" podStartSLOduration=2.48475275 podStartE2EDuration="2.48475275s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 00:48:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:48:51.483485086 +0000 UTC m=+1245.628492003" watchObservedRunningTime="2026-10-31 00:48:51.48475275 +0000 UTC m=+1245.629759666"
Oct 31 00:48:51.741237 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:51.740412    4284 patch_prober.go:29] interesting pod/e2e-worker-w4-ef2572-openshift-default-7649dd87-lc4wb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused" start-of-body=
Oct 31 00:48:51.741237 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:51.740500    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-lc4wb" podUID="cdded00d-879b-491a-aa0b-8736512b0907" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused"
Oct 31 00:48:52.139368 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:52.139321    4284 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-ef2572/distinct-2-w4-ef2572-kserve-547fccbd5-dfpfx" podUID="cb311a47-264c-4b33-bc31-377b94b0c59b" containerName="main" probeResult="failure" output="Get \"https://10.134.0.73:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:48:52.479773 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:52.479738    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:48:52.639495 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:52.639457    4284 generic.go:356] "Generic (PLEG): container finished" podID="e78a1e6c-10b0-4faf-b773-ffd07127b046" containerID="b821160793a38fdf501f377bb267c0e485d23d9170d6cdcd1feae00f50311c5e" exitCode=0
Oct 31 00:48:52.674239 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:52.647710    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="models-as-a-service/payload-processing-external-model-84cf56844-6pbmt" event={"ID":"e78a1e6c-10b0-4faf-b773-ffd07127b046","Type":"ContainerDied","Data":"b821160793a38fdf501f377bb267c0e485d23d9170d6cdcd1feae00f50311c5e"}
Oct 31 00:48:52.674239 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:52.655668    4284 patch_prober.go:29] interesting pod/e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-7n64p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused" start-of-body=
Oct 31 00:48:52.674239 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:52.655734    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-7n64p" podUID="b4556139-5a27-42e5-ae33-a85c75ea1209" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused"
Oct 31 00:48:52.821460 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:52.821363    4284 patch_prober.go:29] interesting pod/e2e-worker-w4-ef2572-openshift-default-7649dd87-lc4wb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused" start-of-body=
Oct 31 00:48:52.831239 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:52.821684    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-lc4wb" podUID="cdded00d-879b-491a-aa0b-8736512b0907" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused"
Oct 31 00:48:53.173239 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:53.165854    4284 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-b63c24/premium-w1-b63c24-kserve-549f786d9-k7g8j" podUID="b6846d68-7b58-4637-a47f-cd55bbf79b98" containerName="main" probeResult="failure" output="Get \"https://10.134.0.59:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:48:53.519421 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:53.511021    4284 patch_prober.go:29] interesting pod/e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-7n64p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused" start-of-body=
Oct 31 00:48:53.519421 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:53.511089    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-7n64p" podUID="b4556139-5a27-42e5-ae33-a85c75ea1209" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused"
Oct 31 00:48:53.746245 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:53.742418    4284 patch_prober.go:29] interesting pod/e2e-worker-w4-ef2572-openshift-default-7649dd87-lc4wb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused" start-of-body=
Oct 31 00:48:53.746245 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:53.742485    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-lc4wb" podUID="cdded00d-879b-491a-aa0b-8736512b0907" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused"
Oct 31 00:48:54.076709 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:54.076674    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="models-as-a-service/payload-processing-external-model-84cf56844-6pbmt"
Oct 31 00:48:54.170240 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:54.169309    4284 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-ef2572/distinct-2-w4-ef2572-kserve-547fccbd5-dfpfx" podUID="cb311a47-264c-4b33-bc31-377b94b0c59b" containerName="main" probeResult="failure" output="Get \"https://10.134.0.73:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:48:54.291240 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:54.290664    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e78a1e6c-10b0-4faf-b773-ffd07127b046-kube-api-access-tn7tk\" (UniqueName: \"kubernetes.io/projected/e78a1e6c-10b0-4faf-b773-ffd07127b046-kube-api-access-tn7tk\") pod \"e78a1e6c-10b0-4faf-b773-ffd07127b046\" (UID: \"e78a1e6c-10b0-4faf-b773-ffd07127b046\") "
Oct 31 00:48:54.291240 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:54.290764    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e78a1e6c-10b0-4faf-b773-ffd07127b046-routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/e78a1e6c-10b0-4faf-b773-ffd07127b046-routing-overlay-volume\") pod \"e78a1e6c-10b0-4faf-b773-ffd07127b046\" (UID: \"e78a1e6c-10b0-4faf-b773-ffd07127b046\") "
Oct 31 00:48:54.291240 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:54.290855    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e78a1e6c-10b0-4faf-b773-ffd07127b046-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e78a1e6c-10b0-4faf-b773-ffd07127b046-plugins-config-volume\") pod \"e78a1e6c-10b0-4faf-b773-ffd07127b046\" (UID: \"e78a1e6c-10b0-4faf-b773-ffd07127b046\") "
Oct 31 00:48:54.291240 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:54.290997    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e78a1e6c-10b0-4faf-b773-ffd07127b046-provider-credentials\" (UniqueName: \"kubernetes.io/projected/e78a1e6c-10b0-4faf-b773-ffd07127b046-provider-credentials\") pod \"e78a1e6c-10b0-4faf-b773-ffd07127b046\" (UID: \"e78a1e6c-10b0-4faf-b773-ffd07127b046\") "
Oct 31 00:48:54.380106 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:54.371514    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e78a1e6c-10b0-4faf-b773-ffd07127b046-routing-overlay-volume" pod "e78a1e6c-10b0-4faf-b773-ffd07127b046" (UID: "e78a1e6c-10b0-4faf-b773-ffd07127b046"). InnerVolumeSpecName "routing-overlay-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 00:48:54.463694 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:54.399521    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e78a1e6c-10b0-4faf-b773-ffd07127b046-kube-api-access-tn7tk" pod "e78a1e6c-10b0-4faf-b773-ffd07127b046" (UID: "e78a1e6c-10b0-4faf-b773-ffd07127b046"). InnerVolumeSpecName "kube-api-access-tn7tk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 00:48:54.463932 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:54.419486    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e78a1e6c-10b0-4faf-b773-ffd07127b046-plugins-config-volume" pod "e78a1e6c-10b0-4faf-b773-ffd07127b046" (UID: "e78a1e6c-10b0-4faf-b773-ffd07127b046"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 00:48:54.464032 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:54.447920    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e78a1e6c-10b0-4faf-b773-ffd07127b046-provider-credentials" pod "e78a1e6c-10b0-4faf-b773-ffd07127b046" (UID: "e78a1e6c-10b0-4faf-b773-ffd07127b046"). InnerVolumeSpecName "provider-credentials". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 00:48:54.506239 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:54.506024    4284 reconciler_common.go:299] "Volume detached for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/e78a1e6c-10b0-4faf-b773-ffd07127b046-routing-overlay-volume\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:48:54.506239 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:54.506074    4284 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e78a1e6c-10b0-4faf-b773-ffd07127b046-plugins-config-volume\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:48:54.506239 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:54.506093    4284 reconciler_common.go:299] "Volume detached for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/e78a1e6c-10b0-4faf-b773-ffd07127b046-provider-credentials\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:48:54.506239 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:54.506109    4284 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tn7tk\" (UniqueName: \"kubernetes.io/projected/e78a1e6c-10b0-4faf-b773-ffd07127b046-kube-api-access-tn7tk\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:48:54.529282 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:54.510770    4284 patch_prober.go:29] interesting pod/e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-7n64p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused" start-of-body=
Oct 31 00:48:54.529282 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:54.510847    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-7n64p" podUID="b4556139-5a27-42e5-ae33-a85c75ea1209" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused"
Oct 31 00:48:54.560001 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:54.559955    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="models-as-a-service/payload-processing-external-model-84cf56844-6pbmt" event={"ID":"e78a1e6c-10b0-4faf-b773-ffd07127b046","Type":"ContainerDied","Data":"407f7e1786c82364d4a305cf50d797dc0560ab0f5b950d3e8d68c776808e7089"}
Oct 31 00:48:54.723384 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:54.723347    4284 scope.go:122] "RemoveContainer" containerID="b821160793a38fdf501f377bb267c0e485d23d9170d6cdcd1feae00f50311c5e"
Oct 31 00:48:54.725023 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:54.724993    4284 patch_prober.go:29] interesting pod/e2e-worker-w4-ef2572-openshift-default-7649dd87-lc4wb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused" start-of-body=
Oct 31 00:48:54.770624 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:54.761482    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-lc4wb" podUID="cdded00d-879b-491a-aa0b-8736512b0907" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused"
Oct 31 00:48:55.174013 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:55.169950    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="models-as-a-service/payload-processing-external-model-84cf56844-6pbmt"
Oct 31 00:48:55.451463 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:55.451124    4284 patch_prober.go:29] interesting pod/e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-7n64p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused" start-of-body=
Oct 31 00:48:55.451463 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:55.451203    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-7n64p" podUID="b4556139-5a27-42e5-ae33-a85c75ea1209" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused"
Oct 31 00:48:55.796296 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:55.781648    4284 patch_prober.go:29] interesting pod/e2e-worker-w4-ef2572-openshift-default-7649dd87-lc4wb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused" start-of-body=
Oct 31 00:48:55.796296 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:55.781705    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-lc4wb" podUID="cdded00d-879b-491a-aa0b-8736512b0907" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused"
Oct 31 00:48:55.817321 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:55.817279    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["models-as-a-service/payload-processing-external-model-84cf56844-6pbmt"]
Oct 31 00:48:55.911239 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:55.889619    4284 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["models-as-a-service/payload-processing-external-model-84cf56844-6pbmt"]
Oct 31 00:48:56.122292 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:56.122071    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:48:56.151624 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:56.148389    4284 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-ef2572/distinct-2-w4-ef2572-kserve-547fccbd5-dfpfx" podUID="cb311a47-264c-4b33-bc31-377b94b0c59b" containerName="main" probeResult="failure" output="Get \"https://10.134.0.73:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:48:56.208103 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:56.207355    4284 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-b63c24/premium-w1-b63c24-kserve-549f786d9-k7g8j" podUID="b6846d68-7b58-4637-a47f-cd55bbf79b98" containerName="main" probeResult="failure" output="Get \"https://10.134.0.59:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:48:56.465256 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:56.464844    4284 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e78a1e6c-10b0-4faf-b773-ffd07127b046" path="/var/lib/kubelet/pods/e78a1e6c-10b0-4faf-b773-ffd07127b046/volumes"
Oct 31 00:48:56.485239 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:56.470781    4284 patch_prober.go:29] interesting pod/e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-7n64p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused" start-of-body=
Oct 31 00:48:56.485239 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:56.470842    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-7n64p" podUID="b4556139-5a27-42e5-ae33-a85c75ea1209" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused"
Oct 31 00:48:56.729418 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:56.729323    4284 patch_prober.go:29] interesting pod/e2e-worker-w4-ef2572-openshift-default-7649dd87-lc4wb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused" start-of-body=
Oct 31 00:48:56.763248 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:56.731281    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-lc4wb" podUID="cdded00d-879b-491a-aa0b-8736512b0907" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused"
Oct 31 00:48:57.451442 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:57.451405    4284 patch_prober.go:29] interesting pod/e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-7n64p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused" start-of-body=
Oct 31 00:48:57.451846 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:57.451453    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-7n64p" podUID="b4556139-5a27-42e5-ae33-a85c75ea1209" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused"
Oct 31 00:48:57.738163 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:57.738053    4284 patch_prober.go:29] interesting pod/e2e-worker-w4-ef2572-openshift-default-7649dd87-lc4wb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused" start-of-body=
Oct 31 00:48:57.743255 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:57.740322    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-lc4wb" podUID="cdded00d-879b-491a-aa0b-8736512b0907" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused"
Oct 31 00:48:58.450600 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:58.450561    4284 patch_prober.go:29] interesting pod/e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-7n64p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused" start-of-body=
Oct 31 00:48:58.450784 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:58.450614    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-7n64p" podUID="b4556139-5a27-42e5-ae33-a85c75ea1209" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused"
Oct 31 00:48:58.727777 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:58.727698    4284 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-lc4wb"
Oct 31 00:48:58.728155 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:58.728089    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-lc4wb"
Oct 31 00:48:58.728907 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:58.728888    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-lc4wb"
Oct 31 00:48:59.341041 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:59.341000    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:48:59.471485 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:59.471448    4284 patch_prober.go:29] interesting pod/e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-7n64p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused" start-of-body=
Oct 31 00:48:59.471698 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:48:59.471524    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-7n64p" podUID="b4556139-5a27-42e5-ae33-a85c75ea1209" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused"
Oct 31 00:49:00.450954 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:49:00.450919    4284 patch_prober.go:29] interesting pod/e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-7n64p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused" start-of-body=
Oct 31 00:49:00.451393 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:49:00.450976    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-7n64p" podUID="b4556139-5a27-42e5-ae33-a85c75ea1209" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused"
Oct 31 00:49:01.450566 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:49:01.450532    4284 patch_prober.go:29] interesting pod/e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-7n64p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused" start-of-body=
Oct 31 00:49:01.450750 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:49:01.450591    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-7n64p" podUID="b4556139-5a27-42e5-ae33-a85c75ea1209" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused"
Oct 31 00:49:02.455986 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:49:02.455956    4284 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-7n64p"
Oct 31 00:49:02.456485 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:49:02.456244    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-7n64p"
Oct 31 00:49:02.457105 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:49:02.457083    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-7n64p"
Oct 31 00:49:02.559800 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:49:02.559765    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:49:04.218644 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:49:04.218607    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:49:07.101286 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:49:07.101253    4284 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-522s8"]
Oct 31 00:49:07.101840 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:49:07.101822    4284 cpu_manager.go:414] "containerMap: removing container" podUID="e78a1e6c-10b0-4faf-b773-ffd07127b046" containerName="payload-processing"
Oct 31 00:49:07.101919 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:49:07.101842    4284 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e78a1e6c-10b0-4faf-b773-ffd07127b046" containerName="payload-processing"
Oct 31 00:49:07.102023 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:49:07.102009    4284 memory_manager.go:365] "RemoveStaleState removing state" podUID="e78a1e6c-10b0-4faf-b773-ffd07127b046" containerName="payload-processing"
Oct 31 00:49:07.107369 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:49:07.107346    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-522s8"
Oct 31 00:49:07.209925 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:49:07.209891    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-522s8"]
Oct 31 00:49:07.286654 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:49:07.286617    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7cfa573e-ec98-4eaf-b12f-01f723e5e878-workload-certs\") pod \"e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-522s8\" (UID: \"7cfa573e-ec98-4eaf-b12f-01f723e5e878\") " pod="openshift-ingress/e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-522s8"
Oct 31 00:49:07.286842 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:49:07.286667    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7cfa573e-ec98-4eaf-b12f-01f723e5e878-istio-podinfo\") pod \"e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-522s8\" (UID: \"7cfa573e-ec98-4eaf-b12f-01f723e5e878\") " pod="openshift-ingress/e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-522s8"
Oct 31 00:49:07.286842 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:49:07.286752    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7cfa573e-ec98-4eaf-b12f-01f723e5e878-istiod-ca-cert\") pod \"e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-522s8\" (UID: \"7cfa573e-ec98-4eaf-b12f-01f723e5e878\") " pod="openshift-ingress/e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-522s8"
Oct 31 00:49:07.286842 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:49:07.286800    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7cfa573e-ec98-4eaf-b12f-01f723e5e878-credential-socket\") pod \"e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-522s8\" (UID: \"7cfa573e-ec98-4eaf-b12f-01f723e5e878\") " pod="openshift-ingress/e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-522s8"
Oct 31 00:49:07.287001 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:49:07.286946    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7cfa573e-ec98-4eaf-b12f-01f723e5e878-workload-socket\") pod \"e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-522s8\" (UID: \"7cfa573e-ec98-4eaf-b12f-01f723e5e878\") " pod="openshift-ingress/e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-522s8"
Oct 31 00:49:07.287108 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:49:07.287091    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7cfa573e-ec98-4eaf-b12f-01f723e5e878-istio-token\") pod \"e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-522s8\" (UID: \"7cfa573e-ec98-4eaf-b12f-01f723e5e878\") " pod="openshift-ingress/e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-522s8"
Oct 31 00:49:07.287233 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:49:07.287152    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rd98t\" (UniqueName: \"kubernetes.io/projected/7cfa573e-ec98-4eaf-b12f-01f723e5e878-kube-api-access-rd98t\") pod \"e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-522s8\" (UID: \"7cfa573e-ec98-4eaf-b12f-01f723e5e878\") " pod="openshift-ingress/e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-522s8"
Oct 31 00:49:07.287281 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:49:07.287252    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7cfa573e-ec98-4eaf-b12f-01f723e5e878-istio-envoy\") pod \"e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-522s8\" (UID: \"7cfa573e-ec98-4eaf-b12f-01f723e5e878\") " pod="openshift-ingress/e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-522s8"
Oct 31 00:49:07.287281 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:49:07.287275    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7cfa573e-ec98-4eaf-b12f-01f723e5e878-istio-data\") pod \"e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-522s8\" (UID: \"7cfa573e-ec98-4eaf-b12f-01f723e5e878\") " pod="openshift-ingress/e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-522s8"
Oct 31 00:49:07.388120 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:49:07.388037    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7cfa573e-ec98-4eaf-b12f-01f723e5e878-workload-socket\") pod \"e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-522s8\" (UID: \"7cfa573e-ec98-4eaf-b12f-01f723e5e878\") " pod="openshift-ingress/e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-522s8"
Oct 31 00:49:07.388331 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:49:07.388137    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7cfa573e-ec98-4eaf-b12f-01f723e5e878-istio-token\") pod \"e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-522s8\" (UID: \"7cfa573e-ec98-4eaf-b12f-01f723e5e878\") " pod="openshift-ingress/e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-522s8"
Oct 31 00:49:07.388331 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:49:07.388154    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rd98t\" (UniqueName: \"kubernetes.io/projected/7cfa573e-ec98-4eaf-b12f-01f723e5e878-kube-api-access-rd98t\") pod \"e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-522s8\" (UID: \"7cfa573e-ec98-4eaf-b12f-01f723e5e878\") " pod="openshift-ingress/e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-522s8"
Oct 31 00:49:07.388331 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:49:07.388186    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7cfa573e-ec98-4eaf-b12f-01f723e5e878-istio-envoy\") pod \"e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-522s8\" (UID: \"7cfa573e-ec98-4eaf-b12f-01f723e5e878\") " pod="openshift-ingress/e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-522s8"
Oct 31 00:49:07.388331 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:49:07.388203    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7cfa573e-ec98-4eaf-b12f-01f723e5e878-istio-data\") pod \"e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-522s8\" (UID: \"7cfa573e-ec98-4eaf-b12f-01f723e5e878\") " pod="openshift-ingress/e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-522s8"
Oct 31 00:49:07.388331 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:49:07.388265    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7cfa573e-ec98-4eaf-b12f-01f723e5e878-workload-certs\") pod \"e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-522s8\" (UID: \"7cfa573e-ec98-4eaf-b12f-01f723e5e878\") " pod="openshift-ingress/e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-522s8"
Oct 31 00:49:07.388331 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:49:07.388282    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7cfa573e-ec98-4eaf-b12f-01f723e5e878-istio-podinfo\") pod \"e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-522s8\" (UID: \"7cfa573e-ec98-4eaf-b12f-01f723e5e878\") " pod="openshift-ingress/e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-522s8"
Oct 31 00:49:07.388331 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:49:07.388297    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7cfa573e-ec98-4eaf-b12f-01f723e5e878-istiod-ca-cert\") pod \"e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-522s8\" (UID: \"7cfa573e-ec98-4eaf-b12f-01f723e5e878\") " pod="openshift-ingress/e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-522s8"
Oct 31 00:49:07.388331 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:49:07.388327    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7cfa573e-ec98-4eaf-b12f-01f723e5e878-credential-socket\") pod \"e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-522s8\" (UID: \"7cfa573e-ec98-4eaf-b12f-01f723e5e878\") " pod="openshift-ingress/e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-522s8"
Oct 31 00:49:07.388777 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:49:07.388476    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7cfa573e-ec98-4eaf-b12f-01f723e5e878-workload-socket\") pod \"e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-522s8\" (UID: \"7cfa573e-ec98-4eaf-b12f-01f723e5e878\") " pod="openshift-ingress/e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-522s8"
Oct 31 00:49:07.388777 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:49:07.388637    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7cfa573e-ec98-4eaf-b12f-01f723e5e878-workload-certs\") pod \"e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-522s8\" (UID: \"7cfa573e-ec98-4eaf-b12f-01f723e5e878\") " pod="openshift-ingress/e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-522s8"
Oct 31 00:49:07.388777 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:49:07.388689    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7cfa573e-ec98-4eaf-b12f-01f723e5e878-credential-socket\") pod \"e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-522s8\" (UID: \"7cfa573e-ec98-4eaf-b12f-01f723e5e878\") " pod="openshift-ingress/e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-522s8"
Oct 31 00:49:07.388777 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:49:07.388692    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7cfa573e-ec98-4eaf-b12f-01f723e5e878-istio-data\") pod \"e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-522s8\" (UID: \"7cfa573e-ec98-4eaf-b12f-01f723e5e878\") " pod="openshift-ingress/e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-522s8"
Oct 31 00:49:07.388973 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:49:07.388957    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7cfa573e-ec98-4eaf-b12f-01f723e5e878-istiod-ca-cert\") pod \"e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-522s8\" (UID: \"7cfa573e-ec98-4eaf-b12f-01f723e5e878\") " pod="openshift-ingress/e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-522s8"
Oct 31 00:49:07.390739 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:49:07.390715    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7cfa573e-ec98-4eaf-b12f-01f723e5e878-istio-envoy\") pod \"e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-522s8\" (UID: \"7cfa573e-ec98-4eaf-b12f-01f723e5e878\") " pod="openshift-ingress/e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-522s8"
Oct 31 00:49:07.391381 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:49:07.391362    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7cfa573e-ec98-4eaf-b12f-01f723e5e878-istio-podinfo\") pod \"e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-522s8\" (UID: \"7cfa573e-ec98-4eaf-b12f-01f723e5e878\") " pod="openshift-ingress/e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-522s8"
Oct 31 00:49:07.447294 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:49:07.447262    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7cfa573e-ec98-4eaf-b12f-01f723e5e878-istio-token\") pod \"e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-522s8\" (UID: \"7cfa573e-ec98-4eaf-b12f-01f723e5e878\") " pod="openshift-ingress/e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-522s8"
Oct 31 00:49:07.447560 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:49:07.447537    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rd98t\" (UniqueName: \"kubernetes.io/projected/7cfa573e-ec98-4eaf-b12f-01f723e5e878-kube-api-access-rd98t\") pod \"e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-522s8\" (UID: \"7cfa573e-ec98-4eaf-b12f-01f723e5e878\") " pod="openshift-ingress/e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-522s8"
Oct 31 00:49:07.725413 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:49:07.725382    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-522s8"
Oct 31 00:49:08.148263 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:49:08.148227    4284 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7cfa573e_ec98_4eaf_b12f_01f723e5e878.slice/crio-71dad47ca09eeffbde6ad4f681b754036361d03ce85ff68d1826f9df1b9ea634 WatchSource:0}: Error finding container 71dad47ca09eeffbde6ad4f681b754036361d03ce85ff68d1826f9df1b9ea634: Status 404 returned error can't find the container with id 71dad47ca09eeffbde6ad4f681b754036361d03ce85ff68d1826f9df1b9ea634
Oct 31 00:49:08.149418 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:49:08.149387    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-522s8"]
Oct 31 00:49:08.151266 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:49:08.151188    4284 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 00:49:08.151363 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:49:08.151314    4284 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 00:49:08.151363 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:49:08.151358    4284 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 00:49:08.598611 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:49:08.598558    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-522s8" event={"ID":"7cfa573e-ec98-4eaf-b12f-01f723e5e878","Type":"ContainerStarted","Data":"e115a6c8d0cb6e96dec8c647554181f54149fda01c2429d3ea743bbf1be49c9f"}
Oct 31 00:49:08.598836 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:49:08.598678    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-522s8" event={"ID":"7cfa573e-ec98-4eaf-b12f-01f723e5e878","Type":"ContainerStarted","Data":"71dad47ca09eeffbde6ad4f681b754036361d03ce85ff68d1826f9df1b9ea634"}
Oct 31 00:49:08.716733 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:49:08.716678    4284 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-522s8" podStartSLOduration=1.716662199 podStartE2EDuration="1.716662199s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 00:49:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:49:08.713439628 +0000 UTC m=+1262.858446536" watchObservedRunningTime="2026-10-31 00:49:08.716662199 +0000 UTC m=+1262.861669115"
Oct 31 00:49:08.726196 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:49:08.726163    4284 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-522s8"
Oct 31 00:49:09.078292 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:49:09.078259    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:49:09.727114 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:49:09.727078    4284 patch_prober.go:29] interesting pod/e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-522s8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.78:15021/healthz/ready\": dial tcp 10.134.0.78:15021: connect: connection refused" start-of-body=
Oct 31 00:49:09.727566 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:49:09.727128    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-522s8" podUID="7cfa573e-ec98-4eaf-b12f-01f723e5e878" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.78:15021/healthz/ready\": dial tcp 10.134.0.78:15021: connect: connection refused"
Oct 31 00:49:10.726524 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:49:10.726485    4284 patch_prober.go:29] interesting pod/e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-522s8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.78:15021/healthz/ready\": dial tcp 10.134.0.78:15021: connect: connection refused" start-of-body=
Oct 31 00:49:10.726713 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:49:10.726539    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-522s8" podUID="7cfa573e-ec98-4eaf-b12f-01f723e5e878" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.78:15021/healthz/ready\": dial tcp 10.134.0.78:15021: connect: connection refused"
Oct 31 00:49:11.731681 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:49:11.731637    4284 patch_prober.go:29] interesting pod/e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-522s8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.78:15021/healthz/ready\": dial tcp 10.134.0.78:15021: connect: connection refused" start-of-body=
Oct 31 00:49:11.732334 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:49:11.732301    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-522s8" podUID="7cfa573e-ec98-4eaf-b12f-01f723e5e878" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.78:15021/healthz/ready\": dial tcp 10.134.0.78:15021: connect: connection refused"
Oct 31 00:49:12.732373 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:49:12.732336    4284 patch_prober.go:29] interesting pod/e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-522s8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.78:15021/healthz/ready\": dial tcp 10.134.0.78:15021: connect: connection refused" start-of-body=
Oct 31 00:49:12.732997 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:49:12.732962    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-522s8" podUID="7cfa573e-ec98-4eaf-b12f-01f723e5e878" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.78:15021/healthz/ready\": dial tcp 10.134.0.78:15021: connect: connection refused"
Oct 31 00:49:13.731841 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:49:13.731803    4284 patch_prober.go:29] interesting pod/e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-522s8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.78:15021/healthz/ready\": dial tcp 10.134.0.78:15021: connect: connection refused" start-of-body=
Oct 31 00:49:13.732187 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:49:13.732147    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-522s8" podUID="7cfa573e-ec98-4eaf-b12f-01f723e5e878" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.78:15021/healthz/ready\": dial tcp 10.134.0.78:15021: connect: connection refused"
Oct 31 00:49:14.140422 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:49:14.140308    4284 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-ef2572/distinct-2-w4-ef2572-kserve-547fccbd5-dfpfx" podUID="cb311a47-264c-4b33-bc31-377b94b0c59b" containerName="main" probeResult="failure" output="Get \"https://10.134.0.73:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:49:14.192390 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:49:14.192339    4284 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-b63c24/premium-w1-b63c24-kserve-549f786d9-k7g8j" podUID="b6846d68-7b58-4637-a47f-cd55bbf79b98" containerName="main" probeResult="failure" output="Get \"https://10.134.0.59:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:49:14.818241 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:49:14.759584    4284 patch_prober.go:29] interesting pod/e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-522s8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.78:15021/healthz/ready\": dial tcp 10.134.0.78:15021: connect: connection refused" start-of-body=
Oct 31 00:49:14.917236 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:49:14.914096    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-522s8" podUID="7cfa573e-ec98-4eaf-b12f-01f723e5e878" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.78:15021/healthz/ready\": dial tcp 10.134.0.78:15021: connect: connection refused"
Oct 31 00:49:15.194239 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:49:15.190946    4284 patch_prober.go:29] interesting pod/e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-k89gx container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.69:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 31 00:49:15.194239 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:49:15.191082    4284 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-k89gx" podUID="41a8f8d6-dbbb-48db-a957-5a3bfe8aa418" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.69:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:49:15.443312 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:49:15.442711    4284 patch_prober.go:29] interesting pod/e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-fnnxh container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.74:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 31 00:49:15.443312 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:49:15.442817    4284 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-fnnxh" podUID="55d97eb9-77d5-4b56-ab6d-5f38df8d1deb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.74:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:49:15.797404 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:49:15.797362    4284 patch_prober.go:29] interesting pod/e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-522s8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.78:15021/healthz/ready\": dial tcp 10.134.0.78:15021: connect: connection refused" start-of-body=
Oct 31 00:49:15.805238 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:49:15.801298    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-522s8" podUID="7cfa573e-ec98-4eaf-b12f-01f723e5e878" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.78:15021/healthz/ready\": dial tcp 10.134.0.78:15021: connect: connection refused"
Oct 31 00:49:16.297270 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:49:16.285685    4284 patch_prober.go:29] interesting pod/e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-2sg7k container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.75:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 31 00:49:16.297270 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:49:16.285773    4284 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-2sg7k" podUID="a973c5a8-c25c-41ef-b2d7-7289fdc66b42" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.75:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:49:16.755445 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:49:16.755408    4284 patch_prober.go:29] interesting pod/e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-522s8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.78:15021/healthz/ready\": dial tcp 10.134.0.78:15021: connect: connection refused" start-of-body=
Oct 31 00:49:16.768488 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:49:16.768444    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-522s8" podUID="7cfa573e-ec98-4eaf-b12f-01f723e5e878" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.78:15021/healthz/ready\": dial tcp 10.134.0.78:15021: connect: connection refused"
Oct 31 00:49:17.146438 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:49:17.146335    4284 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-ef2572/distinct-2-w4-ef2572-kserve-547fccbd5-dfpfx" podUID="cb311a47-264c-4b33-bc31-377b94b0c59b" containerName="main" probeResult="failure" output="Get \"https://10.134.0.73:8000/ready\": context deadline exceeded"
Oct 31 00:49:17.228037 ip-10-0-132-173 kubenswrapper[4284]: E1031 00:49:17.227746    4284 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fd8sp" containerName="extractor"
Oct 31 00:49:17.738543 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:49:17.738491    4284 patch_prober.go:29] interesting pod/e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-522s8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.78:15021/healthz/ready\": dial tcp 10.134.0.78:15021: connect: connection refused" start-of-body=
Oct 31 00:49:17.739227 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:49:17.739162    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-522s8" podUID="7cfa573e-ec98-4eaf-b12f-01f723e5e878" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.78:15021/healthz/ready\": dial tcp 10.134.0.78:15021: connect: connection refused"
Oct 31 00:49:17.983664 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:49:17.983318    4284 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6cc458d887-r7hr5 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.65:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 31 00:49:17.983664 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:49:17.983388    4284 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-r7hr5" podUID="6cce1cfc-7109-4827-a231-fe01dd9a4bf9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.65:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:49:18.169296 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:49:18.150313    4284 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-ef2572/distinct-2-w4-ef2572-kserve-547fccbd5-dfpfx" podUID="cb311a47-264c-4b33-bc31-377b94b0c59b" containerName="main" probeResult="failure" output="Get \"https://10.134.0.73:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:49:18.169296 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:49:18.150761    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-ef2572/distinct-2-w4-ef2572-kserve-547fccbd5-dfpfx"
Oct 31 00:49:18.503329 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:49:18.449832    4284 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6cc458d887-d4s6f container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.64:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 31 00:49:18.503329 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:49:18.449900    4284 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-d4s6f" podUID="b398d9e9-9d78-4185-ac8c-4a9eecca9f48" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.64:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:49:18.503329 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:49:18.468492    4284 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6cc458d887-hvt6d container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.63:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 31 00:49:18.503329 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:49:18.468558    4284 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-hvt6d" podUID="907b625f-2ccc-4de2-8181-34ff51ab2620" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.63:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:49:18.755299 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:49:18.754400    4284 patch_prober.go:29] interesting pod/e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-522s8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.78:15021/healthz/ready\": dial tcp 10.134.0.78:15021: connect: connection refused" start-of-body=
Oct 31 00:49:18.755299 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:49:18.754463    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-522s8" podUID="7cfa573e-ec98-4eaf-b12f-01f723e5e878" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.78:15021/healthz/ready\": dial tcp 10.134.0.78:15021: connect: connection refused"
Oct 31 00:49:18.817241 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:49:18.814653    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:49:18.904972 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:49:18.838269    4284 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w4-ef2572/distinct-2-w4-ef2572-kserve-547fccbd5-dfpfx" podUID="cb311a47-264c-4b33-bc31-377b94b0c59b" containerName="main" probeResult="failure" output="Get \"https://10.134.0.73:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:49:19.153468 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:49:19.153350    4284 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-ef2572/distinct-2-w4-ef2572-kserve-547fccbd5-dfpfx" podUID="cb311a47-264c-4b33-bc31-377b94b0c59b" containerName="main" probeResult="failure" output="Get \"https://10.134.0.73:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:49:19.190249 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:49:19.178524    4284 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-b63c24/premium-w1-b63c24-kserve-549f786d9-k7g8j" podUID="b6846d68-7b58-4637-a47f-cd55bbf79b98" containerName="main" probeResult="failure" output="Get \"https://10.134.0.59:8000/ready\": context deadline exceeded"
Oct 31 00:49:19.494316 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:49:19.459580    4284 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-vqsxn"]
Oct 31 00:49:19.555662 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:49:19.555632    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-vqsxn"
Oct 31 00:49:19.600423 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:49:19.600387    4284 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6cc458d887-rr5k6 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.39:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 31 00:49:19.600691 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:49:19.600661    4284 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-rr5k6" podUID="27f8cf21-0f47-4397-831a-bf716741dc67" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.39:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:49:19.611643 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:49:19.603206    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-vqsxn"]
Oct 31 00:49:19.697244 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:49:19.695260    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d3800ace-ae94-47da-ac65-218d3830b13d-credential-socket\") pod \"e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-vqsxn\" (UID: \"d3800ace-ae94-47da-ac65-218d3830b13d\") " pod="openshift-ingress/e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-vqsxn"
Oct 31 00:49:19.707290 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:49:19.701442    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d3800ace-ae94-47da-ac65-218d3830b13d-workload-socket\") pod \"e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-vqsxn\" (UID: \"d3800ace-ae94-47da-ac65-218d3830b13d\") " pod="openshift-ingress/e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-vqsxn"
Oct 31 00:49:19.746292 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:49:19.705302    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d3800ace-ae94-47da-ac65-218d3830b13d-istiod-ca-cert\") pod \"e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-vqsxn\" (UID: \"d3800ace-ae94-47da-ac65-218d3830b13d\") " pod="openshift-ingress/e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-vqsxn"
Oct 31 00:49:19.746292 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:49:19.709357    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d3800ace-ae94-47da-ac65-218d3830b13d-istio-podinfo\") pod \"e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-vqsxn\" (UID: \"d3800ace-ae94-47da-ac65-218d3830b13d\") " pod="openshift-ingress/e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-vqsxn"
Oct 31 00:49:19.758811 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:49:19.746928    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d3800ace-ae94-47da-ac65-218d3830b13d-workload-certs\") pod \"e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-vqsxn\" (UID: \"d3800ace-ae94-47da-ac65-218d3830b13d\") " pod="openshift-ingress/e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-vqsxn"
Oct 31 00:49:19.758811 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:49:19.747153    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d3800ace-ae94-47da-ac65-218d3830b13d-istio-token\") pod \"e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-vqsxn\" (UID: \"d3800ace-ae94-47da-ac65-218d3830b13d\") " pod="openshift-ingress/e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-vqsxn"
Oct 31 00:49:19.758811 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:49:19.747378    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kg8bn\" (UniqueName: \"kubernetes.io/projected/d3800ace-ae94-47da-ac65-218d3830b13d-kube-api-access-kg8bn\") pod \"e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-vqsxn\" (UID: \"d3800ace-ae94-47da-ac65-218d3830b13d\") " pod="openshift-ingress/e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-vqsxn"
Oct 31 00:49:19.758811 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:49:19.747544    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d3800ace-ae94-47da-ac65-218d3830b13d-istio-envoy\") pod \"e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-vqsxn\" (UID: \"d3800ace-ae94-47da-ac65-218d3830b13d\") " pod="openshift-ingress/e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-vqsxn"
Oct 31 00:49:19.813248 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:49:19.765929    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d3800ace-ae94-47da-ac65-218d3830b13d-istio-data\") pod \"e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-vqsxn\" (UID: \"d3800ace-ae94-47da-ac65-218d3830b13d\") " pod="openshift-ingress/e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-vqsxn"
Oct 31 00:49:19.925435 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:49:19.921627    4284 patch_prober.go:29] interesting pod/e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-522s8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.78:15021/healthz/ready\": dial tcp 10.134.0.78:15021: connect: connection refused" start-of-body=
Oct 31 00:49:19.925435 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:49:19.921694    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-522s8" podUID="7cfa573e-ec98-4eaf-b12f-01f723e5e878" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.78:15021/healthz/ready\": dial tcp 10.134.0.78:15021: connect: connection refused"
Oct 31 00:49:20.021297 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:49:20.021177    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d3800ace-ae94-47da-ac65-218d3830b13d-workload-certs\") pod \"e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-vqsxn\" (UID: \"d3800ace-ae94-47da-ac65-218d3830b13d\") " pod="openshift-ingress/e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-vqsxn"
Oct 31 00:49:20.074998 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:49:20.074968    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d3800ace-ae94-47da-ac65-218d3830b13d-istio-token\") pod \"e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-vqsxn\" (UID: \"d3800ace-ae94-47da-ac65-218d3830b13d\") " pod="openshift-ingress/e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-vqsxn"
Oct 31 00:49:20.075338 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:49:20.075313    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kg8bn\" (UniqueName: \"kubernetes.io/projected/d3800ace-ae94-47da-ac65-218d3830b13d-kube-api-access-kg8bn\") pod \"e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-vqsxn\" (UID: \"d3800ace-ae94-47da-ac65-218d3830b13d\") " pod="openshift-ingress/e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-vqsxn"
Oct 31 00:49:20.075527 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:49:20.075497    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d3800ace-ae94-47da-ac65-218d3830b13d-istio-envoy\") pod \"e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-vqsxn\" (UID: \"d3800ace-ae94-47da-ac65-218d3830b13d\") " pod="openshift-ingress/e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-vqsxn"
Oct 31 00:49:20.075796 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:49:20.075774    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d3800ace-ae94-47da-ac65-218d3830b13d-istio-data\") pod \"e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-vqsxn\" (UID: \"d3800ace-ae94-47da-ac65-218d3830b13d\") " pod="openshift-ingress/e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-vqsxn"
Oct 31 00:49:20.075986 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:49:20.075968    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d3800ace-ae94-47da-ac65-218d3830b13d-credential-socket\") pod \"e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-vqsxn\" (UID: \"d3800ace-ae94-47da-ac65-218d3830b13d\") " pod="openshift-ingress/e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-vqsxn"
Oct 31 00:49:20.076133 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:49:20.076116    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d3800ace-ae94-47da-ac65-218d3830b13d-workload-socket\") pod \"e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-vqsxn\" (UID: \"d3800ace-ae94-47da-ac65-218d3830b13d\") " pod="openshift-ingress/e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-vqsxn"
Oct 31 00:49:20.091342 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:49:20.091304    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d3800ace-ae94-47da-ac65-218d3830b13d-istiod-ca-cert\") pod \"e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-vqsxn\" (UID: \"d3800ace-ae94-47da-ac65-218d3830b13d\") " pod="openshift-ingress/e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-vqsxn"
Oct 31 00:49:20.091646 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:49:20.091623    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d3800ace-ae94-47da-ac65-218d3830b13d-istio-podinfo\") pod \"e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-vqsxn\" (UID: \"d3800ace-ae94-47da-ac65-218d3830b13d\") " pod="openshift-ingress/e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-vqsxn"
Oct 31 00:49:20.094179 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:49:20.094142    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d3800ace-ae94-47da-ac65-218d3830b13d-istiod-ca-cert\") pod \"e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-vqsxn\" (UID: \"d3800ace-ae94-47da-ac65-218d3830b13d\") " pod="openshift-ingress/e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-vqsxn"
Oct 31 00:49:20.094345 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:49:20.074685    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d3800ace-ae94-47da-ac65-218d3830b13d-workload-certs\") pod \"e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-vqsxn\" (UID: \"d3800ace-ae94-47da-ac65-218d3830b13d\") " pod="openshift-ingress/e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-vqsxn"
Oct 31 00:49:20.094866 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:49:20.094843    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d3800ace-ae94-47da-ac65-218d3830b13d-istio-data\") pod \"e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-vqsxn\" (UID: \"d3800ace-ae94-47da-ac65-218d3830b13d\") " pod="openshift-ingress/e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-vqsxn"
Oct 31 00:49:20.190238 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:49:20.096992    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d3800ace-ae94-47da-ac65-218d3830b13d-credential-socket\") pod \"e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-vqsxn\" (UID: \"d3800ace-ae94-47da-ac65-218d3830b13d\") " pod="openshift-ingress/e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-vqsxn"
Oct 31 00:49:20.190524 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:49:20.100929    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d3800ace-ae94-47da-ac65-218d3830b13d-workload-socket\") pod \"e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-vqsxn\" (UID: \"d3800ace-ae94-47da-ac65-218d3830b13d\") " pod="openshift-ingress/e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-vqsxn"
Oct 31 00:49:20.190876 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:49:20.187562    4284 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-ef2572/distinct-2-w4-ef2572-kserve-547fccbd5-dfpfx" podUID="cb311a47-264c-4b33-bc31-377b94b0c59b" containerName="main" probeResult="failure" output="Get \"https://10.134.0.73:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:49:20.220737 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:49:20.199321    4284 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-b63c24/premium-w1-b63c24-kserve-549f786d9-k7g8j" podUID="b6846d68-7b58-4637-a47f-cd55bbf79b98" containerName="main" probeResult="failure" output="Get \"https://10.134.0.59:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:49:20.245240 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:49:20.224988    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kg8bn\" (UniqueName: \"kubernetes.io/projected/d3800ace-ae94-47da-ac65-218d3830b13d-kube-api-access-kg8bn\") pod \"e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-vqsxn\" (UID: \"d3800ace-ae94-47da-ac65-218d3830b13d\") " pod="openshift-ingress/e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-vqsxn"
Oct 31 00:49:20.254394 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:49:20.254357    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d3800ace-ae94-47da-ac65-218d3830b13d-istio-envoy\") pod \"e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-vqsxn\" (UID: \"d3800ace-ae94-47da-ac65-218d3830b13d\") " pod="openshift-ingress/e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-vqsxn"
Oct 31 00:49:20.302286 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:49:20.300131    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d3800ace-ae94-47da-ac65-218d3830b13d-istio-token\") pod \"e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-vqsxn\" (UID: \"d3800ace-ae94-47da-ac65-218d3830b13d\") " pod="openshift-ingress/e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-vqsxn"
Oct 31 00:49:20.343251 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:49:20.328398    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-b63c24/premium-w1-b63c24-kserve-549f786d9-k7g8j"
Oct 31 00:49:20.398955 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:49:20.398916    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d3800ace-ae94-47da-ac65-218d3830b13d-istio-podinfo\") pod \"e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-vqsxn\" (UID: \"d3800ace-ae94-47da-ac65-218d3830b13d\") " pod="openshift-ingress/e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-vqsxn"
Oct 31 00:49:20.576311 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:49:20.573478    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-vqsxn"
Oct 31 00:49:20.727191 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:49:20.727146    4284 patch_prober.go:29] interesting pod/e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-522s8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.78:15021/healthz/ready\": dial tcp 10.134.0.78:15021: connect: connection refused" start-of-body=
Oct 31 00:49:20.727525 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:49:20.727463    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-522s8" podUID="7cfa573e-ec98-4eaf-b12f-01f723e5e878" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.78:15021/healthz/ready\": dial tcp 10.134.0.78:15021: connect: connection refused"
Oct 31 00:49:20.850753 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:49:20.850723    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-ef2572/distinct-2-w4-ef2572-kserve-547fccbd5-dfpfx"
Oct 31 00:49:21.017035 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:49:21.017002    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-b63c24/premium-w1-b63c24-kserve-549f786d9-k7g8j"
Oct 31 00:49:21.460348 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:49:21.460313    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-vqsxn"]
Oct 31 00:49:21.462335 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:49:21.462302    4284 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd3800ace_ae94_47da_ac65_218d3830b13d.slice/crio-a12058e8f0f7027e584d94cec8efb1aab88277a2733f3de23399a1fc7afef8fa WatchSource:0}: Error finding container a12058e8f0f7027e584d94cec8efb1aab88277a2733f3de23399a1fc7afef8fa: Status 404 returned error can't find the container with id a12058e8f0f7027e584d94cec8efb1aab88277a2733f3de23399a1fc7afef8fa
Oct 31 00:49:21.465319 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:49:21.465284    4284 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 00:49:21.465438 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:49:21.465351    4284 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 00:49:21.465438 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:49:21.465377    4284 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 00:49:21.725893 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:49:21.725817    4284 patch_prober.go:29] interesting pod/e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-522s8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.78:15021/healthz/ready\": dial tcp 10.134.0.78:15021: connect: connection refused" start-of-body=
Oct 31 00:49:21.725893 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:49:21.725876    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-522s8" podUID="7cfa573e-ec98-4eaf-b12f-01f723e5e878" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.78:15021/healthz/ready\": dial tcp 10.134.0.78:15021: connect: connection refused"
Oct 31 00:49:22.248239 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:49:22.247282    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-vqsxn" event={"ID":"d3800ace-ae94-47da-ac65-218d3830b13d","Type":"ContainerStarted","Data":"5ced87cca8c6f86b39c0ff7814529364b6bb73441ac5e8905e92c7629fe5d76d"}
Oct 31 00:49:22.248239 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:49:22.247349    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-vqsxn" event={"ID":"d3800ace-ae94-47da-ac65-218d3830b13d","Type":"ContainerStarted","Data":"a12058e8f0f7027e584d94cec8efb1aab88277a2733f3de23399a1fc7afef8fa"}
Oct 31 00:49:22.365944 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:49:22.365882    4284 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-vqsxn" podStartSLOduration=3.365863001 podStartE2EDuration="3.365863001s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 00:49:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:49:22.365394692 +0000 UTC m=+1276.510401612" watchObservedRunningTime="2026-10-31 00:49:22.365863001 +0000 UTC m=+1276.510869920"
Oct 31 00:49:22.603298 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:49:22.602598    4284 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-vqsxn"
Oct 31 00:49:22.612311 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:49:22.604951    4284 patch_prober.go:29] interesting pod/e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-vqsxn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.79:15021/healthz/ready\": dial tcp 10.134.0.79:15021: connect: connection refused" start-of-body=
Oct 31 00:49:22.612311 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:49:22.605027    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-vqsxn" podUID="d3800ace-ae94-47da-ac65-218d3830b13d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.79:15021/healthz/ready\": dial tcp 10.134.0.79:15021: connect: connection refused"
Oct 31 00:49:22.726051 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:49:22.726013    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-522s8"
Oct 31 00:49:23.617236 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:49:23.616737    4284 patch_prober.go:29] interesting pod/e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-vqsxn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.79:15021/healthz/ready\": dial tcp 10.134.0.79:15021: connect: connection refused" start-of-body=
Oct 31 00:49:23.617236 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:49:23.616804    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-vqsxn" podUID="d3800ace-ae94-47da-ac65-218d3830b13d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.79:15021/healthz/ready\": dial tcp 10.134.0.79:15021: connect: connection refused"
Oct 31 00:49:23.769417 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:49:23.769366    4284 patch_prober.go:29] interesting pod/e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-522s8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.78:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 31 00:49:23.769604 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:49:23.769441    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-522s8" podUID="7cfa573e-ec98-4eaf-b12f-01f723e5e878" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.78:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:49:24.056642 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:49:24.056606    4284 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-522s8"
Oct 31 00:49:24.058809 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:49:24.058771    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-522s8"
Oct 31 00:49:24.166638 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:49:24.166609    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:49:24.601765 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:49:24.601725    4284 patch_prober.go:29] interesting pod/e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-vqsxn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.79:15021/healthz/ready\": dial tcp 10.134.0.79:15021: connect: connection refused" start-of-body=
Oct 31 00:49:24.609995 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:49:24.609351    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-vqsxn" podUID="d3800ace-ae94-47da-ac65-218d3830b13d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.79:15021/healthz/ready\": dial tcp 10.134.0.79:15021: connect: connection refused"
Oct 31 00:49:24.812834 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:49:24.811812    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:49:25.002911 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:49:25.002880    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:49:25.607236 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:49:25.602610    4284 patch_prober.go:29] interesting pod/e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-vqsxn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.79:15021/healthz/ready\": dial tcp 10.134.0.79:15021: connect: connection refused" start-of-body=
Oct 31 00:49:25.607236 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:49:25.602668    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-vqsxn" podUID="d3800ace-ae94-47da-ac65-218d3830b13d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.79:15021/healthz/ready\": dial tcp 10.134.0.79:15021: connect: connection refused"
Oct 31 00:49:26.221107 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:49:26.221077    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:49:26.377660 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:49:26.377618    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:49:26.530489 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:49:26.530391    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:49:26.610441 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:49:26.610404    4284 patch_prober.go:29] interesting pod/e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-vqsxn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.79:15021/healthz/ready\": dial tcp 10.134.0.79:15021: connect: connection refused" start-of-body=
Oct 31 00:49:26.610728 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:49:26.610674    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-vqsxn" podUID="d3800ace-ae94-47da-ac65-218d3830b13d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.79:15021/healthz/ready\": dial tcp 10.134.0.79:15021: connect: connection refused"
Oct 31 00:49:27.601968 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:49:27.601934    4284 patch_prober.go:29] interesting pod/e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-vqsxn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.79:15021/healthz/ready\": dial tcp 10.134.0.79:15021: connect: connection refused" start-of-body=
Oct 31 00:49:27.602412 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:49:27.601990    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-vqsxn" podUID="d3800ace-ae94-47da-ac65-218d3830b13d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.79:15021/healthz/ready\": dial tcp 10.134.0.79:15021: connect: connection refused"
Oct 31 00:49:28.733238 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:49:28.733176    4284 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-vqsxn"
Oct 31 00:49:28.733753 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:49:28.733729    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-vqsxn"
Oct 31 00:49:28.734456 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:49:28.734437    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-vqsxn"
Oct 31 00:49:29.102688 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:49:29.102588    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:49:30.568339 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:49:30.568288    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:49:31.592588 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:49:31.592557    4284 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-7e6fe88b-openshift-default-5b64b75c89-qzgdm"]
Oct 31 00:49:31.596507 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:49:31.596485    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-7e6fe88b-openshift-default-5b64b75c89-qzgdm"
Oct 31 00:49:31.666590 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:49:31.666561    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-7e6fe88b-openshift-default-5b64b75c89-qzgdm"]
Oct 31 00:49:31.672015 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:49:31.671986    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b58c53d0-4acf-4de9-b322-8ead4dc9b013-workload-certs\") pod \"e2e-mt-7e6fe88b-openshift-default-5b64b75c89-qzgdm\" (UID: \"b58c53d0-4acf-4de9-b322-8ead4dc9b013\") " pod="openshift-ingress/e2e-mt-7e6fe88b-openshift-default-5b64b75c89-qzgdm"
Oct 31 00:49:31.672172 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:49:31.672116    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l89vc\" (UniqueName: \"kubernetes.io/projected/b58c53d0-4acf-4de9-b322-8ead4dc9b013-kube-api-access-l89vc\") pod \"e2e-mt-7e6fe88b-openshift-default-5b64b75c89-qzgdm\" (UID: \"b58c53d0-4acf-4de9-b322-8ead4dc9b013\") " pod="openshift-ingress/e2e-mt-7e6fe88b-openshift-default-5b64b75c89-qzgdm"
Oct 31 00:49:31.672172 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:49:31.672148    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b58c53d0-4acf-4de9-b322-8ead4dc9b013-istio-podinfo\") pod \"e2e-mt-7e6fe88b-openshift-default-5b64b75c89-qzgdm\" (UID: \"b58c53d0-4acf-4de9-b322-8ead4dc9b013\") " pod="openshift-ingress/e2e-mt-7e6fe88b-openshift-default-5b64b75c89-qzgdm"
Oct 31 00:49:31.672172 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:49:31.672169    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b58c53d0-4acf-4de9-b322-8ead4dc9b013-istio-token\") pod \"e2e-mt-7e6fe88b-openshift-default-5b64b75c89-qzgdm\" (UID: \"b58c53d0-4acf-4de9-b322-8ead4dc9b013\") " pod="openshift-ingress/e2e-mt-7e6fe88b-openshift-default-5b64b75c89-qzgdm"
Oct 31 00:49:31.672378 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:49:31.672345    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b58c53d0-4acf-4de9-b322-8ead4dc9b013-istio-envoy\") pod \"e2e-mt-7e6fe88b-openshift-default-5b64b75c89-qzgdm\" (UID: \"b58c53d0-4acf-4de9-b322-8ead4dc9b013\") " pod="openshift-ingress/e2e-mt-7e6fe88b-openshift-default-5b64b75c89-qzgdm"
Oct 31 00:49:31.672436 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:49:31.672423    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b58c53d0-4acf-4de9-b322-8ead4dc9b013-credential-socket\") pod \"e2e-mt-7e6fe88b-openshift-default-5b64b75c89-qzgdm\" (UID: \"b58c53d0-4acf-4de9-b322-8ead4dc9b013\") " pod="openshift-ingress/e2e-mt-7e6fe88b-openshift-default-5b64b75c89-qzgdm"
Oct 31 00:49:31.672488 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:49:31.672462    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b58c53d0-4acf-4de9-b322-8ead4dc9b013-istiod-ca-cert\") pod \"e2e-mt-7e6fe88b-openshift-default-5b64b75c89-qzgdm\" (UID: \"b58c53d0-4acf-4de9-b322-8ead4dc9b013\") " pod="openshift-ingress/e2e-mt-7e6fe88b-openshift-default-5b64b75c89-qzgdm"
Oct 31 00:49:31.672601 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:49:31.672584    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b58c53d0-4acf-4de9-b322-8ead4dc9b013-istio-data\") pod \"e2e-mt-7e6fe88b-openshift-default-5b64b75c89-qzgdm\" (UID: \"b58c53d0-4acf-4de9-b322-8ead4dc9b013\") " pod="openshift-ingress/e2e-mt-7e6fe88b-openshift-default-5b64b75c89-qzgdm"
Oct 31 00:49:31.672662 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:49:31.672647    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b58c53d0-4acf-4de9-b322-8ead4dc9b013-workload-socket\") pod \"e2e-mt-7e6fe88b-openshift-default-5b64b75c89-qzgdm\" (UID: \"b58c53d0-4acf-4de9-b322-8ead4dc9b013\") " pod="openshift-ingress/e2e-mt-7e6fe88b-openshift-default-5b64b75c89-qzgdm"
Oct 31 00:49:31.687734 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:49:31.687697    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:49:31.697144 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:49:31.697110    4284 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-7e6fe88b-openshift-default-dockercfg-c2g5f\""
Oct 31 00:49:31.798247 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:49:31.796043    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b58c53d0-4acf-4de9-b322-8ead4dc9b013-istio-data\") pod \"e2e-mt-7e6fe88b-openshift-default-5b64b75c89-qzgdm\" (UID: \"b58c53d0-4acf-4de9-b322-8ead4dc9b013\") " pod="openshift-ingress/e2e-mt-7e6fe88b-openshift-default-5b64b75c89-qzgdm"
Oct 31 00:49:31.798247 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:49:31.796145    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b58c53d0-4acf-4de9-b322-8ead4dc9b013-workload-socket\") pod \"e2e-mt-7e6fe88b-openshift-default-5b64b75c89-qzgdm\" (UID: \"b58c53d0-4acf-4de9-b322-8ead4dc9b013\") " pod="openshift-ingress/e2e-mt-7e6fe88b-openshift-default-5b64b75c89-qzgdm"
Oct 31 00:49:31.798247 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:49:31.796200    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b58c53d0-4acf-4de9-b322-8ead4dc9b013-workload-certs\") pod \"e2e-mt-7e6fe88b-openshift-default-5b64b75c89-qzgdm\" (UID: \"b58c53d0-4acf-4de9-b322-8ead4dc9b013\") " pod="openshift-ingress/e2e-mt-7e6fe88b-openshift-default-5b64b75c89-qzgdm"
Oct 31 00:49:31.798247 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:49:31.796297    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l89vc\" (UniqueName: \"kubernetes.io/projected/b58c53d0-4acf-4de9-b322-8ead4dc9b013-kube-api-access-l89vc\") pod \"e2e-mt-7e6fe88b-openshift-default-5b64b75c89-qzgdm\" (UID: \"b58c53d0-4acf-4de9-b322-8ead4dc9b013\") " pod="openshift-ingress/e2e-mt-7e6fe88b-openshift-default-5b64b75c89-qzgdm"
Oct 31 00:49:31.798247 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:49:31.796325    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b58c53d0-4acf-4de9-b322-8ead4dc9b013-istio-podinfo\") pod \"e2e-mt-7e6fe88b-openshift-default-5b64b75c89-qzgdm\" (UID: \"b58c53d0-4acf-4de9-b322-8ead4dc9b013\") " pod="openshift-ingress/e2e-mt-7e6fe88b-openshift-default-5b64b75c89-qzgdm"
Oct 31 00:49:31.798247 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:49:31.796358    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b58c53d0-4acf-4de9-b322-8ead4dc9b013-istio-token\") pod \"e2e-mt-7e6fe88b-openshift-default-5b64b75c89-qzgdm\" (UID: \"b58c53d0-4acf-4de9-b322-8ead4dc9b013\") " pod="openshift-ingress/e2e-mt-7e6fe88b-openshift-default-5b64b75c89-qzgdm"
Oct 31 00:49:31.798247 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:49:31.796406    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b58c53d0-4acf-4de9-b322-8ead4dc9b013-istio-envoy\") pod \"e2e-mt-7e6fe88b-openshift-default-5b64b75c89-qzgdm\" (UID: \"b58c53d0-4acf-4de9-b322-8ead4dc9b013\") " pod="openshift-ingress/e2e-mt-7e6fe88b-openshift-default-5b64b75c89-qzgdm"
Oct 31 00:49:31.798247 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:49:31.796473    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b58c53d0-4acf-4de9-b322-8ead4dc9b013-credential-socket\") pod \"e2e-mt-7e6fe88b-openshift-default-5b64b75c89-qzgdm\" (UID: \"b58c53d0-4acf-4de9-b322-8ead4dc9b013\") " pod="openshift-ingress/e2e-mt-7e6fe88b-openshift-default-5b64b75c89-qzgdm"
Oct 31 00:49:31.798247 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:49:31.796498    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b58c53d0-4acf-4de9-b322-8ead4dc9b013-istiod-ca-cert\") pod \"e2e-mt-7e6fe88b-openshift-default-5b64b75c89-qzgdm\" (UID: \"b58c53d0-4acf-4de9-b322-8ead4dc9b013\") " pod="openshift-ingress/e2e-mt-7e6fe88b-openshift-default-5b64b75c89-qzgdm"
Oct 31 00:49:31.810237 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:49:31.806543    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b58c53d0-4acf-4de9-b322-8ead4dc9b013-istio-data\") pod \"e2e-mt-7e6fe88b-openshift-default-5b64b75c89-qzgdm\" (UID: \"b58c53d0-4acf-4de9-b322-8ead4dc9b013\") " pod="openshift-ingress/e2e-mt-7e6fe88b-openshift-default-5b64b75c89-qzgdm"
Oct 31 00:49:31.810237 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:49:31.806826    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b58c53d0-4acf-4de9-b322-8ead4dc9b013-workload-socket\") pod \"e2e-mt-7e6fe88b-openshift-default-5b64b75c89-qzgdm\" (UID: \"b58c53d0-4acf-4de9-b322-8ead4dc9b013\") " pod="openshift-ingress/e2e-mt-7e6fe88b-openshift-default-5b64b75c89-qzgdm"
Oct 31 00:49:31.810237 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:49:31.807053    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b58c53d0-4acf-4de9-b322-8ead4dc9b013-workload-certs\") pod \"e2e-mt-7e6fe88b-openshift-default-5b64b75c89-qzgdm\" (UID: \"b58c53d0-4acf-4de9-b322-8ead4dc9b013\") " pod="openshift-ingress/e2e-mt-7e6fe88b-openshift-default-5b64b75c89-qzgdm"
Oct 31 00:49:31.813680 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:49:31.810858    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b58c53d0-4acf-4de9-b322-8ead4dc9b013-credential-socket\") pod \"e2e-mt-7e6fe88b-openshift-default-5b64b75c89-qzgdm\" (UID: \"b58c53d0-4acf-4de9-b322-8ead4dc9b013\") " pod="openshift-ingress/e2e-mt-7e6fe88b-openshift-default-5b64b75c89-qzgdm"
Oct 31 00:49:31.813680 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:49:31.811524    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b58c53d0-4acf-4de9-b322-8ead4dc9b013-istio-envoy\") pod \"e2e-mt-7e6fe88b-openshift-default-5b64b75c89-qzgdm\" (UID: \"b58c53d0-4acf-4de9-b322-8ead4dc9b013\") " pod="openshift-ingress/e2e-mt-7e6fe88b-openshift-default-5b64b75c89-qzgdm"
Oct 31 00:49:31.815807 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:49:31.815781    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b58c53d0-4acf-4de9-b322-8ead4dc9b013-istiod-ca-cert\") pod \"e2e-mt-7e6fe88b-openshift-default-5b64b75c89-qzgdm\" (UID: \"b58c53d0-4acf-4de9-b322-8ead4dc9b013\") " pod="openshift-ingress/e2e-mt-7e6fe88b-openshift-default-5b64b75c89-qzgdm"
Oct 31 00:49:31.834877 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:49:31.834806    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b58c53d0-4acf-4de9-b322-8ead4dc9b013-istio-podinfo\") pod \"e2e-mt-7e6fe88b-openshift-default-5b64b75c89-qzgdm\" (UID: \"b58c53d0-4acf-4de9-b322-8ead4dc9b013\") " pod="openshift-ingress/e2e-mt-7e6fe88b-openshift-default-5b64b75c89-qzgdm"
Oct 31 00:49:31.871636 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:49:31.871557    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b58c53d0-4acf-4de9-b322-8ead4dc9b013-istio-token\") pod \"e2e-mt-7e6fe88b-openshift-default-5b64b75c89-qzgdm\" (UID: \"b58c53d0-4acf-4de9-b322-8ead4dc9b013\") " pod="openshift-ingress/e2e-mt-7e6fe88b-openshift-default-5b64b75c89-qzgdm"
Oct 31 00:49:31.873275 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:49:31.873243    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l89vc\" (UniqueName: \"kubernetes.io/projected/b58c53d0-4acf-4de9-b322-8ead4dc9b013-kube-api-access-l89vc\") pod \"e2e-mt-7e6fe88b-openshift-default-5b64b75c89-qzgdm\" (UID: \"b58c53d0-4acf-4de9-b322-8ead4dc9b013\") " pod="openshift-ingress/e2e-mt-7e6fe88b-openshift-default-5b64b75c89-qzgdm"
Oct 31 00:49:31.909959 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:49:31.909929    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-7e6fe88b-openshift-default-5b64b75c89-qzgdm"
Oct 31 00:49:32.326257 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:49:32.312931    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:49:32.358288 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:49:32.358257    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-7e6fe88b-openshift-default-5b64b75c89-qzgdm"]
Oct 31 00:49:32.366905 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:49:32.366877    4284 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb58c53d0_4acf_4de9_b322_8ead4dc9b013.slice/crio-b73ff651d7dfa72bca7bb44b1c0d6791795457c590fb2a61f744e456517b42a6 WatchSource:0}: Error finding container b73ff651d7dfa72bca7bb44b1c0d6791795457c590fb2a61f744e456517b42a6: Status 404 returned error can't find the container with id b73ff651d7dfa72bca7bb44b1c0d6791795457c590fb2a61f744e456517b42a6
Oct 31 00:49:32.369638 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:49:32.369605    4284 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 00:49:32.369727 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:49:32.369674    4284 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 00:49:32.369727 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:49:32.369704    4284 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 00:49:33.376982 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:49:33.376948    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-7e6fe88b-openshift-default-5b64b75c89-qzgdm" event={"ID":"b58c53d0-4acf-4de9-b322-8ead4dc9b013","Type":"ContainerStarted","Data":"c7da3b2ece1e204bf15a08c3bb655c87bf30646ca899dde9f0835ee35199ee7a"}
Oct 31 00:49:33.377402 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:49:33.377011    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-7e6fe88b-openshift-default-5b64b75c89-qzgdm" event={"ID":"b58c53d0-4acf-4de9-b322-8ead4dc9b013","Type":"ContainerStarted","Data":"b73ff651d7dfa72bca7bb44b1c0d6791795457c590fb2a61f744e456517b42a6"}
Oct 31 00:49:33.544615 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:49:33.544564    4284 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-7e6fe88b-openshift-default-5b64b75c89-qzgdm" podStartSLOduration=2.544548938 podStartE2EDuration="2.544548938s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 00:49:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:49:33.540696439 +0000 UTC m=+1287.685703380" watchObservedRunningTime="2026-10-31 00:49:33.544548938 +0000 UTC m=+1287.689555854"
Oct 31 00:49:33.910091 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:49:33.910056    4284 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-7e6fe88b-openshift-default-5b64b75c89-qzgdm"
Oct 31 00:49:33.910718 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:49:33.910689    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:49:33.915804 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:49:33.915780    4284 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-7e6fe88b-openshift-default-5b64b75c89-qzgdm"
Oct 31 00:49:34.384057 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:49:34.383960    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-7e6fe88b-openshift-default-5b64b75c89-qzgdm"
Oct 31 00:49:34.385089 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:49:34.385070    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-7e6fe88b-openshift-default-5b64b75c89-qzgdm"
Oct 31 00:49:36.564004 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:49:36.563969    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:49:36.775972 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:49:36.775927    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:49:36.929949 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:49:36.929908    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:49:39.850361 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:49:39.850309    4284 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-ef2572/distinct-2-w4-ef2572-kserve-547fccbd5-dfpfx" podUID="cb311a47-264c-4b33-bc31-377b94b0c59b" containerName="main" probeResult="failure" output="Get \"https://10.134.0.73:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:49:40.024402 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:49:40.024345    4284 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-b63c24/premium-w1-b63c24-kserve-549f786d9-k7g8j" podUID="b6846d68-7b58-4637-a47f-cd55bbf79b98" containerName="main" probeResult="failure" output="Get \"https://10.134.0.59:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:49:41.028238 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:49:41.017253    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:49:41.315974 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:49:41.309175    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:49:41.612308 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:49:41.594164    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:49:41.879565 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:49:41.879453    4284 patch_prober.go:29] interesting pod/e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-nc722 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.54:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 31 00:49:41.879889 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:49:41.879855    4284 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-nc722" podUID="e999d104-c9ea-45a7-9930-7f829013c52e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:49:42.679967 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:49:42.679937    4284 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-7e6fe88b-558b6b9b97-292vk"]
Oct 31 00:49:43.024701 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:49:43.024604    4284 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-b63c24/premium-w1-b63c24-kserve-549f786d9-k7g8j" podUID="b6846d68-7b58-4637-a47f-cd55bbf79b98" containerName="main" probeResult="failure" output="Get \"https://10.134.0.59:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:49:43.026618 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:49:43.026583    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-7e6fe88b-558b6b9b97-292vk"]
Oct 31 00:49:43.155257 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:49:43.036913    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-7e6fe88b-558b6b9b97-292vk"
Oct 31 00:49:43.240624 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:49:43.238996    4284 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-7e6fe88b\""
Oct 31 00:49:43.260401 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:49:43.260370    4284 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-7e6fe88b-dockercfg-8mbks\""
Oct 31 00:49:43.482427 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:49:43.481648    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2b942b8c-c3f7-4a18-9dc2-59de27841c78-plugins-config-volume\") pod \"payload-processing-e2e-mt-7e6fe88b-558b6b9b97-292vk\" (UID: \"2b942b8c-c3f7-4a18-9dc2-59de27841c78\") " pod="openshift-ingress/payload-processing-e2e-mt-7e6fe88b-558b6b9b97-292vk"
Oct 31 00:49:43.482427 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:49:43.481979    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qmwcs\" (UniqueName: \"kubernetes.io/projected/2b942b8c-c3f7-4a18-9dc2-59de27841c78-kube-api-access-qmwcs\") pod \"payload-processing-e2e-mt-7e6fe88b-558b6b9b97-292vk\" (UID: \"2b942b8c-c3f7-4a18-9dc2-59de27841c78\") " pod="openshift-ingress/payload-processing-e2e-mt-7e6fe88b-558b6b9b97-292vk"
Oct 31 00:49:43.692386 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:49:43.692350    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qmwcs\" (UniqueName: \"kubernetes.io/projected/2b942b8c-c3f7-4a18-9dc2-59de27841c78-kube-api-access-qmwcs\") pod \"payload-processing-e2e-mt-7e6fe88b-558b6b9b97-292vk\" (UID: \"2b942b8c-c3f7-4a18-9dc2-59de27841c78\") " pod="openshift-ingress/payload-processing-e2e-mt-7e6fe88b-558b6b9b97-292vk"
Oct 31 00:49:43.704747 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:49:43.704715    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2b942b8c-c3f7-4a18-9dc2-59de27841c78-plugins-config-volume\") pod \"payload-processing-e2e-mt-7e6fe88b-558b6b9b97-292vk\" (UID: \"2b942b8c-c3f7-4a18-9dc2-59de27841c78\") " pod="openshift-ingress/payload-processing-e2e-mt-7e6fe88b-558b6b9b97-292vk"
Oct 31 00:49:43.745549 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:49:43.745446    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2b942b8c-c3f7-4a18-9dc2-59de27841c78-plugins-config-volume\") pod \"payload-processing-e2e-mt-7e6fe88b-558b6b9b97-292vk\" (UID: \"2b942b8c-c3f7-4a18-9dc2-59de27841c78\") " pod="openshift-ingress/payload-processing-e2e-mt-7e6fe88b-558b6b9b97-292vk"
Oct 31 00:49:43.898071 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:49:43.898030    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qmwcs\" (UniqueName: \"kubernetes.io/projected/2b942b8c-c3f7-4a18-9dc2-59de27841c78-kube-api-access-qmwcs\") pod \"payload-processing-e2e-mt-7e6fe88b-558b6b9b97-292vk\" (UID: \"2b942b8c-c3f7-4a18-9dc2-59de27841c78\") " pod="openshift-ingress/payload-processing-e2e-mt-7e6fe88b-558b6b9b97-292vk"
Oct 31 00:49:44.103296 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:49:44.044545    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-7e6fe88b-558b6b9b97-292vk"
Oct 31 00:49:44.326463 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:49:44.324568    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:49:44.764981 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:49:44.759623    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:49:44.868999 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:49:44.868958    4284 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-ef2572/distinct-2-w4-ef2572-kserve-547fccbd5-dfpfx" podUID="cb311a47-264c-4b33-bc31-377b94b0c59b" containerName="main" probeResult="failure" output="Get \"https://10.134.0.73:8000/ready\": context deadline exceeded"
Oct 31 00:49:45.013752 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:49:45.013602    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:49:45.710270 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:49:45.710236    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-7e6fe88b-558b6b9b97-292vk"]
Oct 31 00:49:45.712329 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:49:45.712297    4284 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2b942b8c_c3f7_4a18_9dc2_59de27841c78.slice/crio-42737623750b7d7d206a5e0d5c357d319103924eda7122e297ccb82ca6845857 WatchSource:0}: Error finding container 42737623750b7d7d206a5e0d5c357d319103924eda7122e297ccb82ca6845857: Status 404 returned error can't find the container with id 42737623750b7d7d206a5e0d5c357d319103924eda7122e297ccb82ca6845857
Oct 31 00:49:46.062758 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:49:46.062658    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-7e6fe88b-558b6b9b97-292vk" event={"ID":"2b942b8c-c3f7-4a18-9dc2-59de27841c78","Type":"ContainerStarted","Data":"89c7398f23611b40d68ef9510e3a99335569b9cbe0c5d69c1dd266e52ae2a873"}
Oct 31 00:49:46.062758 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:49:46.062720    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-7e6fe88b-558b6b9b97-292vk" event={"ID":"2b942b8c-c3f7-4a18-9dc2-59de27841c78","Type":"ContainerStarted","Data":"42737623750b7d7d206a5e0d5c357d319103924eda7122e297ccb82ca6845857"}
Oct 31 00:49:46.062758 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:49:46.062751    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-7e6fe88b-558b6b9b97-292vk"
Oct 31 00:49:46.186708 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:49:46.185292    4284 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-7e6fe88b-558b6b9b97-292vk" podStartSLOduration=4.181206921 podStartE2EDuration="4.181206921s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 00:49:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:49:46.177608112 +0000 UTC m=+1300.322615029" watchObservedRunningTime="2026-10-31 00:49:46.181206921 +0000 UTC m=+1300.326213838"
Oct 31 00:49:47.016273 ip-10-0-132-173 kubenswrapper[4284]: E1031 00:49:47.016152    4284 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fd8sp" containerName="extractor"
Oct 31 00:49:47.872990 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:49:47.872951    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:49:51.017674 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:49:51.017633    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:49:52.718557 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:49:52.718519    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:49:56.183582 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:49:56.183536    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:49:56.333306 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:49:56.333261    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:49:57.072043 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:49:57.072012    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-7e6fe88b-558b6b9b97-292vk"
Oct 31 00:49:57.636988 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:49:57.636957    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:49:59.947229 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:49:59.947174    4284 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-4s7kn"]
Oct 31 00:49:59.951337 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:49:59.951316    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-4s7kn"
Oct 31 00:50:00.053844 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:50:00.053806    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-4s7kn"]
Oct 31 00:50:00.055991 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:50:00.055963    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6ptrj\" (UniqueName: \"kubernetes.io/projected/2cba1971-72c7-4516-9ea2-83e31bc9fc5a-kube-api-access-6ptrj\") pod \"e2e-worker-w4-ef2572-openshift-default-7649dd87-4s7kn\" (UID: \"2cba1971-72c7-4516-9ea2-83e31bc9fc5a\") " pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-4s7kn"
Oct 31 00:50:00.056460 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:50:00.056422    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2cba1971-72c7-4516-9ea2-83e31bc9fc5a-credential-socket\") pod \"e2e-worker-w4-ef2572-openshift-default-7649dd87-4s7kn\" (UID: \"2cba1971-72c7-4516-9ea2-83e31bc9fc5a\") " pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-4s7kn"
Oct 31 00:50:00.056582 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:50:00.056512    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2cba1971-72c7-4516-9ea2-83e31bc9fc5a-istio-envoy\") pod \"e2e-worker-w4-ef2572-openshift-default-7649dd87-4s7kn\" (UID: \"2cba1971-72c7-4516-9ea2-83e31bc9fc5a\") " pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-4s7kn"
Oct 31 00:50:00.056582 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:50:00.056552    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2cba1971-72c7-4516-9ea2-83e31bc9fc5a-istiod-ca-cert\") pod \"e2e-worker-w4-ef2572-openshift-default-7649dd87-4s7kn\" (UID: \"2cba1971-72c7-4516-9ea2-83e31bc9fc5a\") " pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-4s7kn"
Oct 31 00:50:00.056704 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:50:00.056622    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2cba1971-72c7-4516-9ea2-83e31bc9fc5a-istio-token\") pod \"e2e-worker-w4-ef2572-openshift-default-7649dd87-4s7kn\" (UID: \"2cba1971-72c7-4516-9ea2-83e31bc9fc5a\") " pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-4s7kn"
Oct 31 00:50:00.056764 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:50:00.056745    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2cba1971-72c7-4516-9ea2-83e31bc9fc5a-istio-data\") pod \"e2e-worker-w4-ef2572-openshift-default-7649dd87-4s7kn\" (UID: \"2cba1971-72c7-4516-9ea2-83e31bc9fc5a\") " pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-4s7kn"
Oct 31 00:50:00.056818 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:50:00.056779    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2cba1971-72c7-4516-9ea2-83e31bc9fc5a-istio-podinfo\") pod \"e2e-worker-w4-ef2572-openshift-default-7649dd87-4s7kn\" (UID: \"2cba1971-72c7-4516-9ea2-83e31bc9fc5a\") " pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-4s7kn"
Oct 31 00:50:00.056871 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:50:00.056849    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2cba1971-72c7-4516-9ea2-83e31bc9fc5a-workload-socket\") pod \"e2e-worker-w4-ef2572-openshift-default-7649dd87-4s7kn\" (UID: \"2cba1971-72c7-4516-9ea2-83e31bc9fc5a\") " pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-4s7kn"
Oct 31 00:50:00.056980 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:50:00.056958    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2cba1971-72c7-4516-9ea2-83e31bc9fc5a-workload-certs\") pod \"e2e-worker-w4-ef2572-openshift-default-7649dd87-4s7kn\" (UID: \"2cba1971-72c7-4516-9ea2-83e31bc9fc5a\") " pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-4s7kn"
Oct 31 00:50:00.158541 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:50:00.158496    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2cba1971-72c7-4516-9ea2-83e31bc9fc5a-workload-socket\") pod \"e2e-worker-w4-ef2572-openshift-default-7649dd87-4s7kn\" (UID: \"2cba1971-72c7-4516-9ea2-83e31bc9fc5a\") " pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-4s7kn"
Oct 31 00:50:00.158728 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:50:00.158584    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2cba1971-72c7-4516-9ea2-83e31bc9fc5a-workload-certs\") pod \"e2e-worker-w4-ef2572-openshift-default-7649dd87-4s7kn\" (UID: \"2cba1971-72c7-4516-9ea2-83e31bc9fc5a\") " pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-4s7kn"
Oct 31 00:50:00.158728 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:50:00.158642    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6ptrj\" (UniqueName: \"kubernetes.io/projected/2cba1971-72c7-4516-9ea2-83e31bc9fc5a-kube-api-access-6ptrj\") pod \"e2e-worker-w4-ef2572-openshift-default-7649dd87-4s7kn\" (UID: \"2cba1971-72c7-4516-9ea2-83e31bc9fc5a\") " pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-4s7kn"
Oct 31 00:50:00.158728 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:50:00.158675    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2cba1971-72c7-4516-9ea2-83e31bc9fc5a-credential-socket\") pod \"e2e-worker-w4-ef2572-openshift-default-7649dd87-4s7kn\" (UID: \"2cba1971-72c7-4516-9ea2-83e31bc9fc5a\") " pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-4s7kn"
Oct 31 00:50:00.158728 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:50:00.158709    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2cba1971-72c7-4516-9ea2-83e31bc9fc5a-istio-envoy\") pod \"e2e-worker-w4-ef2572-openshift-default-7649dd87-4s7kn\" (UID: \"2cba1971-72c7-4516-9ea2-83e31bc9fc5a\") " pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-4s7kn"
Oct 31 00:50:00.158728 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:50:00.158730    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2cba1971-72c7-4516-9ea2-83e31bc9fc5a-istiod-ca-cert\") pod \"e2e-worker-w4-ef2572-openshift-default-7649dd87-4s7kn\" (UID: \"2cba1971-72c7-4516-9ea2-83e31bc9fc5a\") " pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-4s7kn"
Oct 31 00:50:00.158982 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:50:00.158858    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2cba1971-72c7-4516-9ea2-83e31bc9fc5a-istio-token\") pod \"e2e-worker-w4-ef2572-openshift-default-7649dd87-4s7kn\" (UID: \"2cba1971-72c7-4516-9ea2-83e31bc9fc5a\") " pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-4s7kn"
Oct 31 00:50:00.158982 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:50:00.158958    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2cba1971-72c7-4516-9ea2-83e31bc9fc5a-istio-data\") pod \"e2e-worker-w4-ef2572-openshift-default-7649dd87-4s7kn\" (UID: \"2cba1971-72c7-4516-9ea2-83e31bc9fc5a\") " pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-4s7kn"
Oct 31 00:50:00.158982 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:50:00.158967    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2cba1971-72c7-4516-9ea2-83e31bc9fc5a-workload-socket\") pod \"e2e-worker-w4-ef2572-openshift-default-7649dd87-4s7kn\" (UID: \"2cba1971-72c7-4516-9ea2-83e31bc9fc5a\") " pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-4s7kn"
Oct 31 00:50:00.159139 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:50:00.158980    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2cba1971-72c7-4516-9ea2-83e31bc9fc5a-workload-certs\") pod \"e2e-worker-w4-ef2572-openshift-default-7649dd87-4s7kn\" (UID: \"2cba1971-72c7-4516-9ea2-83e31bc9fc5a\") " pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-4s7kn"
Oct 31 00:50:00.159139 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:50:00.158995    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2cba1971-72c7-4516-9ea2-83e31bc9fc5a-istio-podinfo\") pod \"e2e-worker-w4-ef2572-openshift-default-7649dd87-4s7kn\" (UID: \"2cba1971-72c7-4516-9ea2-83e31bc9fc5a\") " pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-4s7kn"
Oct 31 00:50:00.159139 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:50:00.159102    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2cba1971-72c7-4516-9ea2-83e31bc9fc5a-credential-socket\") pod \"e2e-worker-w4-ef2572-openshift-default-7649dd87-4s7kn\" (UID: \"2cba1971-72c7-4516-9ea2-83e31bc9fc5a\") " pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-4s7kn"
Oct 31 00:50:00.159309 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:50:00.159287    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2cba1971-72c7-4516-9ea2-83e31bc9fc5a-istio-data\") pod \"e2e-worker-w4-ef2572-openshift-default-7649dd87-4s7kn\" (UID: \"2cba1971-72c7-4516-9ea2-83e31bc9fc5a\") " pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-4s7kn"
Oct 31 00:50:00.159633 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:50:00.159607    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2cba1971-72c7-4516-9ea2-83e31bc9fc5a-istiod-ca-cert\") pod \"e2e-worker-w4-ef2572-openshift-default-7649dd87-4s7kn\" (UID: \"2cba1971-72c7-4516-9ea2-83e31bc9fc5a\") " pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-4s7kn"
Oct 31 00:50:00.161439 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:50:00.161416    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2cba1971-72c7-4516-9ea2-83e31bc9fc5a-istio-envoy\") pod \"e2e-worker-w4-ef2572-openshift-default-7649dd87-4s7kn\" (UID: \"2cba1971-72c7-4516-9ea2-83e31bc9fc5a\") " pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-4s7kn"
Oct 31 00:50:00.161622 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:50:00.161600    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2cba1971-72c7-4516-9ea2-83e31bc9fc5a-istio-podinfo\") pod \"e2e-worker-w4-ef2572-openshift-default-7649dd87-4s7kn\" (UID: \"2cba1971-72c7-4516-9ea2-83e31bc9fc5a\") " pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-4s7kn"
Oct 31 00:50:00.218079 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:50:00.218007    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2cba1971-72c7-4516-9ea2-83e31bc9fc5a-istio-token\") pod \"e2e-worker-w4-ef2572-openshift-default-7649dd87-4s7kn\" (UID: \"2cba1971-72c7-4516-9ea2-83e31bc9fc5a\") " pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-4s7kn"
Oct 31 00:50:00.218241 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:50:00.218130    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6ptrj\" (UniqueName: \"kubernetes.io/projected/2cba1971-72c7-4516-9ea2-83e31bc9fc5a-kube-api-access-6ptrj\") pod \"e2e-worker-w4-ef2572-openshift-default-7649dd87-4s7kn\" (UID: \"2cba1971-72c7-4516-9ea2-83e31bc9fc5a\") " pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-4s7kn"
Oct 31 00:50:00.267383 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:50:00.267350    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-4s7kn"
Oct 31 00:50:00.690027 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:50:00.690001    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-4s7kn"]
Oct 31 00:50:00.690431 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:50:00.690409    4284 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2cba1971_72c7_4516_9ea2_83e31bc9fc5a.slice/crio-178e4456c35b125b26b2323bee8615bc41094646b619cefe313ce831bb5271d0 WatchSource:0}: Error finding container 178e4456c35b125b26b2323bee8615bc41094646b619cefe313ce831bb5271d0: Status 404 returned error can't find the container with id 178e4456c35b125b26b2323bee8615bc41094646b619cefe313ce831bb5271d0
Oct 31 00:50:00.692980 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:50:00.692949    4284 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 00:50:00.693080 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:50:00.693026    4284 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 00:50:00.693080 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:50:00.693073    4284 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 00:50:01.166929 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:50:01.166889    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-4s7kn" event={"ID":"2cba1971-72c7-4516-9ea2-83e31bc9fc5a","Type":"ContainerStarted","Data":"15d2b54f9369b27c98788b30d52997a999475b5874b1faad5c8fe2b56b839ccb"}
Oct 31 00:50:01.167604 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:50:01.167546    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-4s7kn" event={"ID":"2cba1971-72c7-4516-9ea2-83e31bc9fc5a","Type":"ContainerStarted","Data":"178e4456c35b125b26b2323bee8615bc41094646b619cefe313ce831bb5271d0"}
Oct 31 00:50:01.268568 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:50:01.268487    4284 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-4s7kn"
Oct 31 00:50:01.270125 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:50:01.270099    4284 patch_prober.go:29] interesting pod/e2e-worker-w4-ef2572-openshift-default-7649dd87-4s7kn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.82:15021/healthz/ready\": dial tcp 10.134.0.82:15021: connect: connection refused" start-of-body=
Oct 31 00:50:01.270312 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:50:01.270143    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-4s7kn" podUID="2cba1971-72c7-4516-9ea2-83e31bc9fc5a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.82:15021/healthz/ready\": dial tcp 10.134.0.82:15021: connect: connection refused"
Oct 31 00:50:01.297238 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:50:01.286780    4284 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-4s7kn" podStartSLOduration=2.286757853 podStartE2EDuration="2.286757853s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 00:49:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:50:01.284311602 +0000 UTC m=+1315.429318510" watchObservedRunningTime="2026-10-31 00:50:01.286757853 +0000 UTC m=+1315.431764782"
Oct 31 00:50:02.267785 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:50:02.267744    4284 patch_prober.go:29] interesting pod/e2e-worker-w4-ef2572-openshift-default-7649dd87-4s7kn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.82:15021/healthz/ready\": dial tcp 10.134.0.82:15021: connect: connection refused" start-of-body=
Oct 31 00:50:02.268319 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:50:02.267804    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-4s7kn" podUID="2cba1971-72c7-4516-9ea2-83e31bc9fc5a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.82:15021/healthz/ready\": dial tcp 10.134.0.82:15021: connect: connection refused"
Oct 31 00:50:02.529540 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:50:02.529443    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:50:03.267959 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:50:03.267925    4284 patch_prober.go:29] interesting pod/e2e-worker-w4-ef2572-openshift-default-7649dd87-4s7kn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.82:15021/healthz/ready\": dial tcp 10.134.0.82:15021: connect: connection refused" start-of-body=
Oct 31 00:50:03.268364 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:50:03.267974    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-4s7kn" podUID="2cba1971-72c7-4516-9ea2-83e31bc9fc5a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.82:15021/healthz/ready\": dial tcp 10.134.0.82:15021: connect: connection refused"
Oct 31 00:50:04.268649 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:50:04.268611    4284 patch_prober.go:29] interesting pod/e2e-worker-w4-ef2572-openshift-default-7649dd87-4s7kn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.82:15021/healthz/ready\": dial tcp 10.134.0.82:15021: connect: connection refused" start-of-body=
Oct 31 00:50:04.269060 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:50:04.268669    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-4s7kn" podUID="2cba1971-72c7-4516-9ea2-83e31bc9fc5a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.82:15021/healthz/ready\": dial tcp 10.134.0.82:15021: connect: connection refused"
Oct 31 00:50:05.267791 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:50:05.267752    4284 patch_prober.go:29] interesting pod/e2e-worker-w4-ef2572-openshift-default-7649dd87-4s7kn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.82:15021/healthz/ready\": dial tcp 10.134.0.82:15021: connect: connection refused" start-of-body=
Oct 31 00:50:05.267997 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:50:05.267829    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-4s7kn" podUID="2cba1971-72c7-4516-9ea2-83e31bc9fc5a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.82:15021/healthz/ready\": dial tcp 10.134.0.82:15021: connect: connection refused"
Oct 31 00:50:06.268709 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:50:06.268670    4284 patch_prober.go:29] interesting pod/e2e-worker-w4-ef2572-openshift-default-7649dd87-4s7kn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.82:15021/healthz/ready\": dial tcp 10.134.0.82:15021: connect: connection refused" start-of-body=
Oct 31 00:50:06.269112 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:50:06.268742    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-4s7kn" podUID="2cba1971-72c7-4516-9ea2-83e31bc9fc5a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.82:15021/healthz/ready\": dial tcp 10.134.0.82:15021: connect: connection refused"
Oct 31 00:50:07.268520 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:50:07.268483    4284 patch_prober.go:29] interesting pod/e2e-worker-w4-ef2572-openshift-default-7649dd87-4s7kn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.82:15021/healthz/ready\": dial tcp 10.134.0.82:15021: connect: connection refused" start-of-body=
Oct 31 00:50:07.268723 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:50:07.268555    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-4s7kn" podUID="2cba1971-72c7-4516-9ea2-83e31bc9fc5a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.82:15021/healthz/ready\": dial tcp 10.134.0.82:15021: connect: connection refused"
Oct 31 00:50:07.304310 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:50:07.304270    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:50:08.273078 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:50:08.273048    4284 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-4s7kn"
Oct 31 00:50:08.273450 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:50:08.273430    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-4s7kn"
Oct 31 00:50:08.274295 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:50:08.274274    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-4s7kn"
Oct 31 00:50:13.508240 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:50:13.507045    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:50:13.710618 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:50:13.710577    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:50:14.140967 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:50:14.140927    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:50:14.931826 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:50:14.929488    4284 patch_prober.go:29] interesting pod/e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-vqsxn container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.79:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 31 00:50:14.931826 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:50:14.929565    4284 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-vqsxn" podUID="d3800ace-ae94-47da-ac65-218d3830b13d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.79:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:50:14.954257 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:50:14.954186    4284 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-ef2572/distinct-2-w4-ef2572-kserve-547fccbd5-dfpfx" podUID="cb311a47-264c-4b33-bc31-377b94b0c59b" containerName="main" probeResult="failure" output="Get \"https://10.134.0.73:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:50:15.055963 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:50:15.041472    4284 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-b63c24/premium-w1-b63c24-kserve-549f786d9-k7g8j" podUID="b6846d68-7b58-4637-a47f-cd55bbf79b98" containerName="main" probeResult="failure" output="Get \"https://10.134.0.59:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:50:16.082335 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:50:16.082263    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:50:16.763787 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:50:16.763753    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:50:16.777877 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:50:16.777845    4284 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-8c2357-openshift-default-659d454f6f-chhkx"]
Oct 31 00:50:16.877461 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:50:16.866647    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-8c2357-openshift-default-659d454f6f-chhkx"]
Oct 31 00:50:16.877461 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:50:16.866786    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-8c2357-openshift-default-659d454f6f-chhkx"
Oct 31 00:50:16.974250 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:50:16.954963    4284 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-8c2357-openshift-default-dockercfg-frnml\""
Oct 31 00:50:17.021316 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:50:16.992101    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/41ab0dd0-6d29-4caf-b5b8-35255519b562-istio-envoy\") pod \"e2e-api-a-8c2357-openshift-default-659d454f6f-chhkx\" (UID: \"41ab0dd0-6d29-4caf-b5b8-35255519b562\") " pod="openshift-ingress/e2e-api-a-8c2357-openshift-default-659d454f6f-chhkx"
Oct 31 00:50:17.022347 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:50:17.022312    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/41ab0dd0-6d29-4caf-b5b8-35255519b562-istiod-ca-cert\") pod \"e2e-api-a-8c2357-openshift-default-659d454f6f-chhkx\" (UID: \"41ab0dd0-6d29-4caf-b5b8-35255519b562\") " pod="openshift-ingress/e2e-api-a-8c2357-openshift-default-659d454f6f-chhkx"
Oct 31 00:50:17.034393 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:50:17.034360    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/41ab0dd0-6d29-4caf-b5b8-35255519b562-istio-data\") pod \"e2e-api-a-8c2357-openshift-default-659d454f6f-chhkx\" (UID: \"41ab0dd0-6d29-4caf-b5b8-35255519b562\") " pod="openshift-ingress/e2e-api-a-8c2357-openshift-default-659d454f6f-chhkx"
Oct 31 00:50:17.060922 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:50:17.034633    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/41ab0dd0-6d29-4caf-b5b8-35255519b562-workload-socket\") pod \"e2e-api-a-8c2357-openshift-default-659d454f6f-chhkx\" (UID: \"41ab0dd0-6d29-4caf-b5b8-35255519b562\") " pod="openshift-ingress/e2e-api-a-8c2357-openshift-default-659d454f6f-chhkx"
Oct 31 00:50:17.060922 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:50:17.041303    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/41ab0dd0-6d29-4caf-b5b8-35255519b562-workload-certs\") pod \"e2e-api-a-8c2357-openshift-default-659d454f6f-chhkx\" (UID: \"41ab0dd0-6d29-4caf-b5b8-35255519b562\") " pod="openshift-ingress/e2e-api-a-8c2357-openshift-default-659d454f6f-chhkx"
Oct 31 00:50:17.060922 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:50:17.041441    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/41ab0dd0-6d29-4caf-b5b8-35255519b562-istio-podinfo\") pod \"e2e-api-a-8c2357-openshift-default-659d454f6f-chhkx\" (UID: \"41ab0dd0-6d29-4caf-b5b8-35255519b562\") " pod="openshift-ingress/e2e-api-a-8c2357-openshift-default-659d454f6f-chhkx"
Oct 31 00:50:17.060922 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:50:17.041487    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p7t4c\" (UniqueName: \"kubernetes.io/projected/41ab0dd0-6d29-4caf-b5b8-35255519b562-kube-api-access-p7t4c\") pod \"e2e-api-a-8c2357-openshift-default-659d454f6f-chhkx\" (UID: \"41ab0dd0-6d29-4caf-b5b8-35255519b562\") " pod="openshift-ingress/e2e-api-a-8c2357-openshift-default-659d454f6f-chhkx"
Oct 31 00:50:17.060922 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:50:17.041565    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/41ab0dd0-6d29-4caf-b5b8-35255519b562-credential-socket\") pod \"e2e-api-a-8c2357-openshift-default-659d454f6f-chhkx\" (UID: \"41ab0dd0-6d29-4caf-b5b8-35255519b562\") " pod="openshift-ingress/e2e-api-a-8c2357-openshift-default-659d454f6f-chhkx"
Oct 31 00:50:17.060922 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:50:17.041675    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/41ab0dd0-6d29-4caf-b5b8-35255519b562-istio-token\") pod \"e2e-api-a-8c2357-openshift-default-659d454f6f-chhkx\" (UID: \"41ab0dd0-6d29-4caf-b5b8-35255519b562\") " pod="openshift-ingress/e2e-api-a-8c2357-openshift-default-659d454f6f-chhkx"
Oct 31 00:50:17.098735 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:50:17.098700    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:50:17.185277 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:50:17.180236    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/41ab0dd0-6d29-4caf-b5b8-35255519b562-istio-podinfo\") pod \"e2e-api-a-8c2357-openshift-default-659d454f6f-chhkx\" (UID: \"41ab0dd0-6d29-4caf-b5b8-35255519b562\") " pod="openshift-ingress/e2e-api-a-8c2357-openshift-default-659d454f6f-chhkx"
Oct 31 00:50:17.185277 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:50:17.180295    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p7t4c\" (UniqueName: \"kubernetes.io/projected/41ab0dd0-6d29-4caf-b5b8-35255519b562-kube-api-access-p7t4c\") pod \"e2e-api-a-8c2357-openshift-default-659d454f6f-chhkx\" (UID: \"41ab0dd0-6d29-4caf-b5b8-35255519b562\") " pod="openshift-ingress/e2e-api-a-8c2357-openshift-default-659d454f6f-chhkx"
Oct 31 00:50:17.185277 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:50:17.180336    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/41ab0dd0-6d29-4caf-b5b8-35255519b562-credential-socket\") pod \"e2e-api-a-8c2357-openshift-default-659d454f6f-chhkx\" (UID: \"41ab0dd0-6d29-4caf-b5b8-35255519b562\") " pod="openshift-ingress/e2e-api-a-8c2357-openshift-default-659d454f6f-chhkx"
Oct 31 00:50:17.185277 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:50:17.180415    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/41ab0dd0-6d29-4caf-b5b8-35255519b562-istio-token\") pod \"e2e-api-a-8c2357-openshift-default-659d454f6f-chhkx\" (UID: \"41ab0dd0-6d29-4caf-b5b8-35255519b562\") " pod="openshift-ingress/e2e-api-a-8c2357-openshift-default-659d454f6f-chhkx"
Oct 31 00:50:17.185277 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:50:17.180509    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/41ab0dd0-6d29-4caf-b5b8-35255519b562-istio-envoy\") pod \"e2e-api-a-8c2357-openshift-default-659d454f6f-chhkx\" (UID: \"41ab0dd0-6d29-4caf-b5b8-35255519b562\") " pod="openshift-ingress/e2e-api-a-8c2357-openshift-default-659d454f6f-chhkx"
Oct 31 00:50:17.185277 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:50:17.180564    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/41ab0dd0-6d29-4caf-b5b8-35255519b562-istiod-ca-cert\") pod \"e2e-api-a-8c2357-openshift-default-659d454f6f-chhkx\" (UID: \"41ab0dd0-6d29-4caf-b5b8-35255519b562\") " pod="openshift-ingress/e2e-api-a-8c2357-openshift-default-659d454f6f-chhkx"
Oct 31 00:50:17.185277 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:50:17.180595    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/41ab0dd0-6d29-4caf-b5b8-35255519b562-istio-data\") pod \"e2e-api-a-8c2357-openshift-default-659d454f6f-chhkx\" (UID: \"41ab0dd0-6d29-4caf-b5b8-35255519b562\") " pod="openshift-ingress/e2e-api-a-8c2357-openshift-default-659d454f6f-chhkx"
Oct 31 00:50:17.185277 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:50:17.180642    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/41ab0dd0-6d29-4caf-b5b8-35255519b562-workload-socket\") pod \"e2e-api-a-8c2357-openshift-default-659d454f6f-chhkx\" (UID: \"41ab0dd0-6d29-4caf-b5b8-35255519b562\") " pod="openshift-ingress/e2e-api-a-8c2357-openshift-default-659d454f6f-chhkx"
Oct 31 00:50:17.185277 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:50:17.180674    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/41ab0dd0-6d29-4caf-b5b8-35255519b562-workload-certs\") pod \"e2e-api-a-8c2357-openshift-default-659d454f6f-chhkx\" (UID: \"41ab0dd0-6d29-4caf-b5b8-35255519b562\") " pod="openshift-ingress/e2e-api-a-8c2357-openshift-default-659d454f6f-chhkx"
Oct 31 00:50:17.185277 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:50:17.181260    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/41ab0dd0-6d29-4caf-b5b8-35255519b562-workload-certs\") pod \"e2e-api-a-8c2357-openshift-default-659d454f6f-chhkx\" (UID: \"41ab0dd0-6d29-4caf-b5b8-35255519b562\") " pod="openshift-ingress/e2e-api-a-8c2357-openshift-default-659d454f6f-chhkx"
Oct 31 00:50:17.222477 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:50:17.222442    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/41ab0dd0-6d29-4caf-b5b8-35255519b562-istio-data\") pod \"e2e-api-a-8c2357-openshift-default-659d454f6f-chhkx\" (UID: \"41ab0dd0-6d29-4caf-b5b8-35255519b562\") " pod="openshift-ingress/e2e-api-a-8c2357-openshift-default-659d454f6f-chhkx"
Oct 31 00:50:17.268837 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:50:17.223412    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/41ab0dd0-6d29-4caf-b5b8-35255519b562-istiod-ca-cert\") pod \"e2e-api-a-8c2357-openshift-default-659d454f6f-chhkx\" (UID: \"41ab0dd0-6d29-4caf-b5b8-35255519b562\") " pod="openshift-ingress/e2e-api-a-8c2357-openshift-default-659d454f6f-chhkx"
Oct 31 00:50:17.269106 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:50:17.223665    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/41ab0dd0-6d29-4caf-b5b8-35255519b562-workload-socket\") pod \"e2e-api-a-8c2357-openshift-default-659d454f6f-chhkx\" (UID: \"41ab0dd0-6d29-4caf-b5b8-35255519b562\") " pod="openshift-ingress/e2e-api-a-8c2357-openshift-default-659d454f6f-chhkx"
Oct 31 00:50:17.269250 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:50:17.223898    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/41ab0dd0-6d29-4caf-b5b8-35255519b562-credential-socket\") pod \"e2e-api-a-8c2357-openshift-default-659d454f6f-chhkx\" (UID: \"41ab0dd0-6d29-4caf-b5b8-35255519b562\") " pod="openshift-ingress/e2e-api-a-8c2357-openshift-default-659d454f6f-chhkx"
Oct 31 00:50:17.466275 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:50:17.435466    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/41ab0dd0-6d29-4caf-b5b8-35255519b562-istio-envoy\") pod \"e2e-api-a-8c2357-openshift-default-659d454f6f-chhkx\" (UID: \"41ab0dd0-6d29-4caf-b5b8-35255519b562\") " pod="openshift-ingress/e2e-api-a-8c2357-openshift-default-659d454f6f-chhkx"
Oct 31 00:50:17.653099 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:50:17.653068    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/41ab0dd0-6d29-4caf-b5b8-35255519b562-istio-podinfo\") pod \"e2e-api-a-8c2357-openshift-default-659d454f6f-chhkx\" (UID: \"41ab0dd0-6d29-4caf-b5b8-35255519b562\") " pod="openshift-ingress/e2e-api-a-8c2357-openshift-default-659d454f6f-chhkx"
Oct 31 00:50:17.662440 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:50:17.662377    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p7t4c\" (UniqueName: \"kubernetes.io/projected/41ab0dd0-6d29-4caf-b5b8-35255519b562-kube-api-access-p7t4c\") pod \"e2e-api-a-8c2357-openshift-default-659d454f6f-chhkx\" (UID: \"41ab0dd0-6d29-4caf-b5b8-35255519b562\") " pod="openshift-ingress/e2e-api-a-8c2357-openshift-default-659d454f6f-chhkx"
Oct 31 00:50:17.684069 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:50:17.683995    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/41ab0dd0-6d29-4caf-b5b8-35255519b562-istio-token\") pod \"e2e-api-a-8c2357-openshift-default-659d454f6f-chhkx\" (UID: \"41ab0dd0-6d29-4caf-b5b8-35255519b562\") " pod="openshift-ingress/e2e-api-a-8c2357-openshift-default-659d454f6f-chhkx"
Oct 31 00:50:17.715702 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:50:17.715403    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-8c2357-openshift-default-659d454f6f-chhkx"
Oct 31 00:50:17.717572 ip-10-0-132-173 kubenswrapper[4284]: E1031 00:50:17.717491    4284 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fd8sp" containerName="extractor"
Oct 31 00:50:18.042791 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:50:18.016040    4284 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6cc458d887-r7hr5 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.65:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 31 00:50:18.042791 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:50:18.016106    4284 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-r7hr5" podUID="6cce1cfc-7109-4827-a231-fe01dd9a4bf9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.65:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:50:18.435292 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:50:18.432269    4284 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6cc458d887-d4s6f container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.64:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 31 00:50:18.435292 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:50:18.432333    4284 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-d4s6f" podUID="b398d9e9-9d78-4185-ac8c-4a9eecca9f48" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.64:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:50:18.499914 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:50:18.499800    4284 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6cc458d887-hvt6d container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.63:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 31 00:50:18.499914 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:50:18.499863    4284 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-hvt6d" podUID="907b625f-2ccc-4de2-8181-34ff51ab2620" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.63:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:50:18.664630 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:50:18.664339    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-8c2357-openshift-default-659d454f6f-chhkx"]
Oct 31 00:50:18.666316 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:50:18.666279    4284 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 00:50:18.666422 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:50:18.666369    4284 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 00:50:18.666422 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:50:18.666415    4284 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 00:50:18.908174 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:50:18.908139    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-8c2357-openshift-default-659d454f6f-chhkx" event={"ID":"41ab0dd0-6d29-4caf-b5b8-35255519b562","Type":"ContainerStarted","Data":"bbebe8da85ba4299b1c54300acf6aa922c3f4bf7bebc82e484318abcc91c278a"}
Oct 31 00:50:18.908353 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:50:18.908191    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-8c2357-openshift-default-659d454f6f-chhkx" event={"ID":"41ab0dd0-6d29-4caf-b5b8-35255519b562","Type":"ContainerStarted","Data":"db5e22c369cc43f26cb18ecfdc026469991162f613a7f4408d750568d99faa56"}
Oct 31 00:50:19.027775 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:50:19.027662    4284 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-8c2357-openshift-default-659d454f6f-chhkx" podStartSLOduration=3.027647861 podStartE2EDuration="3.027647861s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 00:50:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:50:19.023750968 +0000 UTC m=+1333.168757884" watchObservedRunningTime="2026-10-31 00:50:19.027647861 +0000 UTC m=+1333.172654776"
Oct 31 00:50:19.450808 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:50:19.450770    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:50:19.716921 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:50:19.716838    4284 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-8c2357-openshift-default-659d454f6f-chhkx"
Oct 31 00:50:19.722011 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:50:19.721986    4284 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-8c2357-openshift-default-659d454f6f-chhkx"
Oct 31 00:50:19.914717 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:50:19.914687    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-8c2357-openshift-default-659d454f6f-chhkx"
Oct 31 00:50:19.915761 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:50:19.915744    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-8c2357-openshift-default-659d454f6f-chhkx"
Oct 31 00:50:21.881920 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:50:21.881873    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:50:22.041725 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:50:22.041688    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:50:22.191841 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:50:22.191799    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:50:24.470919 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:50:24.470881    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:50:25.020037 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:50:25.020006    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:50:25.250238 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:50:25.250020    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:50:25.583765 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:50:25.583737    4284 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-8c2357-655589dfb7-9txvh"]
Oct 31 00:50:25.587955 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:50:25.587935    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-8c2357-655589dfb7-9txvh"
Oct 31 00:50:25.676693 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:50:25.676639    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d29ce763-9fd5-4bb3-ac9b-c8209da9d59a-plugins-config-volume\") pod \"payload-processing-e2e-api-a-8c2357-655589dfb7-9txvh\" (UID: \"d29ce763-9fd5-4bb3-ac9b-c8209da9d59a\") " pod="openshift-ingress/payload-processing-e2e-api-a-8c2357-655589dfb7-9txvh"
Oct 31 00:50:25.676940 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:50:25.676916    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6nqsz\" (UniqueName: \"kubernetes.io/projected/d29ce763-9fd5-4bb3-ac9b-c8209da9d59a-kube-api-access-6nqsz\") pod \"payload-processing-e2e-api-a-8c2357-655589dfb7-9txvh\" (UID: \"d29ce763-9fd5-4bb3-ac9b-c8209da9d59a\") " pod="openshift-ingress/payload-processing-e2e-api-a-8c2357-655589dfb7-9txvh"
Oct 31 00:50:25.684863 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:50:25.684828    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-8c2357-655589dfb7-9txvh"]
Oct 31 00:50:25.866975 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:50:25.866883    4284 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-a-8c2357\""
Oct 31 00:50:25.867149 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:50:25.867043    4284 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-a-8c2357-dockercfg-tswj7\""
Oct 31 00:50:25.882322 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:50:25.881918    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d29ce763-9fd5-4bb3-ac9b-c8209da9d59a-plugins-config-volume\") pod \"payload-processing-e2e-api-a-8c2357-655589dfb7-9txvh\" (UID: \"d29ce763-9fd5-4bb3-ac9b-c8209da9d59a\") " pod="openshift-ingress/payload-processing-e2e-api-a-8c2357-655589dfb7-9txvh"
Oct 31 00:50:25.882322 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:50:25.881962    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6nqsz\" (UniqueName: \"kubernetes.io/projected/d29ce763-9fd5-4bb3-ac9b-c8209da9d59a-kube-api-access-6nqsz\") pod \"payload-processing-e2e-api-a-8c2357-655589dfb7-9txvh\" (UID: \"d29ce763-9fd5-4bb3-ac9b-c8209da9d59a\") " pod="openshift-ingress/payload-processing-e2e-api-a-8c2357-655589dfb7-9txvh"
Oct 31 00:50:25.882807 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:50:25.882781    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d29ce763-9fd5-4bb3-ac9b-c8209da9d59a-plugins-config-volume\") pod \"payload-processing-e2e-api-a-8c2357-655589dfb7-9txvh\" (UID: \"d29ce763-9fd5-4bb3-ac9b-c8209da9d59a\") " pod="openshift-ingress/payload-processing-e2e-api-a-8c2357-655589dfb7-9txvh"
Oct 31 00:50:25.940908 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:50:25.940871    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6nqsz\" (UniqueName: \"kubernetes.io/projected/d29ce763-9fd5-4bb3-ac9b-c8209da9d59a-kube-api-access-6nqsz\") pod \"payload-processing-e2e-api-a-8c2357-655589dfb7-9txvh\" (UID: \"d29ce763-9fd5-4bb3-ac9b-c8209da9d59a\") " pod="openshift-ingress/payload-processing-e2e-api-a-8c2357-655589dfb7-9txvh"
Oct 31 00:50:25.941127 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:50:25.941108    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-8c2357-655589dfb7-9txvh"
Oct 31 00:50:26.312358 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:50:26.312332    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-8c2357-655589dfb7-9txvh"]
Oct 31 00:50:26.313681 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:50:26.313653    4284 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd29ce763_9fd5_4bb3_ac9b_c8209da9d59a.slice/crio-3fcfaa7b40b345aac21bf4a0360c2a57774f369afbcad04fc1f09f600269d24c WatchSource:0}: Error finding container 3fcfaa7b40b345aac21bf4a0360c2a57774f369afbcad04fc1f09f600269d24c: Status 404 returned error can't find the container with id 3fcfaa7b40b345aac21bf4a0360c2a57774f369afbcad04fc1f09f600269d24c
Oct 31 00:50:26.960469 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:50:26.960430    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-8c2357-655589dfb7-9txvh" event={"ID":"d29ce763-9fd5-4bb3-ac9b-c8209da9d59a","Type":"ContainerStarted","Data":"607a9748c4d27b5d86c4155ff9bd0f440176f5804ced8ed9abfc45309fb8ebff"}
Oct 31 00:50:26.960892 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:50:26.960490    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-8c2357-655589dfb7-9txvh" event={"ID":"d29ce763-9fd5-4bb3-ac9b-c8209da9d59a","Type":"ContainerStarted","Data":"3fcfaa7b40b345aac21bf4a0360c2a57774f369afbcad04fc1f09f600269d24c"}
Oct 31 00:50:26.960892 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:50:26.960670    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-a-8c2357-655589dfb7-9txvh"
Oct 31 00:50:27.075155 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:50:27.075103    4284 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-a-8c2357-655589dfb7-9txvh" podStartSLOduration=2.075087222 podStartE2EDuration="2.075087222s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 00:50:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:50:27.073835744 +0000 UTC m=+1341.218842660" watchObservedRunningTime="2026-10-31 00:50:27.075087222 +0000 UTC m=+1341.220094142"
Oct 31 00:50:29.689813 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:50:29.689773    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:50:34.595469 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:50:34.595427    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:50:37.970932 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:50:37.970902    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-a-8c2357-655589dfb7-9txvh"
Oct 31 00:50:39.322556 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:50:39.322511    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:50:43.717322 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:50:43.717289    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:50:44.258090 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:50:44.258054    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:50:47.036360 ip-10-0-132-173 kubenswrapper[4284]: E1031 00:50:47.036321    4284 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fd8sp" containerName="extractor"
Oct 31 00:50:49.210944 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:50:49.210904    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:50:54.173026 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:50:54.172991    4284 generic.go:356] "Generic (PLEG): container finished" podID="b58c53d0-4acf-4de9-b322-8ead4dc9b013" containerID="c7da3b2ece1e204bf15a08c3bb655c87bf30646ca899dde9f0835ee35199ee7a" exitCode=0
Oct 31 00:50:54.173520 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:50:54.173059    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-7e6fe88b-openshift-default-5b64b75c89-qzgdm" event={"ID":"b58c53d0-4acf-4de9-b322-8ead4dc9b013","Type":"ContainerDied","Data":"c7da3b2ece1e204bf15a08c3bb655c87bf30646ca899dde9f0835ee35199ee7a"}
Oct 31 00:50:54.173520 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:50:54.173512    4284 scope.go:122] "RemoveContainer" containerID="c7da3b2ece1e204bf15a08c3bb655c87bf30646ca899dde9f0835ee35199ee7a"
Oct 31 00:50:54.175044 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:50:54.175015    4284 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 00:50:54.175124 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:50:54.175083    4284 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 00:50:54.175124 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:50:54.175116    4284 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 00:50:54.199241 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:50:54.199183    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:50:55.182031 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:50:55.181997    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-7e6fe88b-openshift-default-5b64b75c89-qzgdm" event={"ID":"b58c53d0-4acf-4de9-b322-8ead4dc9b013","Type":"ContainerStarted","Data":"051b5ac4888bf6adde575026483832446a241a62588127de7ac51a390e228ee2"}
Oct 31 00:50:55.914665 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:50:55.914592    4284 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-7e6fe88b-openshift-default-5b64b75c89-qzgdm"
Oct 31 00:50:55.914889 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:50:55.914724    4284 patch_prober.go:29] interesting pod/e2e-mt-7e6fe88b-openshift-default-5b64b75c89-qzgdm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.80:15021/healthz/ready\": dial tcp 10.134.0.80:15021: connect: connection refused" start-of-body=
Oct 31 00:50:55.914889 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:50:55.914768    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-7e6fe88b-openshift-default-5b64b75c89-qzgdm" podUID="b58c53d0-4acf-4de9-b322-8ead4dc9b013" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.80:15021/healthz/ready\": dial tcp 10.134.0.80:15021: connect: connection refused"
Oct 31 00:50:56.514510 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:50:56.514471    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:50:56.910663 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:50:56.910625    4284 patch_prober.go:29] interesting pod/e2e-mt-7e6fe88b-openshift-default-5b64b75c89-qzgdm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.80:15021/healthz/ready\": dial tcp 10.134.0.80:15021: connect: connection refused" start-of-body=
Oct 31 00:50:56.910989 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:50:56.910960    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-7e6fe88b-openshift-default-5b64b75c89-qzgdm" podUID="b58c53d0-4acf-4de9-b322-8ead4dc9b013" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.80:15021/healthz/ready\": dial tcp 10.134.0.80:15021: connect: connection refused"
Oct 31 00:50:57.910763 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:50:57.910724    4284 patch_prober.go:29] interesting pod/e2e-mt-7e6fe88b-openshift-default-5b64b75c89-qzgdm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.80:15021/healthz/ready\": dial tcp 10.134.0.80:15021: connect: connection refused" start-of-body=
Oct 31 00:50:57.911198 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:50:57.910782    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-7e6fe88b-openshift-default-5b64b75c89-qzgdm" podUID="b58c53d0-4acf-4de9-b322-8ead4dc9b013" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.80:15021/healthz/ready\": dial tcp 10.134.0.80:15021: connect: connection refused"
Oct 31 00:50:58.216265 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:50:58.216156    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:50:58.910779 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:50:58.910742    4284 patch_prober.go:29] interesting pod/e2e-mt-7e6fe88b-openshift-default-5b64b75c89-qzgdm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.80:15021/healthz/ready\": dial tcp 10.134.0.80:15021: connect: connection refused" start-of-body=
Oct 31 00:50:58.911228 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:50:58.910821    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-7e6fe88b-openshift-default-5b64b75c89-qzgdm" podUID="b58c53d0-4acf-4de9-b322-8ead4dc9b013" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.80:15021/healthz/ready\": dial tcp 10.134.0.80:15021: connect: connection refused"
Oct 31 00:50:59.078180 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:50:59.078145    4284 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-8c2357-59889f6855-9l5vs"]
Oct 31 00:50:59.082599 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:50:59.082576    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-8c2357-59889f6855-9l5vs"
Oct 31 00:50:59.173654 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:50:59.173556    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/16527711-37f1-4870-8f17-cc76516411c2-metrics-certs\") pod \"maas-api-e2e-api-a-8c2357-59889f6855-9l5vs\" (UID: \"16527711-37f1-4870-8f17-cc76516411c2\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-8c2357-59889f6855-9l5vs"
Oct 31 00:50:59.173808 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:50:59.173676    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5pqbv\" (UniqueName: \"kubernetes.io/projected/16527711-37f1-4870-8f17-cc76516411c2-kube-api-access-5pqbv\") pod \"maas-api-e2e-api-a-8c2357-59889f6855-9l5vs\" (UID: \"16527711-37f1-4870-8f17-cc76516411c2\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-8c2357-59889f6855-9l5vs"
Oct 31 00:50:59.173808 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:50:59.173708    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/16527711-37f1-4870-8f17-cc76516411c2-openshift-service-ca\") pod \"maas-api-e2e-api-a-8c2357-59889f6855-9l5vs\" (UID: \"16527711-37f1-4870-8f17-cc76516411c2\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-8c2357-59889f6855-9l5vs"
Oct 31 00:50:59.173898 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:50:59.173862    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/16527711-37f1-4870-8f17-cc76516411c2-maas-api-tls\") pod \"maas-api-e2e-api-a-8c2357-59889f6855-9l5vs\" (UID: \"16527711-37f1-4870-8f17-cc76516411c2\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-8c2357-59889f6855-9l5vs"
Oct 31 00:50:59.209504 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:50:59.209471    4284 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-8c2357-serving-cert\""
Oct 31 00:50:59.215265 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:50:59.213452    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-8c2357-59889f6855-9l5vs"]
Oct 31 00:50:59.274746 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:50:59.274716    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/16527711-37f1-4870-8f17-cc76516411c2-metrics-certs\") pod \"maas-api-e2e-api-a-8c2357-59889f6855-9l5vs\" (UID: \"16527711-37f1-4870-8f17-cc76516411c2\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-8c2357-59889f6855-9l5vs"
Oct 31 00:50:59.274929 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:50:59.274781    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5pqbv\" (UniqueName: \"kubernetes.io/projected/16527711-37f1-4870-8f17-cc76516411c2-kube-api-access-5pqbv\") pod \"maas-api-e2e-api-a-8c2357-59889f6855-9l5vs\" (UID: \"16527711-37f1-4870-8f17-cc76516411c2\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-8c2357-59889f6855-9l5vs"
Oct 31 00:50:59.274929 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:50:59.274814    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/16527711-37f1-4870-8f17-cc76516411c2-openshift-service-ca\") pod \"maas-api-e2e-api-a-8c2357-59889f6855-9l5vs\" (UID: \"16527711-37f1-4870-8f17-cc76516411c2\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-8c2357-59889f6855-9l5vs"
Oct 31 00:50:59.274929 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:50:59.274920    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/16527711-37f1-4870-8f17-cc76516411c2-maas-api-tls\") pod \"maas-api-e2e-api-a-8c2357-59889f6855-9l5vs\" (UID: \"16527711-37f1-4870-8f17-cc76516411c2\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-8c2357-59889f6855-9l5vs"
Oct 31 00:50:59.275103 ip-10-0-132-173 kubenswrapper[4284]: E1031 00:50:59.275083    4284 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-a-8c2357-serving-cert: secret "maas-api-e2e-api-a-8c2357-serving-cert" not found
Oct 31 00:50:59.275166 ip-10-0-132-173 kubenswrapper[4284]: E1031 00:50:59.275156    4284 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/16527711-37f1-4870-8f17-cc76516411c2-maas-api-tls podName:16527711-37f1-4870-8f17-cc76516411c2 nodeName:}" failed. No retries permitted until 2026-10-31 00:50:59.775141169 +0000 UTC m=+1373.920148063 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/16527711-37f1-4870-8f17-cc76516411c2-maas-api-tls") pod "maas-api-e2e-api-a-8c2357-59889f6855-9l5vs" (UID: "16527711-37f1-4870-8f17-cc76516411c2") : secret "maas-api-e2e-api-a-8c2357-serving-cert" not found
Oct 31 00:50:59.275719 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:50:59.275698    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/16527711-37f1-4870-8f17-cc76516411c2-openshift-service-ca\") pod \"maas-api-e2e-api-a-8c2357-59889f6855-9l5vs\" (UID: \"16527711-37f1-4870-8f17-cc76516411c2\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-8c2357-59889f6855-9l5vs"
Oct 31 00:50:59.277413 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:50:59.277398    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/16527711-37f1-4870-8f17-cc76516411c2-metrics-certs\") pod \"maas-api-e2e-api-a-8c2357-59889f6855-9l5vs\" (UID: \"16527711-37f1-4870-8f17-cc76516411c2\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-8c2357-59889f6855-9l5vs"
Oct 31 00:50:59.280193 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:50:59.280165    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:50:59.381002 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:50:59.380970    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5pqbv\" (UniqueName: \"kubernetes.io/projected/16527711-37f1-4870-8f17-cc76516411c2-kube-api-access-5pqbv\") pod \"maas-api-e2e-api-a-8c2357-59889f6855-9l5vs\" (UID: \"16527711-37f1-4870-8f17-cc76516411c2\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-8c2357-59889f6855-9l5vs"
Oct 31 00:50:59.470356 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:50:59.470266    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:50:59.638407 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:50:59.638364    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:50:59.810561 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:50:59.810463    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/16527711-37f1-4870-8f17-cc76516411c2-maas-api-tls\") pod \"maas-api-e2e-api-a-8c2357-59889f6855-9l5vs\" (UID: \"16527711-37f1-4870-8f17-cc76516411c2\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-8c2357-59889f6855-9l5vs"
Oct 31 00:50:59.813232 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:50:59.813191    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/16527711-37f1-4870-8f17-cc76516411c2-maas-api-tls\") pod \"maas-api-e2e-api-a-8c2357-59889f6855-9l5vs\" (UID: \"16527711-37f1-4870-8f17-cc76516411c2\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-8c2357-59889f6855-9l5vs"
Oct 31 00:50:59.910653 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:50:59.910614    4284 patch_prober.go:29] interesting pod/e2e-mt-7e6fe88b-openshift-default-5b64b75c89-qzgdm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.80:15021/healthz/ready\": dial tcp 10.134.0.80:15021: connect: connection refused" start-of-body=
Oct 31 00:50:59.910816 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:50:59.910710    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-7e6fe88b-openshift-default-5b64b75c89-qzgdm" podUID="b58c53d0-4acf-4de9-b322-8ead4dc9b013" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.80:15021/healthz/ready\": dial tcp 10.134.0.80:15021: connect: connection refused"
Oct 31 00:51:00.019715 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:51:00.019678    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-8c2357-59889f6855-9l5vs"
Oct 31 00:51:00.395915 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:51:00.395885    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-8c2357-59889f6855-9l5vs"]
Oct 31 00:51:00.396601 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:51:00.396571    4284 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod16527711_37f1_4870_8f17_cc76516411c2.slice/crio-f4a03da59a0464352480b5b845c3c2924eaf377a321b940bbb8ad2e229372bb5 WatchSource:0}: Error finding container f4a03da59a0464352480b5b845c3c2924eaf377a321b940bbb8ad2e229372bb5: Status 404 returned error can't find the container with id f4a03da59a0464352480b5b845c3c2924eaf377a321b940bbb8ad2e229372bb5
Oct 31 00:51:00.398161 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:51:00.398142    4284 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 31 00:51:00.920405 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:51:00.920365    4284 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-7e6fe88b-openshift-default-5b64b75c89-qzgdm"
Oct 31 00:51:00.921911 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:51:00.921882    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-7e6fe88b-openshift-default-5b64b75c89-qzgdm"
Oct 31 00:51:00.922800 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:51:00.922777    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-7e6fe88b-openshift-default-5b64b75c89-qzgdm"
Oct 31 00:51:01.249198 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:51:01.249118    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-8c2357-59889f6855-9l5vs" event={"ID":"16527711-37f1-4870-8f17-cc76516411c2","Type":"ContainerStarted","Data":"f4a03da59a0464352480b5b845c3c2924eaf377a321b940bbb8ad2e229372bb5"}
Oct 31 00:51:02.415531 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:51:02.415489    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-8c2357-59889f6855-9l5vs" event={"ID":"16527711-37f1-4870-8f17-cc76516411c2","Type":"ContainerStarted","Data":"f72af9c2213ba6029f3a08be1cb6c1c210a14deeeac81a196f2e7e21d39bc592"}
Oct 31 00:51:02.416059 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:51:02.415819    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-8c2357-59889f6855-9l5vs"
Oct 31 00:51:02.531690 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:51:02.531630    4284 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-8c2357-59889f6855-9l5vs" podStartSLOduration=2.512713054 podStartE2EDuration="3.531616266s" totalImagesPullingTime="1.018903212s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 00:50:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:51:02.53056923 +0000 UTC m=+1376.675576145" watchObservedRunningTime="2026-10-31 00:51:02.531616266 +0000 UTC m=+1376.676623181"
Oct 31 00:51:04.484679 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:51:04.484648    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:51:08.427282 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:51:08.427251    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-8c2357-59889f6855-9l5vs"
Oct 31 00:51:09.322206 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:51:09.322164    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:51:14.186691 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:51:14.186651    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:51:17.015128 ip-10-0-132-173 kubenswrapper[4284]: E1031 00:51:17.015047    4284 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fd8sp" containerName="extractor"
Oct 31 00:51:18.980596 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:51:18.980560    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:51:22.637386 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:51:22.637337    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:51:23.972921 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:51:23.972880    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:51:25.954074 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:51:25.954032    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:51:28.932497 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:51:28.932456    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:51:32.460804 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:51:32.460768    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:51:33.800528 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:51:33.800490    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:51:34.660445 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:51:34.660416    4284 generic.go:356] "Generic (PLEG): container finished" podID="41ab0dd0-6d29-4caf-b5b8-35255519b562" containerID="bbebe8da85ba4299b1c54300acf6aa922c3f4bf7bebc82e484318abcc91c278a" exitCode=0
Oct 31 00:51:34.660621 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:51:34.660471    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-8c2357-openshift-default-659d454f6f-chhkx" event={"ID":"41ab0dd0-6d29-4caf-b5b8-35255519b562","Type":"ContainerDied","Data":"bbebe8da85ba4299b1c54300acf6aa922c3f4bf7bebc82e484318abcc91c278a"}
Oct 31 00:51:34.660912 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:51:34.660885    4284 scope.go:122] "RemoveContainer" containerID="bbebe8da85ba4299b1c54300acf6aa922c3f4bf7bebc82e484318abcc91c278a"
Oct 31 00:51:34.662431 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:51:34.662404    4284 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 00:51:34.662494 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:51:34.662475    4284 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 00:51:34.662530 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:51:34.662505    4284 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 00:51:34.914883 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:51:34.914786    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-8c2357-openshift-default-659d454f6f-chhkx"
Oct 31 00:51:35.682164 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:51:35.682129    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-8c2357-openshift-default-659d454f6f-chhkx" event={"ID":"41ab0dd0-6d29-4caf-b5b8-35255519b562","Type":"ContainerStarted","Data":"0bc37d0088f14c598aeb9ceb31452f65833336daebedc347e70114cce549ae8d"}
Oct 31 00:51:35.716371 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:51:35.716333    4284 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-8c2357-openshift-default-659d454f6f-chhkx"
Oct 31 00:51:35.717833 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:51:35.717800    4284 patch_prober.go:29] interesting pod/e2e-api-a-8c2357-openshift-default-659d454f6f-chhkx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.83:15021/healthz/ready\": dial tcp 10.134.0.83:15021: connect: connection refused" start-of-body=
Oct 31 00:51:35.717961 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:51:35.717850    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-8c2357-openshift-default-659d454f6f-chhkx" podUID="41ab0dd0-6d29-4caf-b5b8-35255519b562" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.83:15021/healthz/ready\": dial tcp 10.134.0.83:15021: connect: connection refused"
Oct 31 00:51:36.716787 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:51:36.716753    4284 patch_prober.go:29] interesting pod/e2e-api-a-8c2357-openshift-default-659d454f6f-chhkx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.83:15021/healthz/ready\": dial tcp 10.134.0.83:15021: connect: connection refused" start-of-body=
Oct 31 00:51:36.717288 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:51:36.716811    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-8c2357-openshift-default-659d454f6f-chhkx" podUID="41ab0dd0-6d29-4caf-b5b8-35255519b562" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.83:15021/healthz/ready\": dial tcp 10.134.0.83:15021: connect: connection refused"
Oct 31 00:51:37.716594 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:51:37.716562    4284 patch_prober.go:29] interesting pod/e2e-api-a-8c2357-openshift-default-659d454f6f-chhkx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.83:15021/healthz/ready\": dial tcp 10.134.0.83:15021: connect: connection refused" start-of-body=
Oct 31 00:51:37.716827 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:51:37.716634    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-8c2357-openshift-default-659d454f6f-chhkx" podUID="41ab0dd0-6d29-4caf-b5b8-35255519b562" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.83:15021/healthz/ready\": dial tcp 10.134.0.83:15021: connect: connection refused"
Oct 31 00:51:38.716794 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:51:38.716755    4284 patch_prober.go:29] interesting pod/e2e-api-a-8c2357-openshift-default-659d454f6f-chhkx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.83:15021/healthz/ready\": dial tcp 10.134.0.83:15021: connect: connection refused" start-of-body=
Oct 31 00:51:38.717004 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:51:38.716835    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-8c2357-openshift-default-659d454f6f-chhkx" podUID="41ab0dd0-6d29-4caf-b5b8-35255519b562" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.83:15021/healthz/ready\": dial tcp 10.134.0.83:15021: connect: connection refused"
Oct 31 00:51:38.728204 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:51:38.728172    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:51:39.716667 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:51:39.716632    4284 patch_prober.go:29] interesting pod/e2e-api-a-8c2357-openshift-default-659d454f6f-chhkx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.83:15021/healthz/ready\": dial tcp 10.134.0.83:15021: connect: connection refused" start-of-body=
Oct 31 00:51:39.716988 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:51:39.716703    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-8c2357-openshift-default-659d454f6f-chhkx" podUID="41ab0dd0-6d29-4caf-b5b8-35255519b562" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.83:15021/healthz/ready\": dial tcp 10.134.0.83:15021: connect: connection refused"
Oct 31 00:51:40.717364 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:51:40.717321    4284 patch_prober.go:29] interesting pod/e2e-api-a-8c2357-openshift-default-659d454f6f-chhkx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.83:15021/healthz/ready\": dial tcp 10.134.0.83:15021: connect: connection refused" start-of-body=
Oct 31 00:51:40.717890 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:51:40.717390    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-8c2357-openshift-default-659d454f6f-chhkx" podUID="41ab0dd0-6d29-4caf-b5b8-35255519b562" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.83:15021/healthz/ready\": dial tcp 10.134.0.83:15021: connect: connection refused"
Oct 31 00:51:41.721090 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:51:41.721056    4284 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-8c2357-openshift-default-659d454f6f-chhkx"
Oct 31 00:51:41.721695 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:51:41.721671    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-8c2357-openshift-default-659d454f6f-chhkx"
Oct 31 00:51:41.722726 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:51:41.722708    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-8c2357-openshift-default-659d454f6f-chhkx"
Oct 31 00:51:42.909869 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:51:42.909828    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:51:43.701011 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:51:43.700973    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:51:47.194494 ip-10-0-132-173 kubenswrapper[4284]: E1031 00:51:47.194416    4284 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fd8sp" containerName="extractor"
Oct 31 00:51:47.949382 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:51:47.949341    4284 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6cc458d887-r7hr5 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.65:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 31 00:51:47.949704 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:51:47.949671    4284 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-r7hr5" podUID="6cce1cfc-7109-4827-a231-fe01dd9a4bf9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.65:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:51:48.393407 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:51:48.393307    4284 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6cc458d887-d4s6f container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.64:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 31 00:51:48.393977 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:51:48.393944    4284 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-d4s6f" podUID="b398d9e9-9d78-4185-ac8c-4a9eecca9f48" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.64:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:51:48.778026 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:51:48.777986    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:51:49.313378 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:51:49.313343    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:51:49.484037 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:51:49.483999    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:51:49.635701 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:51:49.635594    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:51:49.819459 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:51:49.819422    4284 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-69a4bc-openshift-default-797b9d8659-rgs5q"]
Oct 31 00:51:49.823202 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:51:49.823179    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-69a4bc-openshift-default-797b9d8659-rgs5q"
Oct 31 00:51:49.878242 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:51:49.876630    4284 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-69a4bc-openshift-default-dockercfg-w97l5\""
Oct 31 00:51:49.918849 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:51:49.918815    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:51:49.924695 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:51:49.924668    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-69a4bc-openshift-default-797b9d8659-rgs5q"]
Oct 31 00:51:49.961951 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:51:49.961910    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/66210261-9baa-4961-855e-b92e2c4e154e-istio-envoy\") pod \"e2e-api-a-69a4bc-openshift-default-797b9d8659-rgs5q\" (UID: \"66210261-9baa-4961-855e-b92e2c4e154e\") " pod="openshift-ingress/e2e-api-a-69a4bc-openshift-default-797b9d8659-rgs5q"
Oct 31 00:51:49.962155 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:51:49.961965    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k8sv9\" (UniqueName: \"kubernetes.io/projected/66210261-9baa-4961-855e-b92e2c4e154e-kube-api-access-k8sv9\") pod \"e2e-api-a-69a4bc-openshift-default-797b9d8659-rgs5q\" (UID: \"66210261-9baa-4961-855e-b92e2c4e154e\") " pod="openshift-ingress/e2e-api-a-69a4bc-openshift-default-797b9d8659-rgs5q"
Oct 31 00:51:49.962155 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:51:49.962120    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/66210261-9baa-4961-855e-b92e2c4e154e-istio-podinfo\") pod \"e2e-api-a-69a4bc-openshift-default-797b9d8659-rgs5q\" (UID: \"66210261-9baa-4961-855e-b92e2c4e154e\") " pod="openshift-ingress/e2e-api-a-69a4bc-openshift-default-797b9d8659-rgs5q"
Oct 31 00:51:49.962334 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:51:49.962175    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/66210261-9baa-4961-855e-b92e2c4e154e-credential-socket\") pod \"e2e-api-a-69a4bc-openshift-default-797b9d8659-rgs5q\" (UID: \"66210261-9baa-4961-855e-b92e2c4e154e\") " pod="openshift-ingress/e2e-api-a-69a4bc-openshift-default-797b9d8659-rgs5q"
Oct 31 00:51:49.962334 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:51:49.962306    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/66210261-9baa-4961-855e-b92e2c4e154e-istio-data\") pod \"e2e-api-a-69a4bc-openshift-default-797b9d8659-rgs5q\" (UID: \"66210261-9baa-4961-855e-b92e2c4e154e\") " pod="openshift-ingress/e2e-api-a-69a4bc-openshift-default-797b9d8659-rgs5q"
Oct 31 00:51:49.962455 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:51:49.962349    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/66210261-9baa-4961-855e-b92e2c4e154e-istiod-ca-cert\") pod \"e2e-api-a-69a4bc-openshift-default-797b9d8659-rgs5q\" (UID: \"66210261-9baa-4961-855e-b92e2c4e154e\") " pod="openshift-ingress/e2e-api-a-69a4bc-openshift-default-797b9d8659-rgs5q"
Oct 31 00:51:49.962455 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:51:49.962386    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/66210261-9baa-4961-855e-b92e2c4e154e-workload-socket\") pod \"e2e-api-a-69a4bc-openshift-default-797b9d8659-rgs5q\" (UID: \"66210261-9baa-4961-855e-b92e2c4e154e\") " pod="openshift-ingress/e2e-api-a-69a4bc-openshift-default-797b9d8659-rgs5q"
Oct 31 00:51:49.962455 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:51:49.962424    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/66210261-9baa-4961-855e-b92e2c4e154e-istio-token\") pod \"e2e-api-a-69a4bc-openshift-default-797b9d8659-rgs5q\" (UID: \"66210261-9baa-4961-855e-b92e2c4e154e\") " pod="openshift-ingress/e2e-api-a-69a4bc-openshift-default-797b9d8659-rgs5q"
Oct 31 00:51:49.962623 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:51:49.962508    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/66210261-9baa-4961-855e-b92e2c4e154e-workload-certs\") pod \"e2e-api-a-69a4bc-openshift-default-797b9d8659-rgs5q\" (UID: \"66210261-9baa-4961-855e-b92e2c4e154e\") " pod="openshift-ingress/e2e-api-a-69a4bc-openshift-default-797b9d8659-rgs5q"
Oct 31 00:51:50.063470 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:51:50.063434    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/66210261-9baa-4961-855e-b92e2c4e154e-istio-token\") pod \"e2e-api-a-69a4bc-openshift-default-797b9d8659-rgs5q\" (UID: \"66210261-9baa-4961-855e-b92e2c4e154e\") " pod="openshift-ingress/e2e-api-a-69a4bc-openshift-default-797b9d8659-rgs5q"
Oct 31 00:51:50.063695 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:51:50.063665    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/66210261-9baa-4961-855e-b92e2c4e154e-workload-certs\") pod \"e2e-api-a-69a4bc-openshift-default-797b9d8659-rgs5q\" (UID: \"66210261-9baa-4961-855e-b92e2c4e154e\") " pod="openshift-ingress/e2e-api-a-69a4bc-openshift-default-797b9d8659-rgs5q"
Oct 31 00:51:50.063812 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:51:50.063786    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/66210261-9baa-4961-855e-b92e2c4e154e-istio-envoy\") pod \"e2e-api-a-69a4bc-openshift-default-797b9d8659-rgs5q\" (UID: \"66210261-9baa-4961-855e-b92e2c4e154e\") " pod="openshift-ingress/e2e-api-a-69a4bc-openshift-default-797b9d8659-rgs5q"
Oct 31 00:51:50.063874 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:51:50.063823    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k8sv9\" (UniqueName: \"kubernetes.io/projected/66210261-9baa-4961-855e-b92e2c4e154e-kube-api-access-k8sv9\") pod \"e2e-api-a-69a4bc-openshift-default-797b9d8659-rgs5q\" (UID: \"66210261-9baa-4961-855e-b92e2c4e154e\") " pod="openshift-ingress/e2e-api-a-69a4bc-openshift-default-797b9d8659-rgs5q"
Oct 31 00:51:50.063971 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:51:50.063946    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/66210261-9baa-4961-855e-b92e2c4e154e-istio-podinfo\") pod \"e2e-api-a-69a4bc-openshift-default-797b9d8659-rgs5q\" (UID: \"66210261-9baa-4961-855e-b92e2c4e154e\") " pod="openshift-ingress/e2e-api-a-69a4bc-openshift-default-797b9d8659-rgs5q"
Oct 31 00:51:50.064034 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:51:50.063994    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/66210261-9baa-4961-855e-b92e2c4e154e-workload-certs\") pod \"e2e-api-a-69a4bc-openshift-default-797b9d8659-rgs5q\" (UID: \"66210261-9baa-4961-855e-b92e2c4e154e\") " pod="openshift-ingress/e2e-api-a-69a4bc-openshift-default-797b9d8659-rgs5q"
Oct 31 00:51:50.064090 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:51:50.064052    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/66210261-9baa-4961-855e-b92e2c4e154e-credential-socket\") pod \"e2e-api-a-69a4bc-openshift-default-797b9d8659-rgs5q\" (UID: \"66210261-9baa-4961-855e-b92e2c4e154e\") " pod="openshift-ingress/e2e-api-a-69a4bc-openshift-default-797b9d8659-rgs5q"
Oct 31 00:51:50.064150 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:51:50.064122    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/66210261-9baa-4961-855e-b92e2c4e154e-istio-data\") pod \"e2e-api-a-69a4bc-openshift-default-797b9d8659-rgs5q\" (UID: \"66210261-9baa-4961-855e-b92e2c4e154e\") " pod="openshift-ingress/e2e-api-a-69a4bc-openshift-default-797b9d8659-rgs5q"
Oct 31 00:51:50.064232 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:51:50.064149    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/66210261-9baa-4961-855e-b92e2c4e154e-istiod-ca-cert\") pod \"e2e-api-a-69a4bc-openshift-default-797b9d8659-rgs5q\" (UID: \"66210261-9baa-4961-855e-b92e2c4e154e\") " pod="openshift-ingress/e2e-api-a-69a4bc-openshift-default-797b9d8659-rgs5q"
Oct 31 00:51:50.064232 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:51:50.064171    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/66210261-9baa-4961-855e-b92e2c4e154e-workload-socket\") pod \"e2e-api-a-69a4bc-openshift-default-797b9d8659-rgs5q\" (UID: \"66210261-9baa-4961-855e-b92e2c4e154e\") " pod="openshift-ingress/e2e-api-a-69a4bc-openshift-default-797b9d8659-rgs5q"
Oct 31 00:51:50.064536 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:51:50.064458    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/66210261-9baa-4961-855e-b92e2c4e154e-workload-socket\") pod \"e2e-api-a-69a4bc-openshift-default-797b9d8659-rgs5q\" (UID: \"66210261-9baa-4961-855e-b92e2c4e154e\") " pod="openshift-ingress/e2e-api-a-69a4bc-openshift-default-797b9d8659-rgs5q"
Oct 31 00:51:50.064536 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:51:50.064542    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/66210261-9baa-4961-855e-b92e2c4e154e-credential-socket\") pod \"e2e-api-a-69a4bc-openshift-default-797b9d8659-rgs5q\" (UID: \"66210261-9baa-4961-855e-b92e2c4e154e\") " pod="openshift-ingress/e2e-api-a-69a4bc-openshift-default-797b9d8659-rgs5q"
Oct 31 00:51:50.064782 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:51:50.064603    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/66210261-9baa-4961-855e-b92e2c4e154e-istio-data\") pod \"e2e-api-a-69a4bc-openshift-default-797b9d8659-rgs5q\" (UID: \"66210261-9baa-4961-855e-b92e2c4e154e\") " pod="openshift-ingress/e2e-api-a-69a4bc-openshift-default-797b9d8659-rgs5q"
Oct 31 00:51:50.065392 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:51:50.065366    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/66210261-9baa-4961-855e-b92e2c4e154e-istiod-ca-cert\") pod \"e2e-api-a-69a4bc-openshift-default-797b9d8659-rgs5q\" (UID: \"66210261-9baa-4961-855e-b92e2c4e154e\") " pod="openshift-ingress/e2e-api-a-69a4bc-openshift-default-797b9d8659-rgs5q"
Oct 31 00:51:50.066187 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:51:50.066164    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/66210261-9baa-4961-855e-b92e2c4e154e-istio-envoy\") pod \"e2e-api-a-69a4bc-openshift-default-797b9d8659-rgs5q\" (UID: \"66210261-9baa-4961-855e-b92e2c4e154e\") " pod="openshift-ingress/e2e-api-a-69a4bc-openshift-default-797b9d8659-rgs5q"
Oct 31 00:51:50.066569 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:51:50.066552    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/66210261-9baa-4961-855e-b92e2c4e154e-istio-podinfo\") pod \"e2e-api-a-69a4bc-openshift-default-797b9d8659-rgs5q\" (UID: \"66210261-9baa-4961-855e-b92e2c4e154e\") " pod="openshift-ingress/e2e-api-a-69a4bc-openshift-default-797b9d8659-rgs5q"
Oct 31 00:51:50.121761 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:51:50.121722    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:51:50.129457 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:51:50.129429    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k8sv9\" (UniqueName: \"kubernetes.io/projected/66210261-9baa-4961-855e-b92e2c4e154e-kube-api-access-k8sv9\") pod \"e2e-api-a-69a4bc-openshift-default-797b9d8659-rgs5q\" (UID: \"66210261-9baa-4961-855e-b92e2c4e154e\") " pod="openshift-ingress/e2e-api-a-69a4bc-openshift-default-797b9d8659-rgs5q"
Oct 31 00:51:50.129604 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:51:50.129428    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/66210261-9baa-4961-855e-b92e2c4e154e-istio-token\") pod \"e2e-api-a-69a4bc-openshift-default-797b9d8659-rgs5q\" (UID: \"66210261-9baa-4961-855e-b92e2c4e154e\") " pod="openshift-ingress/e2e-api-a-69a4bc-openshift-default-797b9d8659-rgs5q"
Oct 31 00:51:50.136169 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:51:50.136130    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-69a4bc-openshift-default-797b9d8659-rgs5q"
Oct 31 00:51:50.399240 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:51:50.388025    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:51:50.588287 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:51:50.586158    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:51:50.696297 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:51:50.696262    4284 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod66210261_9baa_4961_855e_b92e2c4e154e.slice/crio-c415a3d6cc811e0f480d827e796cd4f1070ed5ea1f425f32c0a7b2cc52d6defd WatchSource:0}: Error finding container c415a3d6cc811e0f480d827e796cd4f1070ed5ea1f425f32c0a7b2cc52d6defd: Status 404 returned error can't find the container with id c415a3d6cc811e0f480d827e796cd4f1070ed5ea1f425f32c0a7b2cc52d6defd
Oct 31 00:51:50.696813 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:51:50.696783    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-69a4bc-openshift-default-797b9d8659-rgs5q"]
Oct 31 00:51:50.699385 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:51:50.699349    4284 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 00:51:50.699466 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:51:50.699452    4284 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 00:51:50.699523 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:51:50.699505    4284 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 00:51:50.972294 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:51:50.972253    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-69a4bc-openshift-default-797b9d8659-rgs5q" event={"ID":"66210261-9baa-4961-855e-b92e2c4e154e","Type":"ContainerStarted","Data":"ec2024aa775dd9368e0a4c1c2655250955a14d7564e3d508b509215a11518476"}
Oct 31 00:51:50.972557 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:51:50.972324    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-69a4bc-openshift-default-797b9d8659-rgs5q" event={"ID":"66210261-9baa-4961-855e-b92e2c4e154e","Type":"ContainerStarted","Data":"c415a3d6cc811e0f480d827e796cd4f1070ed5ea1f425f32c0a7b2cc52d6defd"}
Oct 31 00:51:51.090088 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:51:51.090027    4284 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-69a4bc-openshift-default-797b9d8659-rgs5q" podStartSLOduration=2.090006379 podStartE2EDuration="2.090006379s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 00:51:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:51:51.086920894 +0000 UTC m=+1425.231927845" watchObservedRunningTime="2026-10-31 00:51:51.090006379 +0000 UTC m=+1425.235013298"
Oct 31 00:51:51.136863 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:51:51.136826    4284 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-69a4bc-openshift-default-797b9d8659-rgs5q"
Oct 31 00:51:51.142837 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:51:51.142810    4284 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-69a4bc-openshift-default-797b9d8659-rgs5q"
Oct 31 00:51:52.044523 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:51:52.044494    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-69a4bc-openshift-default-797b9d8659-rgs5q"
Oct 31 00:51:52.049179 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:51:52.049147    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-69a4bc-openshift-default-797b9d8659-rgs5q"
Oct 31 00:51:54.071012 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:51:54.070978    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:51:54.230040 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:51:54.230006    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:51:54.374595 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:51:54.374508    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:51:54.596578 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:51:54.596530    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:51:54.739714 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:51:54.739685    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:51:54.904845 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:51:54.904811    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:51:56.826763 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:51:56.826721    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:51:57.198508 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:51:57.198470    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:51:57.436805 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:51:57.436595    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:51:57.654440 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:51:57.654365    4284 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-69a4bc-854d4ddb7-5gwjd"]
Oct 31 00:51:57.674483 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:51:57.674450    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-69a4bc-854d4ddb7-5gwjd"
Oct 31 00:51:57.702323 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:51:57.702283    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-69a4bc-854d4ddb7-5gwjd"]
Oct 31 00:51:57.729237 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:51:57.725934    4284 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-a-69a4bc\""
Oct 31 00:51:57.821702 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:51:57.821667    4284 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-a-69a4bc-dockercfg-479bm\""
Oct 31 00:51:57.837240 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:51:57.832952    4284 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-69a4bc-69465f7c7-kvvq4"]
Oct 31 00:51:57.903301 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:51:57.903262    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-742lx\" (UniqueName: \"kubernetes.io/projected/fa5fde86-708a-4f57-9e4b-3eb7e311e09c-kube-api-access-742lx\") pod \"payload-pre-processing-e2e-api-a-69a4bc-854d4ddb7-5gwjd\" (UID: \"fa5fde86-708a-4f57-9e4b-3eb7e311e09c\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-69a4bc-854d4ddb7-5gwjd"
Oct 31 00:51:57.903871 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:51:57.903839    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fa5fde86-708a-4f57-9e4b-3eb7e311e09c-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-69a4bc-854d4ddb7-5gwjd\" (UID: \"fa5fde86-708a-4f57-9e4b-3eb7e311e09c\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-69a4bc-854d4ddb7-5gwjd"
Oct 31 00:51:57.960437 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:51:57.926592    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-69a4bc-69465f7c7-kvvq4"
Oct 31 00:51:58.058859 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:51:58.058817    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-742lx\" (UniqueName: \"kubernetes.io/projected/fa5fde86-708a-4f57-9e4b-3eb7e311e09c-kube-api-access-742lx\") pod \"payload-pre-processing-e2e-api-a-69a4bc-854d4ddb7-5gwjd\" (UID: \"fa5fde86-708a-4f57-9e4b-3eb7e311e09c\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-69a4bc-854d4ddb7-5gwjd"
Oct 31 00:51:58.059066 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:51:58.059051    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fa5fde86-708a-4f57-9e4b-3eb7e311e09c-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-69a4bc-854d4ddb7-5gwjd\" (UID: \"fa5fde86-708a-4f57-9e4b-3eb7e311e09c\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-69a4bc-854d4ddb7-5gwjd"
Oct 31 00:51:58.059604 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:51:58.059576    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-69a4bc-69465f7c7-kvvq4"]
Oct 31 00:51:58.060461 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:51:58.060432    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fa5fde86-708a-4f57-9e4b-3eb7e311e09c-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-69a4bc-854d4ddb7-5gwjd\" (UID: \"fa5fde86-708a-4f57-9e4b-3eb7e311e09c\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-69a4bc-854d4ddb7-5gwjd"
Oct 31 00:51:58.118461 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:51:58.118427    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-742lx\" (UniqueName: \"kubernetes.io/projected/fa5fde86-708a-4f57-9e4b-3eb7e311e09c-kube-api-access-742lx\") pod \"payload-pre-processing-e2e-api-a-69a4bc-854d4ddb7-5gwjd\" (UID: \"fa5fde86-708a-4f57-9e4b-3eb7e311e09c\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-69a4bc-854d4ddb7-5gwjd"
Oct 31 00:51:58.161109 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:51:58.161065    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/79898810-a2f5-42bb-ba69-e9a0bd57b0de-plugins-config-volume\") pod \"payload-processing-e2e-api-a-69a4bc-69465f7c7-kvvq4\" (UID: \"79898810-a2f5-42bb-ba69-e9a0bd57b0de\") " pod="openshift-ingress/payload-processing-e2e-api-a-69a4bc-69465f7c7-kvvq4"
Oct 31 00:51:58.161314 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:51:58.161196    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-flnfr\" (UniqueName: \"kubernetes.io/projected/79898810-a2f5-42bb-ba69-e9a0bd57b0de-kube-api-access-flnfr\") pod \"payload-processing-e2e-api-a-69a4bc-69465f7c7-kvvq4\" (UID: \"79898810-a2f5-42bb-ba69-e9a0bd57b0de\") " pod="openshift-ingress/payload-processing-e2e-api-a-69a4bc-69465f7c7-kvvq4"
Oct 31 00:51:58.247574 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:51:58.247500    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-69a4bc-854d4ddb7-5gwjd"
Oct 31 00:51:58.262419 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:51:58.262388    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-flnfr\" (UniqueName: \"kubernetes.io/projected/79898810-a2f5-42bb-ba69-e9a0bd57b0de-kube-api-access-flnfr\") pod \"payload-processing-e2e-api-a-69a4bc-69465f7c7-kvvq4\" (UID: \"79898810-a2f5-42bb-ba69-e9a0bd57b0de\") " pod="openshift-ingress/payload-processing-e2e-api-a-69a4bc-69465f7c7-kvvq4"
Oct 31 00:51:58.262666 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:51:58.262649    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/79898810-a2f5-42bb-ba69-e9a0bd57b0de-plugins-config-volume\") pod \"payload-processing-e2e-api-a-69a4bc-69465f7c7-kvvq4\" (UID: \"79898810-a2f5-42bb-ba69-e9a0bd57b0de\") " pod="openshift-ingress/payload-processing-e2e-api-a-69a4bc-69465f7c7-kvvq4"
Oct 31 00:51:58.263349 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:51:58.263328    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/79898810-a2f5-42bb-ba69-e9a0bd57b0de-plugins-config-volume\") pod \"payload-processing-e2e-api-a-69a4bc-69465f7c7-kvvq4\" (UID: \"79898810-a2f5-42bb-ba69-e9a0bd57b0de\") " pod="openshift-ingress/payload-processing-e2e-api-a-69a4bc-69465f7c7-kvvq4"
Oct 31 00:51:58.325549 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:51:58.325345    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-flnfr\" (UniqueName: \"kubernetes.io/projected/79898810-a2f5-42bb-ba69-e9a0bd57b0de-kube-api-access-flnfr\") pod \"payload-processing-e2e-api-a-69a4bc-69465f7c7-kvvq4\" (UID: \"79898810-a2f5-42bb-ba69-e9a0bd57b0de\") " pod="openshift-ingress/payload-processing-e2e-api-a-69a4bc-69465f7c7-kvvq4"
Oct 31 00:51:58.359143 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:51:58.359098    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-69a4bc-69465f7c7-kvvq4"
Oct 31 00:51:58.621129 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:51:58.621098    4284 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfa5fde86_708a_4f57_9e4b_3eb7e311e09c.slice/crio-5b5fd32e78dbd4dcadd04e9ae07410c30f5d2bbff439cf536bb6549f2dfd71e8 WatchSource:0}: Error finding container 5b5fd32e78dbd4dcadd04e9ae07410c30f5d2bbff439cf536bb6549f2dfd71e8: Status 404 returned error can't find the container with id 5b5fd32e78dbd4dcadd04e9ae07410c30f5d2bbff439cf536bb6549f2dfd71e8
Oct 31 00:51:58.621914 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:51:58.621888    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-69a4bc-854d4ddb7-5gwjd"]
Oct 31 00:51:58.760788 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:51:58.760758    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-69a4bc-69465f7c7-kvvq4"]
Oct 31 00:51:58.762037 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:51:58.762004    4284 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod79898810_a2f5_42bb_ba69_e9a0bd57b0de.slice/crio-cd346096d27b53882b386cb67cc53954693aa1063db437b1c9dd8e0198a2352a WatchSource:0}: Error finding container cd346096d27b53882b386cb67cc53954693aa1063db437b1c9dd8e0198a2352a: Status 404 returned error can't find the container with id cd346096d27b53882b386cb67cc53954693aa1063db437b1c9dd8e0198a2352a
Oct 31 00:51:59.093860 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:51:59.093819    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-69a4bc-854d4ddb7-5gwjd" event={"ID":"fa5fde86-708a-4f57-9e4b-3eb7e311e09c","Type":"ContainerStarted","Data":"53004a879cdd829a49f534c3bc6b222750a09e3ab8fa06b9a0f97b950f440728"}
Oct 31 00:51:59.094423 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:51:59.093881    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-69a4bc-854d4ddb7-5gwjd" event={"ID":"fa5fde86-708a-4f57-9e4b-3eb7e311e09c","Type":"ContainerStarted","Data":"5b5fd32e78dbd4dcadd04e9ae07410c30f5d2bbff439cf536bb6549f2dfd71e8"}
Oct 31 00:51:59.094423 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:51:59.093935    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-69a4bc-854d4ddb7-5gwjd"
Oct 31 00:51:59.095574 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:51:59.095550    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-69a4bc-69465f7c7-kvvq4" event={"ID":"79898810-a2f5-42bb-ba69-e9a0bd57b0de","Type":"ContainerStarted","Data":"4c1e7b95e5e485d5a7a158ba9fbeecc54dba0b184083e3d900396e49ba5c3507"}
Oct 31 00:51:59.095694 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:51:59.095592    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-a-69a4bc-69465f7c7-kvvq4"
Oct 31 00:51:59.095694 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:51:59.095608    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-69a4bc-69465f7c7-kvvq4" event={"ID":"79898810-a2f5-42bb-ba69-e9a0bd57b0de","Type":"ContainerStarted","Data":"cd346096d27b53882b386cb67cc53954693aa1063db437b1c9dd8e0198a2352a"}
Oct 31 00:51:59.210032 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:51:59.209964    4284 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-a-69a4bc-854d4ddb7-5gwjd" podStartSLOduration=2.209943157 podStartE2EDuration="2.209943157s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 00:51:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:51:59.205010091 +0000 UTC m=+1433.350017009" watchObservedRunningTime="2026-10-31 00:51:59.209943157 +0000 UTC m=+1433.354950075"
Oct 31 00:51:59.333417 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:51:59.325680    4284 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-a-69a4bc-69465f7c7-kvvq4" podStartSLOduration=2.325659635 podStartE2EDuration="2.325659635s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 00:51:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:51:59.322133321 +0000 UTC m=+1433.467140251" watchObservedRunningTime="2026-10-31 00:51:59.325659635 +0000 UTC m=+1433.470666552"
Oct 31 00:51:59.935930 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:51:59.935887    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:52:01.993312 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:52:01.993281    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:52:02.364297 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:52:02.357813    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:52:02.486250 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:52:02.471298    4284 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-3c636f-5fd8bf89c5-bdr88"]
Oct 31 00:52:02.518072 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:52:02.518042    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-3c636f-5fd8bf89c5-bdr88"]
Oct 31 00:52:02.518484 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:52:02.518407    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-3c636f-5fd8bf89c5-bdr88"
Oct 31 00:52:02.615394 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:52:02.610652    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/044d4725-d84b-4f94-a7d4-451cb16c2d6b-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-3c636f-5fd8bf89c5-bdr88\" (UID: \"044d4725-d84b-4f94-a7d4-451cb16c2d6b\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-3c636f-5fd8bf89c5-bdr88"
Oct 31 00:52:02.615394 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:52:02.610767    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5vlhx\" (UniqueName: \"kubernetes.io/projected/044d4725-d84b-4f94-a7d4-451cb16c2d6b-kube-api-access-5vlhx\") pod \"payload-pre-processing-e2e-api-b-3c636f-5fd8bf89c5-bdr88\" (UID: \"044d4725-d84b-4f94-a7d4-451cb16c2d6b\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-3c636f-5fd8bf89c5-bdr88"
Oct 31 00:52:02.657046 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:52:02.616381    4284 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-3c636f-6948699f5-hhkc2"]
Oct 31 00:52:02.657046 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:52:02.653269    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-3c636f-6948699f5-hhkc2"
Oct 31 00:52:02.657046 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:52:02.656672    4284 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-b-3c636f-dockercfg-4gpm7\""
Oct 31 00:52:02.657046 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:52:02.656774    4284 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-b-3c636f\""
Oct 31 00:52:02.678238 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:52:02.677317    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-3c636f-6948699f5-hhkc2"]
Oct 31 00:52:02.738506 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:52:02.738466    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/044d4725-d84b-4f94-a7d4-451cb16c2d6b-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-3c636f-5fd8bf89c5-bdr88\" (UID: \"044d4725-d84b-4f94-a7d4-451cb16c2d6b\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-3c636f-5fd8bf89c5-bdr88"
Oct 31 00:52:02.738719 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:52:02.738576    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5vlhx\" (UniqueName: \"kubernetes.io/projected/044d4725-d84b-4f94-a7d4-451cb16c2d6b-kube-api-access-5vlhx\") pod \"payload-pre-processing-e2e-api-b-3c636f-5fd8bf89c5-bdr88\" (UID: \"044d4725-d84b-4f94-a7d4-451cb16c2d6b\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-3c636f-5fd8bf89c5-bdr88"
Oct 31 00:52:02.738803 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:52:02.738784    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k2h8d\" (UniqueName: \"kubernetes.io/projected/53dde021-74fb-46c6-8340-6a84bc94a627-kube-api-access-k2h8d\") pod \"payload-processing-e2e-api-b-3c636f-6948699f5-hhkc2\" (UID: \"53dde021-74fb-46c6-8340-6a84bc94a627\") " pod="openshift-ingress/payload-processing-e2e-api-b-3c636f-6948699f5-hhkc2"
Oct 31 00:52:02.738870 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:52:02.738824    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/53dde021-74fb-46c6-8340-6a84bc94a627-plugins-config-volume\") pod \"payload-processing-e2e-api-b-3c636f-6948699f5-hhkc2\" (UID: \"53dde021-74fb-46c6-8340-6a84bc94a627\") " pod="openshift-ingress/payload-processing-e2e-api-b-3c636f-6948699f5-hhkc2"
Oct 31 00:52:02.740570 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:52:02.740535    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/044d4725-d84b-4f94-a7d4-451cb16c2d6b-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-3c636f-5fd8bf89c5-bdr88\" (UID: \"044d4725-d84b-4f94-a7d4-451cb16c2d6b\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-3c636f-5fd8bf89c5-bdr88"
Oct 31 00:52:02.843234 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:52:02.841537    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k2h8d\" (UniqueName: \"kubernetes.io/projected/53dde021-74fb-46c6-8340-6a84bc94a627-kube-api-access-k2h8d\") pod \"payload-processing-e2e-api-b-3c636f-6948699f5-hhkc2\" (UID: \"53dde021-74fb-46c6-8340-6a84bc94a627\") " pod="openshift-ingress/payload-processing-e2e-api-b-3c636f-6948699f5-hhkc2"
Oct 31 00:52:02.843234 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:52:02.841574    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/53dde021-74fb-46c6-8340-6a84bc94a627-plugins-config-volume\") pod \"payload-processing-e2e-api-b-3c636f-6948699f5-hhkc2\" (UID: \"53dde021-74fb-46c6-8340-6a84bc94a627\") " pod="openshift-ingress/payload-processing-e2e-api-b-3c636f-6948699f5-hhkc2"
Oct 31 00:52:02.843234 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:52:02.842314    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/53dde021-74fb-46c6-8340-6a84bc94a627-plugins-config-volume\") pod \"payload-processing-e2e-api-b-3c636f-6948699f5-hhkc2\" (UID: \"53dde021-74fb-46c6-8340-6a84bc94a627\") " pod="openshift-ingress/payload-processing-e2e-api-b-3c636f-6948699f5-hhkc2"
Oct 31 00:52:02.843234 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:52:02.842694    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5vlhx\" (UniqueName: \"kubernetes.io/projected/044d4725-d84b-4f94-a7d4-451cb16c2d6b-kube-api-access-5vlhx\") pod \"payload-pre-processing-e2e-api-b-3c636f-5fd8bf89c5-bdr88\" (UID: \"044d4725-d84b-4f94-a7d4-451cb16c2d6b\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-3c636f-5fd8bf89c5-bdr88"
Oct 31 00:52:02.901760 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:52:02.901679    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k2h8d\" (UniqueName: \"kubernetes.io/projected/53dde021-74fb-46c6-8340-6a84bc94a627-kube-api-access-k2h8d\") pod \"payload-processing-e2e-api-b-3c636f-6948699f5-hhkc2\" (UID: \"53dde021-74fb-46c6-8340-6a84bc94a627\") " pod="openshift-ingress/payload-processing-e2e-api-b-3c636f-6948699f5-hhkc2"
Oct 31 00:52:02.951679 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:52:02.951643    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-3c636f-5fd8bf89c5-bdr88"
Oct 31 00:52:03.051193 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:52:03.050882    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-3c636f-6948699f5-hhkc2"
Oct 31 00:52:03.960752 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:52:03.930559    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-3c636f-5fd8bf89c5-bdr88"]
Oct 31 00:52:04.169560 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:52:04.169444    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-3c636f-6948699f5-hhkc2"]
Oct 31 00:52:04.183934 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:52:04.183892    4284 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod53dde021_74fb_46c6_8340_6a84bc94a627.slice/crio-4d9b99596e9fbb3f1abac8e51b271db613f081242026d9eb14ceb4c40cc756c7 WatchSource:0}: Error finding container 4d9b99596e9fbb3f1abac8e51b271db613f081242026d9eb14ceb4c40cc756c7: Status 404 returned error can't find the container with id 4d9b99596e9fbb3f1abac8e51b271db613f081242026d9eb14ceb4c40cc756c7
Oct 31 00:52:04.521267 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:52:04.516148    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-3c636f-5fd8bf89c5-bdr88" event={"ID":"044d4725-d84b-4f94-a7d4-451cb16c2d6b","Type":"ContainerStarted","Data":"ae64bb430f49fc62f3dff9644bc33cfbf9bf5bbed7417eee8f338cab0a6c65ad"}
Oct 31 00:52:04.620238 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:52:04.611370    4284 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6cc458d887-rr5k6 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.39:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 31 00:52:04.620238 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:52:04.611487    4284 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-rr5k6" podUID="27f8cf21-0f47-4397-831a-bf716741dc67" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.39:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:52:05.447435 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:52:05.447400    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:52:05.526670 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:52:05.526635    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-3c636f-6948699f5-hhkc2" event={"ID":"53dde021-74fb-46c6-8340-6a84bc94a627","Type":"ContainerStarted","Data":"cd13b0606e45f7a97b8505fe06fffa3762158887131beccd349447ec44c17ed8"}
Oct 31 00:52:05.526880 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:52:05.526698    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-3c636f-6948699f5-hhkc2" event={"ID":"53dde021-74fb-46c6-8340-6a84bc94a627","Type":"ContainerStarted","Data":"4d9b99596e9fbb3f1abac8e51b271db613f081242026d9eb14ceb4c40cc756c7"}
Oct 31 00:52:05.526880 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:52:05.526768    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-b-3c636f-6948699f5-hhkc2"
Oct 31 00:52:05.528934 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:52:05.528896    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-3c636f-5fd8bf89c5-bdr88" event={"ID":"044d4725-d84b-4f94-a7d4-451cb16c2d6b","Type":"ContainerStarted","Data":"d52797a238599a5193274b48145d7f2dc3a9221ebad04c23e2bdaf64646e45c5"}
Oct 31 00:52:05.529065 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:52:05.528982    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-3c636f-5fd8bf89c5-bdr88"
Oct 31 00:52:05.737589 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:52:05.737485    4284 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-b-3c636f-6948699f5-hhkc2" podStartSLOduration=3.737470925 podStartE2EDuration="3.737470925s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 00:52:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:52:05.734840743 +0000 UTC m=+1439.879847660" watchObservedRunningTime="2026-10-31 00:52:05.737470925 +0000 UTC m=+1439.882477840"
Oct 31 00:52:05.863615 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:52:05.863537    4284 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-b-3c636f-5fd8bf89c5-bdr88" podStartSLOduration=3.86351444 podStartE2EDuration="3.86351444s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 00:52:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:52:05.84667817 +0000 UTC m=+1439.991685103" watchObservedRunningTime="2026-10-31 00:52:05.86351444 +0000 UTC m=+1440.008521355"
Oct 31 00:52:10.105627 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:52:10.105598    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-a-69a4bc-69465f7c7-kvvq4"
Oct 31 00:52:10.106047 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:52:10.105670    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-69a4bc-854d4ddb7-5gwjd"
Oct 31 00:52:10.291513 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:52:10.291475    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:52:16.137813 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:52:16.137766    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:52:16.538821 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:52:16.538787    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-b-3c636f-6948699f5-hhkc2"
Oct 31 00:52:16.538994 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:52:16.538857    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-3c636f-5fd8bf89c5-bdr88"
Oct 31 00:52:17.014062 ip-10-0-132-173 kubenswrapper[4284]: E1031 00:52:17.014025    4284 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fd8sp" containerName="extractor"
Oct 31 00:52:19.856351 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:52:19.856307    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:52:24.601016 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:52:24.600974    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:52:29.512343 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:52:29.512312    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:52:34.211090 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:52:34.211054    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:52:34.418085 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:52:34.418045    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:52:34.569317 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:52:34.569197    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:52:38.942664 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:52:38.942617    4284 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-3c636f-586fcffd55-lxv6t"]
Oct 31 00:52:38.948302 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:52:38.948272    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-3c636f-586fcffd55-lxv6t"
Oct 31 00:52:39.002622 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:52:39.002586    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-3c636f-586fcffd55-lxv6t"]
Oct 31 00:52:39.045471 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:52:39.045439    4284 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-3c636f-serving-cert\""
Oct 31 00:52:39.065615 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:52:39.065585    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/250cb265-61b0-4cb7-aac7-1b9a0f46c783-maas-api-tls\") pod \"maas-api-e2e-api-b-3c636f-586fcffd55-lxv6t\" (UID: \"250cb265-61b0-4cb7-aac7-1b9a0f46c783\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-3c636f-586fcffd55-lxv6t"
Oct 31 00:52:39.065807 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:52:39.065642    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/250cb265-61b0-4cb7-aac7-1b9a0f46c783-metrics-certs\") pod \"maas-api-e2e-api-b-3c636f-586fcffd55-lxv6t\" (UID: \"250cb265-61b0-4cb7-aac7-1b9a0f46c783\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-3c636f-586fcffd55-lxv6t"
Oct 31 00:52:39.065900 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:52:39.065879    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/250cb265-61b0-4cb7-aac7-1b9a0f46c783-openshift-service-ca\") pod \"maas-api-e2e-api-b-3c636f-586fcffd55-lxv6t\" (UID: \"250cb265-61b0-4cb7-aac7-1b9a0f46c783\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-3c636f-586fcffd55-lxv6t"
Oct 31 00:52:39.065959 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:52:39.065913    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wfz5c\" (UniqueName: \"kubernetes.io/projected/250cb265-61b0-4cb7-aac7-1b9a0f46c783-kube-api-access-wfz5c\") pod \"maas-api-e2e-api-b-3c636f-586fcffd55-lxv6t\" (UID: \"250cb265-61b0-4cb7-aac7-1b9a0f46c783\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-3c636f-586fcffd55-lxv6t"
Oct 31 00:52:39.131448 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:52:39.131412    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:52:39.167741 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:52:39.167701    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/250cb265-61b0-4cb7-aac7-1b9a0f46c783-maas-api-tls\") pod \"maas-api-e2e-api-b-3c636f-586fcffd55-lxv6t\" (UID: \"250cb265-61b0-4cb7-aac7-1b9a0f46c783\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-3c636f-586fcffd55-lxv6t"
Oct 31 00:52:39.167912 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:52:39.167795    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/250cb265-61b0-4cb7-aac7-1b9a0f46c783-metrics-certs\") pod \"maas-api-e2e-api-b-3c636f-586fcffd55-lxv6t\" (UID: \"250cb265-61b0-4cb7-aac7-1b9a0f46c783\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-3c636f-586fcffd55-lxv6t"
Oct 31 00:52:39.167912 ip-10-0-132-173 kubenswrapper[4284]: E1031 00:52:39.167851    4284 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-b-3c636f-serving-cert: secret "maas-api-e2e-api-b-3c636f-serving-cert" not found
Oct 31 00:52:39.168040 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:52:39.167914    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/250cb265-61b0-4cb7-aac7-1b9a0f46c783-openshift-service-ca\") pod \"maas-api-e2e-api-b-3c636f-586fcffd55-lxv6t\" (UID: \"250cb265-61b0-4cb7-aac7-1b9a0f46c783\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-3c636f-586fcffd55-lxv6t"
Oct 31 00:52:39.168040 ip-10-0-132-173 kubenswrapper[4284]: E1031 00:52:39.167928    4284 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/250cb265-61b0-4cb7-aac7-1b9a0f46c783-maas-api-tls podName:250cb265-61b0-4cb7-aac7-1b9a0f46c783 nodeName:}" failed. No retries permitted until 2026-10-31 00:52:39.667908937 +0000 UTC m=+1473.812915844 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/250cb265-61b0-4cb7-aac7-1b9a0f46c783-maas-api-tls") pod "maas-api-e2e-api-b-3c636f-586fcffd55-lxv6t" (UID: "250cb265-61b0-4cb7-aac7-1b9a0f46c783") : secret "maas-api-e2e-api-b-3c636f-serving-cert" not found
Oct 31 00:52:39.168040 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:52:39.167959    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wfz5c\" (UniqueName: \"kubernetes.io/projected/250cb265-61b0-4cb7-aac7-1b9a0f46c783-kube-api-access-wfz5c\") pod \"maas-api-e2e-api-b-3c636f-586fcffd55-lxv6t\" (UID: \"250cb265-61b0-4cb7-aac7-1b9a0f46c783\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-3c636f-586fcffd55-lxv6t"
Oct 31 00:52:39.168793 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:52:39.168765    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/250cb265-61b0-4cb7-aac7-1b9a0f46c783-openshift-service-ca\") pod \"maas-api-e2e-api-b-3c636f-586fcffd55-lxv6t\" (UID: \"250cb265-61b0-4cb7-aac7-1b9a0f46c783\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-3c636f-586fcffd55-lxv6t"
Oct 31 00:52:39.185273 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:52:39.185116    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/250cb265-61b0-4cb7-aac7-1b9a0f46c783-metrics-certs\") pod \"maas-api-e2e-api-b-3c636f-586fcffd55-lxv6t\" (UID: \"250cb265-61b0-4cb7-aac7-1b9a0f46c783\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-3c636f-586fcffd55-lxv6t"
Oct 31 00:52:39.237118 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:52:39.237040    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wfz5c\" (UniqueName: \"kubernetes.io/projected/250cb265-61b0-4cb7-aac7-1b9a0f46c783-kube-api-access-wfz5c\") pod \"maas-api-e2e-api-b-3c636f-586fcffd55-lxv6t\" (UID: \"250cb265-61b0-4cb7-aac7-1b9a0f46c783\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-3c636f-586fcffd55-lxv6t"
Oct 31 00:52:39.677056 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:52:39.677018    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/250cb265-61b0-4cb7-aac7-1b9a0f46c783-maas-api-tls\") pod \"maas-api-e2e-api-b-3c636f-586fcffd55-lxv6t\" (UID: \"250cb265-61b0-4cb7-aac7-1b9a0f46c783\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-3c636f-586fcffd55-lxv6t"
Oct 31 00:52:39.679694 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:52:39.679666    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/250cb265-61b0-4cb7-aac7-1b9a0f46c783-maas-api-tls\") pod \"maas-api-e2e-api-b-3c636f-586fcffd55-lxv6t\" (UID: \"250cb265-61b0-4cb7-aac7-1b9a0f46c783\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-3c636f-586fcffd55-lxv6t"
Oct 31 00:52:39.862740 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:52:39.862708    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-3c636f-586fcffd55-lxv6t"
Oct 31 00:52:40.128737 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:52:40.128703    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:52:40.241768 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:52:40.241738    4284 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod250cb265_61b0_4cb7_aac7_1b9a0f46c783.slice/crio-74497a73825e5c4db7407d96e45e93ccbe04bfe5748144a05c5e192474be93a7 WatchSource:0}: Error finding container 74497a73825e5c4db7407d96e45e93ccbe04bfe5748144a05c5e192474be93a7: Status 404 returned error can't find the container with id 74497a73825e5c4db7407d96e45e93ccbe04bfe5748144a05c5e192474be93a7
Oct 31 00:52:40.242454 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:52:40.242432    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-3c636f-586fcffd55-lxv6t"]
Oct 31 00:52:40.847655 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:52:40.847621    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-3c636f-586fcffd55-lxv6t" event={"ID":"250cb265-61b0-4cb7-aac7-1b9a0f46c783","Type":"ContainerStarted","Data":"74497a73825e5c4db7407d96e45e93ccbe04bfe5748144a05c5e192474be93a7"}
Oct 31 00:52:41.944727 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:52:41.944687    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-3c636f-586fcffd55-lxv6t" event={"ID":"250cb265-61b0-4cb7-aac7-1b9a0f46c783","Type":"ContainerStarted","Data":"af9cccb3cb693970e7ce6b540b8e456f82d5f520d1e0a2208e9e6cd11001823b"}
Oct 31 00:52:41.946656 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:52:41.946635    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-3c636f-586fcffd55-lxv6t"
Oct 31 00:52:42.062426 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:52:42.062373    4284 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-3c636f-586fcffd55-lxv6t" podStartSLOduration=3.075545866 podStartE2EDuration="4.062355749s" totalImagesPullingTime="986.809883ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 00:52:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:52:42.060552977 +0000 UTC m=+1476.205559892" watchObservedRunningTime="2026-10-31 00:52:42.062355749 +0000 UTC m=+1476.207362664"
Oct 31 00:52:44.066911 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:52:44.066871    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:52:46.322047 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:52:46.321995    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:52:47.015283 ip-10-0-132-173 kubenswrapper[4284]: E1031 00:52:47.015247    4284 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fd8sp" containerName="extractor"
Oct 31 00:52:47.145280 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:52:47.145233    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:52:48.825090 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:52:48.825054    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:52:48.969938 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:52:48.969404    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-3c636f-586fcffd55-lxv6t"
Oct 31 00:52:53.809283 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:52:53.809241    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:52:56.134003 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:52:56.133971    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:52:56.289797 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:52:56.289760    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:52:57.162454 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:52:57.162406    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:52:57.364956 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:52:57.364921    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:52:58.696032 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:52:58.695987    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:53:03.115281 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:53:03.115251    4284 generic.go:356] "Generic (PLEG): container finished" podID="66210261-9baa-4961-855e-b92e2c4e154e" containerID="ec2024aa775dd9368e0a4c1c2655250955a14d7564e3d508b509215a11518476" exitCode=0
Oct 31 00:53:03.115732 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:53:03.115306    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-69a4bc-openshift-default-797b9d8659-rgs5q" event={"ID":"66210261-9baa-4961-855e-b92e2c4e154e","Type":"ContainerDied","Data":"ec2024aa775dd9368e0a4c1c2655250955a14d7564e3d508b509215a11518476"}
Oct 31 00:53:03.115784 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:53:03.115739    4284 scope.go:122] "RemoveContainer" containerID="ec2024aa775dd9368e0a4c1c2655250955a14d7564e3d508b509215a11518476"
Oct 31 00:53:03.117250 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:53:03.117201    4284 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 00:53:03.117306 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:53:03.117291    4284 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 00:53:03.117352 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:53:03.117340    4284 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 00:53:03.491606 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:53:03.491567    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:53:04.123106 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:53:04.123064    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-69a4bc-openshift-default-797b9d8659-rgs5q" event={"ID":"66210261-9baa-4961-855e-b92e2c4e154e","Type":"ContainerStarted","Data":"b19053f555a381b9431742ed6595a28a718339359ee7db011f0ede450776068a"}
Oct 31 00:53:04.136566 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:53:04.136534    4284 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-69a4bc-openshift-default-797b9d8659-rgs5q"
Oct 31 00:53:04.138037 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:53:04.138012    4284 patch_prober.go:29] interesting pod/e2e-api-a-69a4bc-openshift-default-797b9d8659-rgs5q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.86:15021/healthz/ready\": dial tcp 10.134.0.86:15021: connect: connection refused" start-of-body=
Oct 31 00:53:04.138149 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:53:04.138064    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-69a4bc-openshift-default-797b9d8659-rgs5q" podUID="66210261-9baa-4961-855e-b92e2c4e154e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.86:15021/healthz/ready\": dial tcp 10.134.0.86:15021: connect: connection refused"
Oct 31 00:53:05.137039 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:53:05.137008    4284 patch_prober.go:29] interesting pod/e2e-api-a-69a4bc-openshift-default-797b9d8659-rgs5q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.86:15021/healthz/ready\": dial tcp 10.134.0.86:15021: connect: connection refused" start-of-body=
Oct 31 00:53:05.137556 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:53:05.137072    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-69a4bc-openshift-default-797b9d8659-rgs5q" podUID="66210261-9baa-4961-855e-b92e2c4e154e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.86:15021/healthz/ready\": dial tcp 10.134.0.86:15021: connect: connection refused"
Oct 31 00:53:06.136533 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:53:06.136501    4284 patch_prober.go:29] interesting pod/e2e-api-a-69a4bc-openshift-default-797b9d8659-rgs5q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.86:15021/healthz/ready\": dial tcp 10.134.0.86:15021: connect: connection refused" start-of-body=
Oct 31 00:53:06.136728 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:53:06.136549    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-69a4bc-openshift-default-797b9d8659-rgs5q" podUID="66210261-9baa-4961-855e-b92e2c4e154e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.86:15021/healthz/ready\": dial tcp 10.134.0.86:15021: connect: connection refused"
Oct 31 00:53:07.044311 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:53:07.044279    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-69a4bc-openshift-default-797b9d8659-rgs5q"
Oct 31 00:53:07.108237 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:53:07.107200    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:53:07.136804 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:53:07.136768    4284 patch_prober.go:29] interesting pod/e2e-api-a-69a4bc-openshift-default-797b9d8659-rgs5q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.86:15021/healthz/ready\": dial tcp 10.134.0.86:15021: connect: connection refused" start-of-body=
Oct 31 00:53:07.137066 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:53:07.137037    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-69a4bc-openshift-default-797b9d8659-rgs5q" podUID="66210261-9baa-4961-855e-b92e2c4e154e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.86:15021/healthz/ready\": dial tcp 10.134.0.86:15021: connect: connection refused"
Oct 31 00:53:07.446269 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:53:07.446234    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:53:08.137125 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:53:08.137090    4284 patch_prober.go:29] interesting pod/e2e-api-a-69a4bc-openshift-default-797b9d8659-rgs5q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.86:15021/healthz/ready\": dial tcp 10.134.0.86:15021: connect: connection refused" start-of-body=
Oct 31 00:53:08.137554 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:53:08.137150    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-69a4bc-openshift-default-797b9d8659-rgs5q" podUID="66210261-9baa-4961-855e-b92e2c4e154e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.86:15021/healthz/ready\": dial tcp 10.134.0.86:15021: connect: connection refused"
Oct 31 00:53:08.320366 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:53:08.320334    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:53:09.136652 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:53:09.136614    4284 patch_prober.go:29] interesting pod/e2e-api-a-69a4bc-openshift-default-797b9d8659-rgs5q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.86:15021/healthz/ready\": dial tcp 10.134.0.86:15021: connect: connection refused" start-of-body=
Oct 31 00:53:09.136834 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:53:09.136693    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-69a4bc-openshift-default-797b9d8659-rgs5q" podUID="66210261-9baa-4961-855e-b92e2c4e154e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.86:15021/healthz/ready\": dial tcp 10.134.0.86:15021: connect: connection refused"
Oct 31 00:53:10.142064 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:53:10.142022    4284 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-69a4bc-openshift-default-797b9d8659-rgs5q"
Oct 31 00:53:10.143286 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:53:10.143267    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-69a4bc-openshift-default-797b9d8659-rgs5q"
Oct 31 00:53:10.147441 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:53:10.147409    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:53:10.294556 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:53:10.294514    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:53:13.087277 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:53:13.087236    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:53:17.015158 ip-10-0-132-173 kubenswrapper[4284]: E1031 00:53:17.015121    4284 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fd8sp" containerName="extractor"
Oct 31 00:53:18.240281 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:53:18.240187    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:53:23.530973 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:53:23.530934    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:53:27.443818 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:53:27.443782    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-7e6fe88b-558b6b9b97-292vk"]
Oct 31 00:53:27.818292 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:53:27.817414    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:53:28.030797 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:53:28.030760    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:53:28.718458 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:53:28.718422    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:53:28.909432 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:53:28.909393    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:53:31.125668 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:53:31.125632    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:53:31.882008 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:53:31.881970    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:53:32.444941 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:53:32.444884    4284 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-7e6fe88b-558b6b9b97-292vk" podUID="2b942b8c-c3f7-4a18-9dc2-59de27841c78" containerName="payload-processing" containerID="cri-o://89c7398f23611b40d68ef9510e3a99335569b9cbe0c5d69c1dd266e52ae2a873" gracePeriod=25
Oct 31 00:53:32.475915 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:53:32.475885    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:53:32.721876 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:53:32.721790    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:53:34.011143 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:53:34.011103    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:53:35.386778 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:53:35.386741    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:53:36.880411 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:53:36.880370    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:53:37.070302 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:53:37.070259    4284 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-mt-7e6fe88b-558b6b9b97-292vk" podUID="2b942b8c-c3f7-4a18-9dc2-59de27841c78" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 31 00:53:37.096984 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:53:37.096923    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:53:37.257202 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:53:37.257161    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:53:39.142628 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:53:39.142584    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:53:40.154002 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:53:40.153967    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:53:40.468376 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:53:40.468269    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:53:40.565166 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:53:40.563521    4284 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-e942ab-7db488f9ff-qqfk7"]
Oct 31 00:53:40.583597 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:53:40.583567    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-e942ab-7db488f9ff-qqfk7"
Oct 31 00:53:40.636727 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:53:40.636696    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-e942ab-7db488f9ff-qqfk7"]
Oct 31 00:53:40.698253 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:53:40.691995    4284 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-b-e942ab-dockercfg-vwq9r\""
Oct 31 00:53:40.698253 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:53:40.692084    4284 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-b-e942ab\""
Oct 31 00:53:40.698253 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:53:40.696523    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5d8dt\" (UniqueName: \"kubernetes.io/projected/35f802b7-40d6-4f42-a069-941c04f99562-kube-api-access-5d8dt\") pod \"payload-pre-processing-e2e-api-b-e942ab-7db488f9ff-qqfk7\" (UID: \"35f802b7-40d6-4f42-a069-941c04f99562\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-e942ab-7db488f9ff-qqfk7"
Oct 31 00:53:40.698253 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:53:40.696682    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/35f802b7-40d6-4f42-a069-941c04f99562-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-e942ab-7db488f9ff-qqfk7\" (UID: \"35f802b7-40d6-4f42-a069-941c04f99562\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-e942ab-7db488f9ff-qqfk7"
Oct 31 00:53:40.870324 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:53:40.859931    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5d8dt\" (UniqueName: \"kubernetes.io/projected/35f802b7-40d6-4f42-a069-941c04f99562-kube-api-access-5d8dt\") pod \"payload-pre-processing-e2e-api-b-e942ab-7db488f9ff-qqfk7\" (UID: \"35f802b7-40d6-4f42-a069-941c04f99562\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-e942ab-7db488f9ff-qqfk7"
Oct 31 00:53:40.870324 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:53:40.860026    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/35f802b7-40d6-4f42-a069-941c04f99562-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-e942ab-7db488f9ff-qqfk7\" (UID: \"35f802b7-40d6-4f42-a069-941c04f99562\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-e942ab-7db488f9ff-qqfk7"
Oct 31 00:53:40.949905 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:53:40.911447    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/35f802b7-40d6-4f42-a069-941c04f99562-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-e942ab-7db488f9ff-qqfk7\" (UID: \"35f802b7-40d6-4f42-a069-941c04f99562\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-e942ab-7db488f9ff-qqfk7"
Oct 31 00:53:40.999191 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:53:40.996492    4284 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-e942ab-7979b88bd7-s4r4v"]
Oct 31 00:53:41.007291 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:53:41.003519    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5d8dt\" (UniqueName: \"kubernetes.io/projected/35f802b7-40d6-4f42-a069-941c04f99562-kube-api-access-5d8dt\") pod \"payload-pre-processing-e2e-api-b-e942ab-7db488f9ff-qqfk7\" (UID: \"35f802b7-40d6-4f42-a069-941c04f99562\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-e942ab-7db488f9ff-qqfk7"
Oct 31 00:53:41.007291 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:53:41.005400    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-e942ab-7979b88bd7-s4r4v"
Oct 31 00:53:41.007291 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:53:41.006714    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-e942ab-7979b88bd7-s4r4v"]
Oct 31 00:53:41.017928 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:53:41.017652    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-e942ab-7db488f9ff-qqfk7"
Oct 31 00:53:41.168808 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:53:41.168760    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c940cc14-8405-40e8-8698-db68813e6e2d-plugins-config-volume\") pod \"payload-processing-e2e-api-b-e942ab-7979b88bd7-s4r4v\" (UID: \"c940cc14-8405-40e8-8698-db68813e6e2d\") " pod="openshift-ingress/payload-processing-e2e-api-b-e942ab-7979b88bd7-s4r4v"
Oct 31 00:53:41.169286 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:53:41.169019    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-75czz\" (UniqueName: \"kubernetes.io/projected/c940cc14-8405-40e8-8698-db68813e6e2d-kube-api-access-75czz\") pod \"payload-processing-e2e-api-b-e942ab-7979b88bd7-s4r4v\" (UID: \"c940cc14-8405-40e8-8698-db68813e6e2d\") " pod="openshift-ingress/payload-processing-e2e-api-b-e942ab-7979b88bd7-s4r4v"
Oct 31 00:53:41.270730 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:53:41.270697    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-75czz\" (UniqueName: \"kubernetes.io/projected/c940cc14-8405-40e8-8698-db68813e6e2d-kube-api-access-75czz\") pod \"payload-processing-e2e-api-b-e942ab-7979b88bd7-s4r4v\" (UID: \"c940cc14-8405-40e8-8698-db68813e6e2d\") " pod="openshift-ingress/payload-processing-e2e-api-b-e942ab-7979b88bd7-s4r4v"
Oct 31 00:53:41.270910 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:53:41.270856    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c940cc14-8405-40e8-8698-db68813e6e2d-plugins-config-volume\") pod \"payload-processing-e2e-api-b-e942ab-7979b88bd7-s4r4v\" (UID: \"c940cc14-8405-40e8-8698-db68813e6e2d\") " pod="openshift-ingress/payload-processing-e2e-api-b-e942ab-7979b88bd7-s4r4v"
Oct 31 00:53:41.271537 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:53:41.271518    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c940cc14-8405-40e8-8698-db68813e6e2d-plugins-config-volume\") pod \"payload-processing-e2e-api-b-e942ab-7979b88bd7-s4r4v\" (UID: \"c940cc14-8405-40e8-8698-db68813e6e2d\") " pod="openshift-ingress/payload-processing-e2e-api-b-e942ab-7979b88bd7-s4r4v"
Oct 31 00:53:41.329701 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:53:41.329665    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-75czz\" (UniqueName: \"kubernetes.io/projected/c940cc14-8405-40e8-8698-db68813e6e2d-kube-api-access-75czz\") pod \"payload-processing-e2e-api-b-e942ab-7979b88bd7-s4r4v\" (UID: \"c940cc14-8405-40e8-8698-db68813e6e2d\") " pod="openshift-ingress/payload-processing-e2e-api-b-e942ab-7979b88bd7-s4r4v"
Oct 31 00:53:41.336692 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:53:41.336667    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-e942ab-7979b88bd7-s4r4v"
Oct 31 00:53:41.402253 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:53:41.401859    4284 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod35f802b7_40d6_4f42_a069_941c04f99562.slice/crio-33ff7fd58c03e3407f8890ecb1756a45a74a6a06d072253c367de8d15c92f51e WatchSource:0}: Error finding container 33ff7fd58c03e3407f8890ecb1756a45a74a6a06d072253c367de8d15c92f51e: Status 404 returned error can't find the container with id 33ff7fd58c03e3407f8890ecb1756a45a74a6a06d072253c367de8d15c92f51e
Oct 31 00:53:41.404452 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:53:41.404032    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-e942ab-7db488f9ff-qqfk7"]
Oct 31 00:53:41.527299 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:53:41.527262    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-e942ab-7db488f9ff-qqfk7" event={"ID":"35f802b7-40d6-4f42-a069-941c04f99562","Type":"ContainerStarted","Data":"95747a97a4f778a59e50dddc240abcc1c7962ee95828a801341672fc7c01ee10"}
Oct 31 00:53:41.527470 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:53:41.527328    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-e942ab-7db488f9ff-qqfk7" event={"ID":"35f802b7-40d6-4f42-a069-941c04f99562","Type":"ContainerStarted","Data":"33ff7fd58c03e3407f8890ecb1756a45a74a6a06d072253c367de8d15c92f51e"}
Oct 31 00:53:41.527558 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:53:41.527465    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-e942ab-7db488f9ff-qqfk7"
Oct 31 00:53:41.639102 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:53:41.639048    4284 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-b-e942ab-7db488f9ff-qqfk7" podStartSLOduration=1.639035846 podStartE2EDuration="1.639035846s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 00:53:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:53:41.638348692 +0000 UTC m=+1535.783355607" watchObservedRunningTime="2026-10-31 00:53:41.639035846 +0000 UTC m=+1535.784042761"
Oct 31 00:53:41.708810 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:53:41.708781    4284 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc940cc14_8405_40e8_8698_db68813e6e2d.slice/crio-429774e68afad07dfddf800cf55ca81c324d33e18377c2d9ec245ccf99b5eaa7 WatchSource:0}: Error finding container 429774e68afad07dfddf800cf55ca81c324d33e18377c2d9ec245ccf99b5eaa7: Status 404 returned error can't find the container with id 429774e68afad07dfddf800cf55ca81c324d33e18377c2d9ec245ccf99b5eaa7
Oct 31 00:53:41.708922 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:53:41.708813    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-e942ab-7979b88bd7-s4r4v"]
Oct 31 00:53:42.536648 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:53:42.536600    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-e942ab-7979b88bd7-s4r4v" event={"ID":"c940cc14-8405-40e8-8698-db68813e6e2d","Type":"ContainerStarted","Data":"63ab012d55a3f78b825b32ae20b3c20285a6259463ed910a3367d69c4569f449"}
Oct 31 00:53:42.537060 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:53:42.536670    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-e942ab-7979b88bd7-s4r4v" event={"ID":"c940cc14-8405-40e8-8698-db68813e6e2d","Type":"ContainerStarted","Data":"429774e68afad07dfddf800cf55ca81c324d33e18377c2d9ec245ccf99b5eaa7"}
Oct 31 00:53:42.537060 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:53:42.536932    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-b-e942ab-7979b88bd7-s4r4v"
Oct 31 00:53:42.651172 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:53:42.651123    4284 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-b-e942ab-7979b88bd7-s4r4v" podStartSLOduration=2.651107741 podStartE2EDuration="2.651107741s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 00:53:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:53:42.64826553 +0000 UTC m=+1536.793272445" watchObservedRunningTime="2026-10-31 00:53:42.651107741 +0000 UTC m=+1536.796114693"
Oct 31 00:53:44.693241 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:53:44.689828    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:53:47.015024 ip-10-0-132-173 kubenswrapper[4284]: E1031 00:53:47.014994    4284 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fd8sp" containerName="extractor"
Oct 31 00:53:47.069787 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:53:47.069747    4284 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-mt-7e6fe88b-558b6b9b97-292vk" podUID="2b942b8c-c3f7-4a18-9dc2-59de27841c78" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 31 00:53:48.822356 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:53:48.822321    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:53:48.990144 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:53:48.990109    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:53:49.599849 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:53:49.599811    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:53:51.126079 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:53:51.126043    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-8c2357-655589dfb7-9txvh"]
Oct 31 00:53:51.473359 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:53:51.473321    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:53:51.615099 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:53:51.615066    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:53:52.552462 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:53:52.552433    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-e942ab-7db488f9ff-qqfk7"
Oct 31 00:53:52.897044 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:53:52.896961    4284 generic.go:356] "Generic (PLEG): container finished" podID="2b942b8c-c3f7-4a18-9dc2-59de27841c78" containerID="89c7398f23611b40d68ef9510e3a99335569b9cbe0c5d69c1dd266e52ae2a873" exitCode=0
Oct 31 00:53:52.897505 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:53:52.897475    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-7e6fe88b-558b6b9b97-292vk" event={"ID":"2b942b8c-c3f7-4a18-9dc2-59de27841c78","Type":"ContainerDied","Data":"89c7398f23611b40d68ef9510e3a99335569b9cbe0c5d69c1dd266e52ae2a873"}
Oct 31 00:53:53.282355 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:53:53.282320    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-7e6fe88b-558b6b9b97-292vk"
Oct 31 00:53:53.440131 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:53:53.440099    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2b942b8c-c3f7-4a18-9dc2-59de27841c78-kube-api-access-qmwcs\" (UniqueName: \"kubernetes.io/projected/2b942b8c-c3f7-4a18-9dc2-59de27841c78-kube-api-access-qmwcs\") pod \"2b942b8c-c3f7-4a18-9dc2-59de27841c78\" (UID: \"2b942b8c-c3f7-4a18-9dc2-59de27841c78\") "
Oct 31 00:53:53.440373 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:53:53.440339    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/2b942b8c-c3f7-4a18-9dc2-59de27841c78-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2b942b8c-c3f7-4a18-9dc2-59de27841c78-plugins-config-volume\") pod \"2b942b8c-c3f7-4a18-9dc2-59de27841c78\" (UID: \"2b942b8c-c3f7-4a18-9dc2-59de27841c78\") "
Oct 31 00:53:53.440783 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:53:53.440751    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2b942b8c-c3f7-4a18-9dc2-59de27841c78-plugins-config-volume" pod "2b942b8c-c3f7-4a18-9dc2-59de27841c78" (UID: "2b942b8c-c3f7-4a18-9dc2-59de27841c78"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 00:53:53.441041 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:53:53.440963    4284 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2b942b8c-c3f7-4a18-9dc2-59de27841c78-plugins-config-volume\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:53:53.442922 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:53:53.442895    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2b942b8c-c3f7-4a18-9dc2-59de27841c78-kube-api-access-qmwcs" pod "2b942b8c-c3f7-4a18-9dc2-59de27841c78" (UID: "2b942b8c-c3f7-4a18-9dc2-59de27841c78"). InnerVolumeSpecName "kube-api-access-qmwcs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 00:53:53.542147 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:53:53.542065    4284 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qmwcs\" (UniqueName: \"kubernetes.io/projected/2b942b8c-c3f7-4a18-9dc2-59de27841c78-kube-api-access-qmwcs\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:53:53.545965 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:53:53.545942    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-b-e942ab-7979b88bd7-s4r4v"
Oct 31 00:53:53.905162 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:53:53.905128    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-7e6fe88b-558b6b9b97-292vk" event={"ID":"2b942b8c-c3f7-4a18-9dc2-59de27841c78","Type":"ContainerDied","Data":"42737623750b7d7d206a5e0d5c357d319103924eda7122e297ccb82ca6845857"}
Oct 31 00:53:53.905610 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:53:53.905185    4284 scope.go:122] "RemoveContainer" containerID="89c7398f23611b40d68ef9510e3a99335569b9cbe0c5d69c1dd266e52ae2a873"
Oct 31 00:53:53.905610 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:53:53.905141    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-7e6fe88b-558b6b9b97-292vk"
Oct 31 00:53:54.064538 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:53:54.064502    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-7e6fe88b-558b6b9b97-292vk"]
Oct 31 00:53:54.069531 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:53:54.069497    4284 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-7e6fe88b-558b6b9b97-292vk"]
Oct 31 00:53:54.714741 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:53:54.714696    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:53:55.964648 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:53:55.964619    4284 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2b942b8c-c3f7-4a18-9dc2-59de27841c78" path="/var/lib/kubelet/pods/2b942b8c-c3f7-4a18-9dc2-59de27841c78/volumes"
Oct 31 00:53:56.126555 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:53:56.126509    4284 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-a-8c2357-655589dfb7-9txvh" podUID="d29ce763-9fd5-4bb3-ac9b-c8209da9d59a" containerName="payload-processing" containerID="cri-o://607a9748c4d27b5d86c4155ff9bd0f440176f5804ced8ed9abfc45309fb8ebff" gracePeriod=25
Oct 31 00:53:56.401862 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:53:56.401834    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-8c2357-655589dfb7-9txvh"
Oct 31 00:53:56.422270 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:53:56.422239    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d29ce763-9fd5-4bb3-ac9b-c8209da9d59a-kube-api-access-6nqsz\" (UniqueName: \"kubernetes.io/projected/d29ce763-9fd5-4bb3-ac9b-c8209da9d59a-kube-api-access-6nqsz\") pod \"d29ce763-9fd5-4bb3-ac9b-c8209da9d59a\" (UID: \"d29ce763-9fd5-4bb3-ac9b-c8209da9d59a\") "
Oct 31 00:53:56.422427 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:53:56.422380    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d29ce763-9fd5-4bb3-ac9b-c8209da9d59a-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d29ce763-9fd5-4bb3-ac9b-c8209da9d59a-plugins-config-volume\") pod \"d29ce763-9fd5-4bb3-ac9b-c8209da9d59a\" (UID: \"d29ce763-9fd5-4bb3-ac9b-c8209da9d59a\") "
Oct 31 00:53:56.422730 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:53:56.422705    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d29ce763-9fd5-4bb3-ac9b-c8209da9d59a-plugins-config-volume" pod "d29ce763-9fd5-4bb3-ac9b-c8209da9d59a" (UID: "d29ce763-9fd5-4bb3-ac9b-c8209da9d59a"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 00:53:56.422840 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:53:56.422817    4284 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d29ce763-9fd5-4bb3-ac9b-c8209da9d59a-plugins-config-volume\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:53:56.424502 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:53:56.424477    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d29ce763-9fd5-4bb3-ac9b-c8209da9d59a-kube-api-access-6nqsz" pod "d29ce763-9fd5-4bb3-ac9b-c8209da9d59a" (UID: "d29ce763-9fd5-4bb3-ac9b-c8209da9d59a"). InnerVolumeSpecName "kube-api-access-6nqsz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 00:53:56.524265 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:53:56.524171    4284 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6nqsz\" (UniqueName: \"kubernetes.io/projected/d29ce763-9fd5-4bb3-ac9b-c8209da9d59a-kube-api-access-6nqsz\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:53:56.927163 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:53:56.927126    4284 generic.go:356] "Generic (PLEG): container finished" podID="d29ce763-9fd5-4bb3-ac9b-c8209da9d59a" containerID="607a9748c4d27b5d86c4155ff9bd0f440176f5804ced8ed9abfc45309fb8ebff" exitCode=0
Oct 31 00:53:56.927373 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:53:56.927183    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-8c2357-655589dfb7-9txvh"
Oct 31 00:53:56.927373 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:53:56.927242    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-8c2357-655589dfb7-9txvh" event={"ID":"d29ce763-9fd5-4bb3-ac9b-c8209da9d59a","Type":"ContainerDied","Data":"607a9748c4d27b5d86c4155ff9bd0f440176f5804ced8ed9abfc45309fb8ebff"}
Oct 31 00:53:56.927373 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:53:56.927317    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-8c2357-655589dfb7-9txvh" event={"ID":"d29ce763-9fd5-4bb3-ac9b-c8209da9d59a","Type":"ContainerDied","Data":"3fcfaa7b40b345aac21bf4a0360c2a57774f369afbcad04fc1f09f600269d24c"}
Oct 31 00:53:56.927373 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:53:56.927341    4284 scope.go:122] "RemoveContainer" containerID="607a9748c4d27b5d86c4155ff9bd0f440176f5804ced8ed9abfc45309fb8ebff"
Oct 31 00:53:56.938006 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:53:56.937984    4284 scope.go:122] "RemoveContainer" containerID="607a9748c4d27b5d86c4155ff9bd0f440176f5804ced8ed9abfc45309fb8ebff"
Oct 31 00:53:56.938366 ip-10-0-132-173 kubenswrapper[4284]: E1031 00:53:56.938345    4284 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"607a9748c4d27b5d86c4155ff9bd0f440176f5804ced8ed9abfc45309fb8ebff\": container with ID starting with 607a9748c4d27b5d86c4155ff9bd0f440176f5804ced8ed9abfc45309fb8ebff not found: ID does not exist" containerID="607a9748c4d27b5d86c4155ff9bd0f440176f5804ced8ed9abfc45309fb8ebff"
Oct 31 00:53:56.938451 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:53:56.938376    4284 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"607a9748c4d27b5d86c4155ff9bd0f440176f5804ced8ed9abfc45309fb8ebff"} err="failed to get container status \"607a9748c4d27b5d86c4155ff9bd0f440176f5804ced8ed9abfc45309fb8ebff\": rpc error: code = NotFound desc = could not find container \"607a9748c4d27b5d86c4155ff9bd0f440176f5804ced8ed9abfc45309fb8ebff\": container with ID starting with 607a9748c4d27b5d86c4155ff9bd0f440176f5804ced8ed9abfc45309fb8ebff not found: ID does not exist"
Oct 31 00:53:57.086192 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:53:57.086158    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-8c2357-655589dfb7-9txvh"]
Oct 31 00:53:57.134144 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:53:57.134109    4284 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-8c2357-655589dfb7-9txvh"]
Oct 31 00:53:57.572005 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:53:57.571972    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:53:57.964635 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:53:57.964600    4284 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d29ce763-9fd5-4bb3-ac9b-c8209da9d59a" path="/var/lib/kubelet/pods/d29ce763-9fd5-4bb3-ac9b-c8209da9d59a/volumes"
Oct 31 00:53:59.529928 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:53:59.529890    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:54:04.302277 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:54:04.302241    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:54:09.075637 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:54:09.075598    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:54:13.877156 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:54:13.877103    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:54:17.028997 ip-10-0-132-173 kubenswrapper[4284]: E1031 00:54:17.028917    4284 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fd8sp" containerName="extractor"
Oct 31 00:54:18.522226 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:54:18.522174    4284 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-e942ab-586494d55c-84g9q"]
Oct 31 00:54:18.522866 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:54:18.522851    4284 cpu_manager.go:414] "containerMap: removing container" podUID="d29ce763-9fd5-4bb3-ac9b-c8209da9d59a" containerName="payload-processing"
Oct 31 00:54:18.522914 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:54:18.522868    4284 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d29ce763-9fd5-4bb3-ac9b-c8209da9d59a" containerName="payload-processing"
Oct 31 00:54:18.522914 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:54:18.522878    4284 cpu_manager.go:414] "containerMap: removing container" podUID="2b942b8c-c3f7-4a18-9dc2-59de27841c78" containerName="payload-processing"
Oct 31 00:54:18.522914 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:54:18.522883    4284 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2b942b8c-c3f7-4a18-9dc2-59de27841c78" containerName="payload-processing"
Oct 31 00:54:18.523037 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:54:18.523024    4284 memory_manager.go:365] "RemoveStaleState removing state" podUID="d29ce763-9fd5-4bb3-ac9b-c8209da9d59a" containerName="payload-processing"
Oct 31 00:54:18.523073 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:54:18.523040    4284 memory_manager.go:365] "RemoveStaleState removing state" podUID="2b942b8c-c3f7-4a18-9dc2-59de27841c78" containerName="payload-processing"
Oct 31 00:54:18.525714 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:54:18.525693    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e942ab-586494d55c-84g9q"
Oct 31 00:54:18.577292 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:54:18.577260    4284 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-e942ab-serving-cert\""
Oct 31 00:54:18.584613 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:54:18.584582    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-e942ab-586494d55c-84g9q"]
Oct 31 00:54:18.585625 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:54:18.585073    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/288ecf4a-e496-46e4-9139-b026bca0157d-metrics-certs\") pod \"maas-api-e2e-api-b-e942ab-586494d55c-84g9q\" (UID: \"288ecf4a-e496-46e4-9139-b026bca0157d\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e942ab-586494d55c-84g9q"
Oct 31 00:54:18.585625 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:54:18.585144    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tjjqk\" (UniqueName: \"kubernetes.io/projected/288ecf4a-e496-46e4-9139-b026bca0157d-kube-api-access-tjjqk\") pod \"maas-api-e2e-api-b-e942ab-586494d55c-84g9q\" (UID: \"288ecf4a-e496-46e4-9139-b026bca0157d\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e942ab-586494d55c-84g9q"
Oct 31 00:54:18.585625 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:54:18.585320    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/288ecf4a-e496-46e4-9139-b026bca0157d-openshift-service-ca\") pod \"maas-api-e2e-api-b-e942ab-586494d55c-84g9q\" (UID: \"288ecf4a-e496-46e4-9139-b026bca0157d\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e942ab-586494d55c-84g9q"
Oct 31 00:54:18.585625 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:54:18.585478    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/288ecf4a-e496-46e4-9139-b026bca0157d-maas-api-tls\") pod \"maas-api-e2e-api-b-e942ab-586494d55c-84g9q\" (UID: \"288ecf4a-e496-46e4-9139-b026bca0157d\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e942ab-586494d55c-84g9q"
Oct 31 00:54:18.627202 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:54:18.627164    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:54:18.694245 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:54:18.692681    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/288ecf4a-e496-46e4-9139-b026bca0157d-maas-api-tls\") pod \"maas-api-e2e-api-b-e942ab-586494d55c-84g9q\" (UID: \"288ecf4a-e496-46e4-9139-b026bca0157d\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e942ab-586494d55c-84g9q"
Oct 31 00:54:18.694245 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:54:18.692776    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/288ecf4a-e496-46e4-9139-b026bca0157d-metrics-certs\") pod \"maas-api-e2e-api-b-e942ab-586494d55c-84g9q\" (UID: \"288ecf4a-e496-46e4-9139-b026bca0157d\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e942ab-586494d55c-84g9q"
Oct 31 00:54:18.694245 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:54:18.692834    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tjjqk\" (UniqueName: \"kubernetes.io/projected/288ecf4a-e496-46e4-9139-b026bca0157d-kube-api-access-tjjqk\") pod \"maas-api-e2e-api-b-e942ab-586494d55c-84g9q\" (UID: \"288ecf4a-e496-46e4-9139-b026bca0157d\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e942ab-586494d55c-84g9q"
Oct 31 00:54:18.694245 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:54:18.693016    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/288ecf4a-e496-46e4-9139-b026bca0157d-openshift-service-ca\") pod \"maas-api-e2e-api-b-e942ab-586494d55c-84g9q\" (UID: \"288ecf4a-e496-46e4-9139-b026bca0157d\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e942ab-586494d55c-84g9q"
Oct 31 00:54:18.694245 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:54:18.694053    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/288ecf4a-e496-46e4-9139-b026bca0157d-openshift-service-ca\") pod \"maas-api-e2e-api-b-e942ab-586494d55c-84g9q\" (UID: \"288ecf4a-e496-46e4-9139-b026bca0157d\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e942ab-586494d55c-84g9q"
Oct 31 00:54:18.694245 ip-10-0-132-173 kubenswrapper[4284]: E1031 00:54:18.694232    4284 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-b-e942ab-serving-cert: secret "maas-api-e2e-api-b-e942ab-serving-cert" not found
Oct 31 00:54:18.694725 ip-10-0-132-173 kubenswrapper[4284]: E1031 00:54:18.694306    4284 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/288ecf4a-e496-46e4-9139-b026bca0157d-maas-api-tls podName:288ecf4a-e496-46e4-9139-b026bca0157d nodeName:}" failed. No retries permitted until 2026-10-31 00:54:19.194287675 +0000 UTC m=+1573.339294572 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/288ecf4a-e496-46e4-9139-b026bca0157d-maas-api-tls") pod "maas-api-e2e-api-b-e942ab-586494d55c-84g9q" (UID: "288ecf4a-e496-46e4-9139-b026bca0157d") : secret "maas-api-e2e-api-b-e942ab-serving-cert" not found
Oct 31 00:54:18.726714 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:54:18.726656    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/288ecf4a-e496-46e4-9139-b026bca0157d-metrics-certs\") pod \"maas-api-e2e-api-b-e942ab-586494d55c-84g9q\" (UID: \"288ecf4a-e496-46e4-9139-b026bca0157d\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e942ab-586494d55c-84g9q"
Oct 31 00:54:18.786286 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:54:18.786151    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tjjqk\" (UniqueName: \"kubernetes.io/projected/288ecf4a-e496-46e4-9139-b026bca0157d-kube-api-access-tjjqk\") pod \"maas-api-e2e-api-b-e942ab-586494d55c-84g9q\" (UID: \"288ecf4a-e496-46e4-9139-b026bca0157d\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e942ab-586494d55c-84g9q"
Oct 31 00:54:18.817761 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:54:18.817724    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:54:18.990340 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:54:18.990290    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:54:19.201770 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:54:19.201733    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/288ecf4a-e496-46e4-9139-b026bca0157d-maas-api-tls\") pod \"maas-api-e2e-api-b-e942ab-586494d55c-84g9q\" (UID: \"288ecf4a-e496-46e4-9139-b026bca0157d\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e942ab-586494d55c-84g9q"
Oct 31 00:54:19.204439 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:54:19.204419    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/288ecf4a-e496-46e4-9139-b026bca0157d-maas-api-tls\") pod \"maas-api-e2e-api-b-e942ab-586494d55c-84g9q\" (UID: \"288ecf4a-e496-46e4-9139-b026bca0157d\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e942ab-586494d55c-84g9q"
Oct 31 00:54:19.439083 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:54:19.439053    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e942ab-586494d55c-84g9q"
Oct 31 00:54:19.810266 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:54:19.810239    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-e942ab-586494d55c-84g9q"]
Oct 31 00:54:19.810684 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:54:19.810336    4284 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod288ecf4a_e496_46e4_9139_b026bca0157d.slice/crio-8f47dd74dae175dcec0bbf1c69601e659ede8b51dd676313cc116b8daa5c6fcf WatchSource:0}: Error finding container 8f47dd74dae175dcec0bbf1c69601e659ede8b51dd676313cc116b8daa5c6fcf: Status 404 returned error can't find the container with id 8f47dd74dae175dcec0bbf1c69601e659ede8b51dd676313cc116b8daa5c6fcf
Oct 31 00:54:20.082171 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:54:20.082093    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e942ab-586494d55c-84g9q" event={"ID":"288ecf4a-e496-46e4-9139-b026bca0157d","Type":"ContainerStarted","Data":"8f47dd74dae175dcec0bbf1c69601e659ede8b51dd676313cc116b8daa5c6fcf"}
Oct 31 00:54:21.089781 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:54:21.089658    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e942ab-586494d55c-84g9q" event={"ID":"288ecf4a-e496-46e4-9139-b026bca0157d","Type":"ContainerStarted","Data":"402bab6943c504c437f23b402d720897fc94c031563898b1961fce1e4fe1cd48"}
Oct 31 00:54:21.090266 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:54:21.089886    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e942ab-586494d55c-84g9q"
Oct 31 00:54:21.205959 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:54:21.205903    4284 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e942ab-586494d55c-84g9q" podStartSLOduration=2.178290622 podStartE2EDuration="3.205888402s" totalImagesPullingTime="1.02759778s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 00:54:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:54:21.203034348 +0000 UTC m=+1575.348041266" watchObservedRunningTime="2026-10-31 00:54:21.205888402 +0000 UTC m=+1575.350895373"
Oct 31 00:54:23.592015 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:54:23.591980    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:54:27.103289 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:54:27.103257    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e942ab-586494d55c-84g9q"
Oct 31 00:54:28.388008 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:54:28.387948    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:54:29.417152 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:54:29.417116    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:54:30.163237 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:54:30.155583    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:54:30.436285 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:54:30.427606    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:54:33.258769 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:54:33.258731    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:54:34.262237 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:54:34.255089    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:54:34.411327 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:54:34.411294    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:54:34.556463 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:54:34.556369    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:54:37.404188 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:54:37.404142    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:54:37.631635 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:54:37.631599    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:54:38.028235 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:54:38.014461    4284 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-ac952a-676dc686ff-lzbkn"]
Oct 31 00:54:38.037242 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:54:38.031253    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-ac952a-676dc686ff-lzbkn"
Oct 31 00:54:38.134832 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:54:38.117099    4284 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-a-ac952a\""
Oct 31 00:54:38.134832 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:54:38.117346    4284 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-a-ac952a-dockercfg-twdj4\""
Oct 31 00:54:38.180169 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:54:38.136202    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nhmw5\" (UniqueName: \"kubernetes.io/projected/5870504f-57b8-4e98-b62b-e67c472d45b7-kube-api-access-nhmw5\") pod \"payload-processing-e2e-ipp-a-ac952a-676dc686ff-lzbkn\" (UID: \"5870504f-57b8-4e98-b62b-e67c472d45b7\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-ac952a-676dc686ff-lzbkn"
Oct 31 00:54:38.180169 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:54:38.136423    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5870504f-57b8-4e98-b62b-e67c472d45b7-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-ac952a-676dc686ff-lzbkn\" (UID: \"5870504f-57b8-4e98-b62b-e67c472d45b7\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-ac952a-676dc686ff-lzbkn"
Oct 31 00:54:38.216239 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:54:38.216124    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:54:38.236257 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:54:38.236185    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-ac952a-676dc686ff-lzbkn"]
Oct 31 00:54:38.238077 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:54:38.238048    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nhmw5\" (UniqueName: \"kubernetes.io/projected/5870504f-57b8-4e98-b62b-e67c472d45b7-kube-api-access-nhmw5\") pod \"payload-processing-e2e-ipp-a-ac952a-676dc686ff-lzbkn\" (UID: \"5870504f-57b8-4e98-b62b-e67c472d45b7\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-ac952a-676dc686ff-lzbkn"
Oct 31 00:54:38.238452 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:54:38.238410    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5870504f-57b8-4e98-b62b-e67c472d45b7-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-ac952a-676dc686ff-lzbkn\" (UID: \"5870504f-57b8-4e98-b62b-e67c472d45b7\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-ac952a-676dc686ff-lzbkn"
Oct 31 00:54:38.239826 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:54:38.239782    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5870504f-57b8-4e98-b62b-e67c472d45b7-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-ac952a-676dc686ff-lzbkn\" (UID: \"5870504f-57b8-4e98-b62b-e67c472d45b7\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-ac952a-676dc686ff-lzbkn"
Oct 31 00:54:38.253780 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:54:38.253748    4284 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-ac952a-66d6cb8df5-hdzd6"]
Oct 31 00:54:38.287397 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:54:38.267863    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-ac952a-66d6cb8df5-hdzd6"]
Oct 31 00:54:38.287397 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:54:38.267983    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-ac952a-66d6cb8df5-hdzd6"
Oct 31 00:54:38.310877 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:54:38.310834    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nhmw5\" (UniqueName: \"kubernetes.io/projected/5870504f-57b8-4e98-b62b-e67c472d45b7-kube-api-access-nhmw5\") pod \"payload-processing-e2e-ipp-a-ac952a-676dc686ff-lzbkn\" (UID: \"5870504f-57b8-4e98-b62b-e67c472d45b7\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-ac952a-676dc686ff-lzbkn"
Oct 31 00:54:38.340571 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:54:38.340523    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wd47t\" (UniqueName: \"kubernetes.io/projected/dc6cbe1a-0138-4aa3-92fa-6b80803d3abb-kube-api-access-wd47t\") pod \"payload-pre-processing-e2e-ipp-a-ac952a-66d6cb8df5-hdzd6\" (UID: \"dc6cbe1a-0138-4aa3-92fa-6b80803d3abb\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-ac952a-66d6cb8df5-hdzd6"
Oct 31 00:54:38.340735 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:54:38.340597    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dc6cbe1a-0138-4aa3-92fa-6b80803d3abb-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-ac952a-66d6cb8df5-hdzd6\" (UID: \"dc6cbe1a-0138-4aa3-92fa-6b80803d3abb\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-ac952a-66d6cb8df5-hdzd6"
Oct 31 00:54:38.442309 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:54:38.441985    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wd47t\" (UniqueName: \"kubernetes.io/projected/dc6cbe1a-0138-4aa3-92fa-6b80803d3abb-kube-api-access-wd47t\") pod \"payload-pre-processing-e2e-ipp-a-ac952a-66d6cb8df5-hdzd6\" (UID: \"dc6cbe1a-0138-4aa3-92fa-6b80803d3abb\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-ac952a-66d6cb8df5-hdzd6"
Oct 31 00:54:38.442309 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:54:38.442056    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dc6cbe1a-0138-4aa3-92fa-6b80803d3abb-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-ac952a-66d6cb8df5-hdzd6\" (UID: \"dc6cbe1a-0138-4aa3-92fa-6b80803d3abb\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-ac952a-66d6cb8df5-hdzd6"
Oct 31 00:54:38.460907 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:54:38.460871    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dc6cbe1a-0138-4aa3-92fa-6b80803d3abb-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-ac952a-66d6cb8df5-hdzd6\" (UID: \"dc6cbe1a-0138-4aa3-92fa-6b80803d3abb\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-ac952a-66d6cb8df5-hdzd6"
Oct 31 00:54:38.519510 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:54:38.519154    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-ac952a-676dc686ff-lzbkn"
Oct 31 00:54:38.522190 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:54:38.522126    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wd47t\" (UniqueName: \"kubernetes.io/projected/dc6cbe1a-0138-4aa3-92fa-6b80803d3abb-kube-api-access-wd47t\") pod \"payload-pre-processing-e2e-ipp-a-ac952a-66d6cb8df5-hdzd6\" (UID: \"dc6cbe1a-0138-4aa3-92fa-6b80803d3abb\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-ac952a-66d6cb8df5-hdzd6"
Oct 31 00:54:38.649240 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:54:38.648550    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-ac952a-66d6cb8df5-hdzd6"
Oct 31 00:54:39.018881 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:54:39.018852    4284 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5870504f_57b8_4e98_b62b_e67c472d45b7.slice/crio-798778a4066cd9fe4982b2daefaa522add737b8e5b6d2e09b1bef7dfce1532ab WatchSource:0}: Error finding container 798778a4066cd9fe4982b2daefaa522add737b8e5b6d2e09b1bef7dfce1532ab: Status 404 returned error can't find the container with id 798778a4066cd9fe4982b2daefaa522add737b8e5b6d2e09b1bef7dfce1532ab
Oct 31 00:54:39.019695 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:54:39.019667    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-ac952a-676dc686ff-lzbkn"]
Oct 31 00:54:39.151245 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:54:39.149839    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-ac952a-66d6cb8df5-hdzd6"]
Oct 31 00:54:39.345685 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:54:39.345647    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-ac952a-66d6cb8df5-hdzd6" event={"ID":"dc6cbe1a-0138-4aa3-92fa-6b80803d3abb","Type":"ContainerStarted","Data":"a177de9cc732dfb9a695895f699cb39ce394fd760100fe643f7b5895fe3fbb1b"}
Oct 31 00:54:39.347543 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:54:39.347519    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-ac952a-676dc686ff-lzbkn" event={"ID":"5870504f-57b8-4e98-b62b-e67c472d45b7","Type":"ContainerStarted","Data":"ce5c26b95967b2813f87287af2f9c6c7d612a5fc9df21785590aa875118708f4"}
Oct 31 00:54:39.347679 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:54:39.347561    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-ac952a-676dc686ff-lzbkn" event={"ID":"5870504f-57b8-4e98-b62b-e67c472d45b7","Type":"ContainerStarted","Data":"798778a4066cd9fe4982b2daefaa522add737b8e5b6d2e09b1bef7dfce1532ab"}
Oct 31 00:54:39.862411 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:54:39.862373    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:54:40.075544 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:54:40.075515    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:54:40.283614 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:54:40.283572    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:54:40.366013 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:54:40.365972    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-ac952a-66d6cb8df5-hdzd6" event={"ID":"dc6cbe1a-0138-4aa3-92fa-6b80803d3abb","Type":"ContainerStarted","Data":"ea123e617c3773f314cd03285d3641bb3e84b442dfb3bf95b562aa0ddcca6062"}
Oct 31 00:54:40.477676 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:54:40.477627    4284 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-a-ac952a-676dc686ff-lzbkn" podStartSLOduration=3.47761552 podStartE2EDuration="3.47761552s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 00:54:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:54:40.477442708 +0000 UTC m=+1594.622449625" watchObservedRunningTime="2026-10-31 00:54:40.47761552 +0000 UTC m=+1594.622622435"
Oct 31 00:54:40.609546 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:54:40.609439    4284 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-ac952a-66d6cb8df5-hdzd6" podStartSLOduration=2.609417867 podStartE2EDuration="2.609417867s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 00:54:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:54:40.60397307 +0000 UTC m=+1594.748979987" watchObservedRunningTime="2026-10-31 00:54:40.609417867 +0000 UTC m=+1594.754424785"
Oct 31 00:54:41.372193 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:54:41.372167    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-ac952a-66d6cb8df5-hdzd6"
Oct 31 00:54:44.532526 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:54:44.532495    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-8c2357-59889f6855-9l5vs"]
Oct 31 00:54:44.532942 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:54:44.532690    4284 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-8c2357-59889f6855-9l5vs" podUID="16527711-37f1-4870-8f17-cc76516411c2" containerName="maas-api" containerID="cri-o://f72af9c2213ba6029f3a08be1cb6c1c210a14deeeac81a196f2e7e21d39bc592" gracePeriod=30
Oct 31 00:54:44.656472 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:54:44.656434    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:54:44.843648 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:54:44.843568    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:54:45.011581 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:54:45.011552    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-8c2357-59889f6855-9l5vs"
Oct 31 00:54:45.098281 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:54:45.093810    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/16527711-37f1-4870-8f17-cc76516411c2-metrics-certs\" (UniqueName: \"kubernetes.io/secret/16527711-37f1-4870-8f17-cc76516411c2-metrics-certs\") pod \"16527711-37f1-4870-8f17-cc76516411c2\" (UID: \"16527711-37f1-4870-8f17-cc76516411c2\") "
Oct 31 00:54:45.098281 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:54:45.094402    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/16527711-37f1-4870-8f17-cc76516411c2-kube-api-access-5pqbv\" (UniqueName: \"kubernetes.io/projected/16527711-37f1-4870-8f17-cc76516411c2-kube-api-access-5pqbv\") pod \"16527711-37f1-4870-8f17-cc76516411c2\" (UID: \"16527711-37f1-4870-8f17-cc76516411c2\") "
Oct 31 00:54:45.098281 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:54:45.094519    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/16527711-37f1-4870-8f17-cc76516411c2-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/16527711-37f1-4870-8f17-cc76516411c2-openshift-service-ca\") pod \"16527711-37f1-4870-8f17-cc76516411c2\" (UID: \"16527711-37f1-4870-8f17-cc76516411c2\") "
Oct 31 00:54:45.098281 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:54:45.094899    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/16527711-37f1-4870-8f17-cc76516411c2-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/16527711-37f1-4870-8f17-cc76516411c2-maas-api-tls\") pod \"16527711-37f1-4870-8f17-cc76516411c2\" (UID: \"16527711-37f1-4870-8f17-cc76516411c2\") "
Oct 31 00:54:45.105034 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:54:45.104975    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/16527711-37f1-4870-8f17-cc76516411c2-maas-api-tls" pod "16527711-37f1-4870-8f17-cc76516411c2" (UID: "16527711-37f1-4870-8f17-cc76516411c2"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 00:54:45.105508 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:54:45.105474    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/16527711-37f1-4870-8f17-cc76516411c2-metrics-certs" pod "16527711-37f1-4870-8f17-cc76516411c2" (UID: "16527711-37f1-4870-8f17-cc76516411c2"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 00:54:45.124765 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:54:45.124077    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/16527711-37f1-4870-8f17-cc76516411c2-openshift-service-ca" pod "16527711-37f1-4870-8f17-cc76516411c2" (UID: "16527711-37f1-4870-8f17-cc76516411c2"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 00:54:45.124765 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:54:45.124594    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/16527711-37f1-4870-8f17-cc76516411c2-kube-api-access-5pqbv" pod "16527711-37f1-4870-8f17-cc76516411c2" (UID: "16527711-37f1-4870-8f17-cc76516411c2"). InnerVolumeSpecName "kube-api-access-5pqbv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 00:54:45.197465 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:54:45.197425    4284 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/16527711-37f1-4870-8f17-cc76516411c2-maas-api-tls\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:54:45.197465 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:54:45.197466    4284 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/16527711-37f1-4870-8f17-cc76516411c2-metrics-certs\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:54:45.197726 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:54:45.197483    4284 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5pqbv\" (UniqueName: \"kubernetes.io/projected/16527711-37f1-4870-8f17-cc76516411c2-kube-api-access-5pqbv\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:54:45.197726 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:54:45.197499    4284 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/16527711-37f1-4870-8f17-cc76516411c2-openshift-service-ca\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:54:45.402494 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:54:45.402412    4284 generic.go:356] "Generic (PLEG): container finished" podID="16527711-37f1-4870-8f17-cc76516411c2" containerID="f72af9c2213ba6029f3a08be1cb6c1c210a14deeeac81a196f2e7e21d39bc592" exitCode=0
Oct 31 00:54:45.402494 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:54:45.402454    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-8c2357-59889f6855-9l5vs"
Oct 31 00:54:45.402494 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:54:45.402486    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-8c2357-59889f6855-9l5vs" event={"ID":"16527711-37f1-4870-8f17-cc76516411c2","Type":"ContainerDied","Data":"f72af9c2213ba6029f3a08be1cb6c1c210a14deeeac81a196f2e7e21d39bc592"}
Oct 31 00:54:45.402751 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:54:45.402542    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-8c2357-59889f6855-9l5vs" event={"ID":"16527711-37f1-4870-8f17-cc76516411c2","Type":"ContainerDied","Data":"f4a03da59a0464352480b5b845c3c2924eaf377a321b940bbb8ad2e229372bb5"}
Oct 31 00:54:45.402751 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:54:45.402565    4284 scope.go:122] "RemoveContainer" containerID="f72af9c2213ba6029f3a08be1cb6c1c210a14deeeac81a196f2e7e21d39bc592"
Oct 31 00:54:45.413525 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:54:45.413487    4284 scope.go:122] "RemoveContainer" containerID="f72af9c2213ba6029f3a08be1cb6c1c210a14deeeac81a196f2e7e21d39bc592"
Oct 31 00:54:45.413790 ip-10-0-132-173 kubenswrapper[4284]: E1031 00:54:45.413772    4284 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f72af9c2213ba6029f3a08be1cb6c1c210a14deeeac81a196f2e7e21d39bc592\": container with ID starting with f72af9c2213ba6029f3a08be1cb6c1c210a14deeeac81a196f2e7e21d39bc592 not found: ID does not exist" containerID="f72af9c2213ba6029f3a08be1cb6c1c210a14deeeac81a196f2e7e21d39bc592"
Oct 31 00:54:45.413835 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:54:45.413800    4284 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f72af9c2213ba6029f3a08be1cb6c1c210a14deeeac81a196f2e7e21d39bc592"} err="failed to get container status \"f72af9c2213ba6029f3a08be1cb6c1c210a14deeeac81a196f2e7e21d39bc592\": rpc error: code = NotFound desc = could not find container \"f72af9c2213ba6029f3a08be1cb6c1c210a14deeeac81a196f2e7e21d39bc592\": container with ID starting with f72af9c2213ba6029f3a08be1cb6c1c210a14deeeac81a196f2e7e21d39bc592 not found: ID does not exist"
Oct 31 00:54:45.801391 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:54:45.801277    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-8c2357-59889f6855-9l5vs"]
Oct 31 00:54:45.805177 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:54:45.805144    4284 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-8c2357-59889f6855-9l5vs"]
Oct 31 00:54:45.964485 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:54:45.964451    4284 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="16527711-37f1-4870-8f17-cc76516411c2" path="/var/lib/kubelet/pods/16527711-37f1-4870-8f17-cc76516411c2/volumes"
Oct 31 00:54:46.270939 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:54:46.270900    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:54:46.485764 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:54:46.485729    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:54:46.658831 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:54:46.654121    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:54:47.171293 ip-10-0-132-173 kubenswrapper[4284]: E1031 00:54:47.151138    4284 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fd8sp" containerName="extractor"
Oct 31 00:54:47.191236 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:54:47.190837    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:54:50.366278 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:54:50.366203    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-ac952a-676dc686ff-lzbkn"
Oct 31 00:54:50.368855 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:54:50.368834    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-ac952a-676dc686ff-lzbkn"
Oct 31 00:54:51.923204 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:54:51.923152    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:54:52.383174 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:54:52.383098    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-ac952a-66d6cb8df5-hdzd6"
Oct 31 00:54:52.821569 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:54:52.821531    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:54:53.016498 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:54:53.016456    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:54:53.646565 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:54:53.646529    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:54:53.794488 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:54:53.794456    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:54:54.007602 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:54:54.007573    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-8c2357-openshift-default-659d454f6f-chhkx"]
Oct 31 00:54:54.007804 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:54:54.007784    4284 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-8c2357-openshift-default-659d454f6f-chhkx" podUID="41ab0dd0-6d29-4caf-b5b8-35255519b562" containerName="istio-proxy" containerID="cri-o://0bc37d0088f14c598aeb9ceb31452f65833336daebedc347e70114cce549ae8d" gracePeriod=30
Oct 31 00:54:54.010293 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:54:54.010206    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:54:55.079016 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:54:55.078974    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:54:55.321655 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:54:55.321568    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:54:56.723778 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:54:56.723740    4284 patch_prober.go:29] interesting pod/e2e-api-a-8c2357-openshift-default-659d454f6f-chhkx container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 31 00:54:56.724175 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:54:56.723804    4284 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-8c2357-openshift-default-659d454f6f-chhkx" podUID="41ab0dd0-6d29-4caf-b5b8-35255519b562" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 31 00:54:57.500206 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:54:57.500160    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:54:57.899345 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:54:57.899256    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:54:59.283057 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:54:59.282996    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-8c2357-openshift-default-659d454f6f-chhkx"
Oct 31 00:54:59.395933 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:54:59.395904    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/41ab0dd0-6d29-4caf-b5b8-35255519b562-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/41ab0dd0-6d29-4caf-b5b8-35255519b562-istio-data\") pod \"41ab0dd0-6d29-4caf-b5b8-35255519b562\" (UID: \"41ab0dd0-6d29-4caf-b5b8-35255519b562\") "
Oct 31 00:54:59.396105 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:54:59.395957    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/41ab0dd0-6d29-4caf-b5b8-35255519b562-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/41ab0dd0-6d29-4caf-b5b8-35255519b562-istio-envoy\") pod \"41ab0dd0-6d29-4caf-b5b8-35255519b562\" (UID: \"41ab0dd0-6d29-4caf-b5b8-35255519b562\") "
Oct 31 00:54:59.396105 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:54:59.396049    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/41ab0dd0-6d29-4caf-b5b8-35255519b562-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/41ab0dd0-6d29-4caf-b5b8-35255519b562-credential-socket\") pod \"41ab0dd0-6d29-4caf-b5b8-35255519b562\" (UID: \"41ab0dd0-6d29-4caf-b5b8-35255519b562\") "
Oct 31 00:54:59.396182 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:54:59.396131    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/41ab0dd0-6d29-4caf-b5b8-35255519b562-istio-token\" (UniqueName: \"kubernetes.io/projected/41ab0dd0-6d29-4caf-b5b8-35255519b562-istio-token\") pod \"41ab0dd0-6d29-4caf-b5b8-35255519b562\" (UID: \"41ab0dd0-6d29-4caf-b5b8-35255519b562\") "
Oct 31 00:54:59.396182 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:54:59.396174    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/41ab0dd0-6d29-4caf-b5b8-35255519b562-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/41ab0dd0-6d29-4caf-b5b8-35255519b562-workload-socket\") pod \"41ab0dd0-6d29-4caf-b5b8-35255519b562\" (UID: \"41ab0dd0-6d29-4caf-b5b8-35255519b562\") "
Oct 31 00:54:59.396322 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:54:59.396273    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/41ab0dd0-6d29-4caf-b5b8-35255519b562-istio-data" pod "41ab0dd0-6d29-4caf-b5b8-35255519b562" (UID: "41ab0dd0-6d29-4caf-b5b8-35255519b562"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:54:59.396322 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:54:59.396290    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/41ab0dd0-6d29-4caf-b5b8-35255519b562-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/41ab0dd0-6d29-4caf-b5b8-35255519b562-workload-certs\") pod \"41ab0dd0-6d29-4caf-b5b8-35255519b562\" (UID: \"41ab0dd0-6d29-4caf-b5b8-35255519b562\") "
Oct 31 00:54:59.396424 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:54:59.396343    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/41ab0dd0-6d29-4caf-b5b8-35255519b562-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/41ab0dd0-6d29-4caf-b5b8-35255519b562-istiod-ca-cert\") pod \"41ab0dd0-6d29-4caf-b5b8-35255519b562\" (UID: \"41ab0dd0-6d29-4caf-b5b8-35255519b562\") "
Oct 31 00:54:59.396478 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:54:59.396422    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/41ab0dd0-6d29-4caf-b5b8-35255519b562-credential-socket" pod "41ab0dd0-6d29-4caf-b5b8-35255519b562" (UID: "41ab0dd0-6d29-4caf-b5b8-35255519b562"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:54:59.396542 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:54:59.396485    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/41ab0dd0-6d29-4caf-b5b8-35255519b562-kube-api-access-p7t4c\" (UniqueName: \"kubernetes.io/projected/41ab0dd0-6d29-4caf-b5b8-35255519b562-kube-api-access-p7t4c\") pod \"41ab0dd0-6d29-4caf-b5b8-35255519b562\" (UID: \"41ab0dd0-6d29-4caf-b5b8-35255519b562\") "
Oct 31 00:54:59.396542 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:54:59.396494    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/41ab0dd0-6d29-4caf-b5b8-35255519b562-workload-socket" pod "41ab0dd0-6d29-4caf-b5b8-35255519b562" (UID: "41ab0dd0-6d29-4caf-b5b8-35255519b562"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:54:59.396542 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:54:59.396510    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/41ab0dd0-6d29-4caf-b5b8-35255519b562-workload-certs" pod "41ab0dd0-6d29-4caf-b5b8-35255519b562" (UID: "41ab0dd0-6d29-4caf-b5b8-35255519b562"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:54:59.396677 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:54:59.396575    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/41ab0dd0-6d29-4caf-b5b8-35255519b562-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/41ab0dd0-6d29-4caf-b5b8-35255519b562-istio-podinfo\") pod \"41ab0dd0-6d29-4caf-b5b8-35255519b562\" (UID: \"41ab0dd0-6d29-4caf-b5b8-35255519b562\") "
Oct 31 00:54:59.396724 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:54:59.396696    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/41ab0dd0-6d29-4caf-b5b8-35255519b562-istiod-ca-cert" pod "41ab0dd0-6d29-4caf-b5b8-35255519b562" (UID: "41ab0dd0-6d29-4caf-b5b8-35255519b562"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 00:54:59.397410 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:54:59.397384    4284 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/41ab0dd0-6d29-4caf-b5b8-35255519b562-istio-data\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:54:59.397536 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:54:59.397415    4284 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/41ab0dd0-6d29-4caf-b5b8-35255519b562-credential-socket\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:54:59.397536 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:54:59.397430    4284 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/41ab0dd0-6d29-4caf-b5b8-35255519b562-workload-socket\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:54:59.397536 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:54:59.397445    4284 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/41ab0dd0-6d29-4caf-b5b8-35255519b562-workload-certs\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:54:59.397536 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:54:59.397460    4284 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/41ab0dd0-6d29-4caf-b5b8-35255519b562-istiod-ca-cert\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:54:59.398608 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:54:59.398583    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/41ab0dd0-6d29-4caf-b5b8-35255519b562-istio-envoy" pod "41ab0dd0-6d29-4caf-b5b8-35255519b562" (UID: "41ab0dd0-6d29-4caf-b5b8-35255519b562"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:54:59.398727 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:54:59.398631    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/41ab0dd0-6d29-4caf-b5b8-35255519b562-istio-token" pod "41ab0dd0-6d29-4caf-b5b8-35255519b562" (UID: "41ab0dd0-6d29-4caf-b5b8-35255519b562"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 00:54:59.398727 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:54:59.398661    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/41ab0dd0-6d29-4caf-b5b8-35255519b562-kube-api-access-p7t4c" pod "41ab0dd0-6d29-4caf-b5b8-35255519b562" (UID: "41ab0dd0-6d29-4caf-b5b8-35255519b562"). InnerVolumeSpecName "kube-api-access-p7t4c". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 00:54:59.398841 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:54:59.398818    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/41ab0dd0-6d29-4caf-b5b8-35255519b562-istio-podinfo" pod "41ab0dd0-6d29-4caf-b5b8-35255519b562" (UID: "41ab0dd0-6d29-4caf-b5b8-35255519b562"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 00:54:59.499123 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:54:59.499087    4284 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p7t4c\" (UniqueName: \"kubernetes.io/projected/41ab0dd0-6d29-4caf-b5b8-35255519b562-kube-api-access-p7t4c\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:54:59.499123 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:54:59.499126    4284 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/41ab0dd0-6d29-4caf-b5b8-35255519b562-istio-podinfo\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:54:59.499377 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:54:59.499140    4284 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/41ab0dd0-6d29-4caf-b5b8-35255519b562-istio-envoy\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:54:59.499377 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:54:59.499154    4284 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/41ab0dd0-6d29-4caf-b5b8-35255519b562-istio-token\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:54:59.744236 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:54:59.740660    4284 generic.go:356] "Generic (PLEG): container finished" podID="41ab0dd0-6d29-4caf-b5b8-35255519b562" containerID="0bc37d0088f14c598aeb9ceb31452f65833336daebedc347e70114cce549ae8d" exitCode=0
Oct 31 00:54:59.744236 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:54:59.740770    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-8c2357-openshift-default-659d454f6f-chhkx" event={"ID":"41ab0dd0-6d29-4caf-b5b8-35255519b562","Type":"ContainerDied","Data":"0bc37d0088f14c598aeb9ceb31452f65833336daebedc347e70114cce549ae8d"}
Oct 31 00:54:59.744236 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:54:59.740819    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-8c2357-openshift-default-659d454f6f-chhkx" event={"ID":"41ab0dd0-6d29-4caf-b5b8-35255519b562","Type":"ContainerDied","Data":"db5e22c369cc43f26cb18ecfdc026469991162f613a7f4408d750568d99faa56"}
Oct 31 00:54:59.744236 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:54:59.740844    4284 scope.go:122] "RemoveContainer" containerID="0bc37d0088f14c598aeb9ceb31452f65833336daebedc347e70114cce549ae8d"
Oct 31 00:54:59.744236 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:54:59.741021    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-8c2357-openshift-default-659d454f6f-chhkx"
Oct 31 00:54:59.846127 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:54:59.846102    4284 scope.go:122] "RemoveContainer" containerID="bbebe8da85ba4299b1c54300acf6aa922c3f4bf7bebc82e484318abcc91c278a"
Oct 31 00:54:59.882402 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:54:59.882371    4284 scope.go:122] "RemoveContainer" containerID="0bc37d0088f14c598aeb9ceb31452f65833336daebedc347e70114cce549ae8d"
Oct 31 00:54:59.892975 ip-10-0-132-173 kubenswrapper[4284]: E1031 00:54:59.892941    4284 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0bc37d0088f14c598aeb9ceb31452f65833336daebedc347e70114cce549ae8d\": container with ID starting with 0bc37d0088f14c598aeb9ceb31452f65833336daebedc347e70114cce549ae8d not found: ID does not exist" containerID="0bc37d0088f14c598aeb9ceb31452f65833336daebedc347e70114cce549ae8d"
Oct 31 00:54:59.893204 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:54:59.893179    4284 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0bc37d0088f14c598aeb9ceb31452f65833336daebedc347e70114cce549ae8d"} err="failed to get container status \"0bc37d0088f14c598aeb9ceb31452f65833336daebedc347e70114cce549ae8d\": rpc error: code = NotFound desc = could not find container \"0bc37d0088f14c598aeb9ceb31452f65833336daebedc347e70114cce549ae8d\": container with ID starting with 0bc37d0088f14c598aeb9ceb31452f65833336daebedc347e70114cce549ae8d not found: ID does not exist"
Oct 31 00:54:59.893379 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:54:59.893366    4284 scope.go:122] "RemoveContainer" containerID="bbebe8da85ba4299b1c54300acf6aa922c3f4bf7bebc82e484318abcc91c278a"
Oct 31 00:54:59.893894 ip-10-0-132-173 kubenswrapper[4284]: E1031 00:54:59.893858    4284 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bbebe8da85ba4299b1c54300acf6aa922c3f4bf7bebc82e484318abcc91c278a\": container with ID starting with bbebe8da85ba4299b1c54300acf6aa922c3f4bf7bebc82e484318abcc91c278a not found: ID does not exist" containerID="bbebe8da85ba4299b1c54300acf6aa922c3f4bf7bebc82e484318abcc91c278a"
Oct 31 00:54:59.894032 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:54:59.894011    4284 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bbebe8da85ba4299b1c54300acf6aa922c3f4bf7bebc82e484318abcc91c278a"} err="failed to get container status \"bbebe8da85ba4299b1c54300acf6aa922c3f4bf7bebc82e484318abcc91c278a\": rpc error: code = NotFound desc = could not find container \"bbebe8da85ba4299b1c54300acf6aa922c3f4bf7bebc82e484318abcc91c278a\": container with ID starting with bbebe8da85ba4299b1c54300acf6aa922c3f4bf7bebc82e484318abcc91c278a not found: ID does not exist"
Oct 31 00:54:59.922589 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:54:59.922552    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-8c2357-openshift-default-659d454f6f-chhkx"]
Oct 31 00:54:59.926341 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:54:59.926314    4284 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-8c2357-openshift-default-659d454f6f-chhkx"]
Oct 31 00:54:59.964037 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:54:59.964001    4284 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="41ab0dd0-6d29-4caf-b5b8-35255519b562" path="/var/lib/kubelet/pods/41ab0dd0-6d29-4caf-b5b8-35255519b562/volumes"
Oct 31 00:54:59.969664 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:54:59.969635    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-knm5g"]
Oct 31 00:54:59.969902 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:54:59.969876    4284 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-knm5g" podUID="a75a6e60-7634-4ae3-ac95-f5cdd33c1af8" containerName="istio-proxy" containerID="cri-o://2470536176c418a1f51ecc2c20cfd05d60004bc21cb16997d01a3a87cfa9c85a" gracePeriod=30
Oct 31 00:54:59.973535 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:54:59.973508    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-6njss"]
Oct 31 00:54:59.973776 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:54:59.973744    4284 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-6njss" podUID="7b453508-9670-49c5-8a23-164a9a59e814" containerName="istio-proxy" containerID="cri-o://31e60692429111a3e0a59c25d6fc1c088ba778b121d3749c1b31e3dcb0dc1017" gracePeriod=30
Oct 31 00:54:59.977292 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:54:59.977264    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-lc4wb"]
Oct 31 00:54:59.977520 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:54:59.977497    4284 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-lc4wb" podUID="cdded00d-879b-491a-aa0b-8736512b0907" containerName="istio-proxy" containerID="cri-o://efc8d05c1157193d3ce97f031f5d0169b18d48d7b4d96cc0c774f418ec241af4" gracePeriod=30
Oct 31 00:54:59.980676 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:54:59.980655    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-4s7kn"]
Oct 31 00:54:59.980851 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:54:59.980831    4284 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-4s7kn" podUID="2cba1971-72c7-4516-9ea2-83e31bc9fc5a" containerName="istio-proxy" containerID="cri-o://15d2b54f9369b27c98788b30d52997a999475b5874b1faad5c8fe2b56b839ccb" gracePeriod=30
Oct 31 00:55:00.328628 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:00.328581    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:55:01.485656 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:01.485622    4284 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-a-f66180-openshift-default-55cfc8577c-lfq7v"]
Oct 31 00:55:01.486333 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:01.486318    4284 cpu_manager.go:414] "containerMap: removing container" podUID="41ab0dd0-6d29-4caf-b5b8-35255519b562" containerName="istio-proxy"
Oct 31 00:55:01.486387 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:01.486334    4284 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="41ab0dd0-6d29-4caf-b5b8-35255519b562" containerName="istio-proxy"
Oct 31 00:55:01.486387 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:01.486371    4284 cpu_manager.go:414] "containerMap: removing container" podUID="41ab0dd0-6d29-4caf-b5b8-35255519b562" containerName="istio-proxy"
Oct 31 00:55:01.486387 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:01.486379    4284 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="41ab0dd0-6d29-4caf-b5b8-35255519b562" containerName="istio-proxy"
Oct 31 00:55:01.486483 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:01.486391    4284 cpu_manager.go:414] "containerMap: removing container" podUID="16527711-37f1-4870-8f17-cc76516411c2" containerName="maas-api"
Oct 31 00:55:01.486483 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:01.486396    4284 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="16527711-37f1-4870-8f17-cc76516411c2" containerName="maas-api"
Oct 31 00:55:01.486546 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:01.486514    4284 memory_manager.go:365] "RemoveStaleState removing state" podUID="41ab0dd0-6d29-4caf-b5b8-35255519b562" containerName="istio-proxy"
Oct 31 00:55:01.486546 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:01.486528    4284 memory_manager.go:365] "RemoveStaleState removing state" podUID="16527711-37f1-4870-8f17-cc76516411c2" containerName="maas-api"
Oct 31 00:55:01.486546 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:01.486539    4284 memory_manager.go:365] "RemoveStaleState removing state" podUID="41ab0dd0-6d29-4caf-b5b8-35255519b562" containerName="istio-proxy"
Oct 31 00:55:01.490255 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:01.490238    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-f66180-openshift-default-55cfc8577c-lfq7v"
Oct 31 00:55:01.501665 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:01.501636    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1e029b42-c66c-4ef0-b6e4-0a5ef5670758-workload-certs\") pod \"e2e-ipp-a-f66180-openshift-default-55cfc8577c-lfq7v\" (UID: \"1e029b42-c66c-4ef0-b6e4-0a5ef5670758\") " pod="openshift-ingress/e2e-ipp-a-f66180-openshift-default-55cfc8577c-lfq7v"
Oct 31 00:55:01.501794 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:01.501692    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1e029b42-c66c-4ef0-b6e4-0a5ef5670758-istio-envoy\") pod \"e2e-ipp-a-f66180-openshift-default-55cfc8577c-lfq7v\" (UID: \"1e029b42-c66c-4ef0-b6e4-0a5ef5670758\") " pod="openshift-ingress/e2e-ipp-a-f66180-openshift-default-55cfc8577c-lfq7v"
Oct 31 00:55:01.501855 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:01.501826    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1e029b42-c66c-4ef0-b6e4-0a5ef5670758-workload-socket\") pod \"e2e-ipp-a-f66180-openshift-default-55cfc8577c-lfq7v\" (UID: \"1e029b42-c66c-4ef0-b6e4-0a5ef5670758\") " pod="openshift-ingress/e2e-ipp-a-f66180-openshift-default-55cfc8577c-lfq7v"
Oct 31 00:55:01.501912 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:01.501897    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8kjss\" (UniqueName: \"kubernetes.io/projected/1e029b42-c66c-4ef0-b6e4-0a5ef5670758-kube-api-access-8kjss\") pod \"e2e-ipp-a-f66180-openshift-default-55cfc8577c-lfq7v\" (UID: \"1e029b42-c66c-4ef0-b6e4-0a5ef5670758\") " pod="openshift-ingress/e2e-ipp-a-f66180-openshift-default-55cfc8577c-lfq7v"
Oct 31 00:55:01.501952 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:01.501933    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1e029b42-c66c-4ef0-b6e4-0a5ef5670758-credential-socket\") pod \"e2e-ipp-a-f66180-openshift-default-55cfc8577c-lfq7v\" (UID: \"1e029b42-c66c-4ef0-b6e4-0a5ef5670758\") " pod="openshift-ingress/e2e-ipp-a-f66180-openshift-default-55cfc8577c-lfq7v"
Oct 31 00:55:01.502114 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:01.502095    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1e029b42-c66c-4ef0-b6e4-0a5ef5670758-istio-token\") pod \"e2e-ipp-a-f66180-openshift-default-55cfc8577c-lfq7v\" (UID: \"1e029b42-c66c-4ef0-b6e4-0a5ef5670758\") " pod="openshift-ingress/e2e-ipp-a-f66180-openshift-default-55cfc8577c-lfq7v"
Oct 31 00:55:01.502175 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:01.502136    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1e029b42-c66c-4ef0-b6e4-0a5ef5670758-istio-data\") pod \"e2e-ipp-a-f66180-openshift-default-55cfc8577c-lfq7v\" (UID: \"1e029b42-c66c-4ef0-b6e4-0a5ef5670758\") " pod="openshift-ingress/e2e-ipp-a-f66180-openshift-default-55cfc8577c-lfq7v"
Oct 31 00:55:01.502321 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:01.502304    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1e029b42-c66c-4ef0-b6e4-0a5ef5670758-istio-podinfo\") pod \"e2e-ipp-a-f66180-openshift-default-55cfc8577c-lfq7v\" (UID: \"1e029b42-c66c-4ef0-b6e4-0a5ef5670758\") " pod="openshift-ingress/e2e-ipp-a-f66180-openshift-default-55cfc8577c-lfq7v"
Oct 31 00:55:01.502405 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:01.502382    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1e029b42-c66c-4ef0-b6e4-0a5ef5670758-istiod-ca-cert\") pod \"e2e-ipp-a-f66180-openshift-default-55cfc8577c-lfq7v\" (UID: \"1e029b42-c66c-4ef0-b6e4-0a5ef5670758\") " pod="openshift-ingress/e2e-ipp-a-f66180-openshift-default-55cfc8577c-lfq7v"
Oct 31 00:55:01.543092 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:01.543062    4284 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-a-f66180-openshift-default-dockercfg-csbht\""
Oct 31 00:55:01.550552 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:01.550516    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-f66180-openshift-default-55cfc8577c-lfq7v"]
Oct 31 00:55:01.592177 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:01.592139    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:55:01.603317 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:01.603275    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1e029b42-c66c-4ef0-b6e4-0a5ef5670758-istio-token\") pod \"e2e-ipp-a-f66180-openshift-default-55cfc8577c-lfq7v\" (UID: \"1e029b42-c66c-4ef0-b6e4-0a5ef5670758\") " pod="openshift-ingress/e2e-ipp-a-f66180-openshift-default-55cfc8577c-lfq7v"
Oct 31 00:55:01.603540 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:01.603339    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1e029b42-c66c-4ef0-b6e4-0a5ef5670758-istio-data\") pod \"e2e-ipp-a-f66180-openshift-default-55cfc8577c-lfq7v\" (UID: \"1e029b42-c66c-4ef0-b6e4-0a5ef5670758\") " pod="openshift-ingress/e2e-ipp-a-f66180-openshift-default-55cfc8577c-lfq7v"
Oct 31 00:55:01.603540 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:01.603412    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1e029b42-c66c-4ef0-b6e4-0a5ef5670758-istio-podinfo\") pod \"e2e-ipp-a-f66180-openshift-default-55cfc8577c-lfq7v\" (UID: \"1e029b42-c66c-4ef0-b6e4-0a5ef5670758\") " pod="openshift-ingress/e2e-ipp-a-f66180-openshift-default-55cfc8577c-lfq7v"
Oct 31 00:55:01.603540 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:01.603436    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1e029b42-c66c-4ef0-b6e4-0a5ef5670758-istiod-ca-cert\") pod \"e2e-ipp-a-f66180-openshift-default-55cfc8577c-lfq7v\" (UID: \"1e029b42-c66c-4ef0-b6e4-0a5ef5670758\") " pod="openshift-ingress/e2e-ipp-a-f66180-openshift-default-55cfc8577c-lfq7v"
Oct 31 00:55:01.603540 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:01.603511    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1e029b42-c66c-4ef0-b6e4-0a5ef5670758-workload-certs\") pod \"e2e-ipp-a-f66180-openshift-default-55cfc8577c-lfq7v\" (UID: \"1e029b42-c66c-4ef0-b6e4-0a5ef5670758\") " pod="openshift-ingress/e2e-ipp-a-f66180-openshift-default-55cfc8577c-lfq7v"
Oct 31 00:55:01.603737 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:01.603546    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1e029b42-c66c-4ef0-b6e4-0a5ef5670758-istio-envoy\") pod \"e2e-ipp-a-f66180-openshift-default-55cfc8577c-lfq7v\" (UID: \"1e029b42-c66c-4ef0-b6e4-0a5ef5670758\") " pod="openshift-ingress/e2e-ipp-a-f66180-openshift-default-55cfc8577c-lfq7v"
Oct 31 00:55:01.603737 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:01.603575    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1e029b42-c66c-4ef0-b6e4-0a5ef5670758-workload-socket\") pod \"e2e-ipp-a-f66180-openshift-default-55cfc8577c-lfq7v\" (UID: \"1e029b42-c66c-4ef0-b6e4-0a5ef5670758\") " pod="openshift-ingress/e2e-ipp-a-f66180-openshift-default-55cfc8577c-lfq7v"
Oct 31 00:55:01.603737 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:01.603611    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8kjss\" (UniqueName: \"kubernetes.io/projected/1e029b42-c66c-4ef0-b6e4-0a5ef5670758-kube-api-access-8kjss\") pod \"e2e-ipp-a-f66180-openshift-default-55cfc8577c-lfq7v\" (UID: \"1e029b42-c66c-4ef0-b6e4-0a5ef5670758\") " pod="openshift-ingress/e2e-ipp-a-f66180-openshift-default-55cfc8577c-lfq7v"
Oct 31 00:55:01.603737 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:01.603644    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1e029b42-c66c-4ef0-b6e4-0a5ef5670758-credential-socket\") pod \"e2e-ipp-a-f66180-openshift-default-55cfc8577c-lfq7v\" (UID: \"1e029b42-c66c-4ef0-b6e4-0a5ef5670758\") " pod="openshift-ingress/e2e-ipp-a-f66180-openshift-default-55cfc8577c-lfq7v"
Oct 31 00:55:01.603737 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:01.603722    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1e029b42-c66c-4ef0-b6e4-0a5ef5670758-istio-data\") pod \"e2e-ipp-a-f66180-openshift-default-55cfc8577c-lfq7v\" (UID: \"1e029b42-c66c-4ef0-b6e4-0a5ef5670758\") " pod="openshift-ingress/e2e-ipp-a-f66180-openshift-default-55cfc8577c-lfq7v"
Oct 31 00:55:01.603987 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:01.603884    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1e029b42-c66c-4ef0-b6e4-0a5ef5670758-workload-socket\") pod \"e2e-ipp-a-f66180-openshift-default-55cfc8577c-lfq7v\" (UID: \"1e029b42-c66c-4ef0-b6e4-0a5ef5670758\") " pod="openshift-ingress/e2e-ipp-a-f66180-openshift-default-55cfc8577c-lfq7v"
Oct 31 00:55:01.604038 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:01.604015    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1e029b42-c66c-4ef0-b6e4-0a5ef5670758-workload-certs\") pod \"e2e-ipp-a-f66180-openshift-default-55cfc8577c-lfq7v\" (UID: \"1e029b42-c66c-4ef0-b6e4-0a5ef5670758\") " pod="openshift-ingress/e2e-ipp-a-f66180-openshift-default-55cfc8577c-lfq7v"
Oct 31 00:55:01.604073 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:01.604044    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1e029b42-c66c-4ef0-b6e4-0a5ef5670758-credential-socket\") pod \"e2e-ipp-a-f66180-openshift-default-55cfc8577c-lfq7v\" (UID: \"1e029b42-c66c-4ef0-b6e4-0a5ef5670758\") " pod="openshift-ingress/e2e-ipp-a-f66180-openshift-default-55cfc8577c-lfq7v"
Oct 31 00:55:01.604228 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:01.604195    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1e029b42-c66c-4ef0-b6e4-0a5ef5670758-istiod-ca-cert\") pod \"e2e-ipp-a-f66180-openshift-default-55cfc8577c-lfq7v\" (UID: \"1e029b42-c66c-4ef0-b6e4-0a5ef5670758\") " pod="openshift-ingress/e2e-ipp-a-f66180-openshift-default-55cfc8577c-lfq7v"
Oct 31 00:55:01.606434 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:01.606412    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1e029b42-c66c-4ef0-b6e4-0a5ef5670758-istio-envoy\") pod \"e2e-ipp-a-f66180-openshift-default-55cfc8577c-lfq7v\" (UID: \"1e029b42-c66c-4ef0-b6e4-0a5ef5670758\") " pod="openshift-ingress/e2e-ipp-a-f66180-openshift-default-55cfc8577c-lfq7v"
Oct 31 00:55:01.606561 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:01.606472    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1e029b42-c66c-4ef0-b6e4-0a5ef5670758-istio-podinfo\") pod \"e2e-ipp-a-f66180-openshift-default-55cfc8577c-lfq7v\" (UID: \"1e029b42-c66c-4ef0-b6e4-0a5ef5670758\") " pod="openshift-ingress/e2e-ipp-a-f66180-openshift-default-55cfc8577c-lfq7v"
Oct 31 00:55:01.703342 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:01.695335    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1e029b42-c66c-4ef0-b6e4-0a5ef5670758-istio-token\") pod \"e2e-ipp-a-f66180-openshift-default-55cfc8577c-lfq7v\" (UID: \"1e029b42-c66c-4ef0-b6e4-0a5ef5670758\") " pod="openshift-ingress/e2e-ipp-a-f66180-openshift-default-55cfc8577c-lfq7v"
Oct 31 00:55:01.719281 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:01.703975    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8kjss\" (UniqueName: \"kubernetes.io/projected/1e029b42-c66c-4ef0-b6e4-0a5ef5670758-kube-api-access-8kjss\") pod \"e2e-ipp-a-f66180-openshift-default-55cfc8577c-lfq7v\" (UID: \"1e029b42-c66c-4ef0-b6e4-0a5ef5670758\") " pod="openshift-ingress/e2e-ipp-a-f66180-openshift-default-55cfc8577c-lfq7v"
Oct 31 00:55:01.804042 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:01.803945    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-f66180-openshift-default-55cfc8577c-lfq7v"
Oct 31 00:55:02.049872 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:02.049832    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:55:02.249153 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:55:02.249124    4284 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1e029b42_c66c_4ef0_b6e4_0a5ef5670758.slice/crio-b6c915f018d7e7a15d3bf5f363484fb004438067e42ea735a38f631fff743612 WatchSource:0}: Error finding container b6c915f018d7e7a15d3bf5f363484fb004438067e42ea735a38f631fff743612: Status 404 returned error can't find the container with id b6c915f018d7e7a15d3bf5f363484fb004438067e42ea735a38f631fff743612
Oct 31 00:55:02.249966 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:02.249929    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-f66180-openshift-default-55cfc8577c-lfq7v"]
Oct 31 00:55:02.252030 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:02.251998    4284 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 00:55:02.252117 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:02.252062    4284 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 00:55:02.252117 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:02.252099    4284 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 00:55:02.709907 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:02.709871    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:55:02.831253 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:02.828504    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-f66180-openshift-default-55cfc8577c-lfq7v" event={"ID":"1e029b42-c66c-4ef0-b6e4-0a5ef5670758","Type":"ContainerStarted","Data":"75bee10e954827824a90ad4ba61a1a6327708e401bb1c1759bff7b703d745e5d"}
Oct 31 00:55:02.831253 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:02.828573    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-f66180-openshift-default-55cfc8577c-lfq7v" event={"ID":"1e029b42-c66c-4ef0-b6e4-0a5ef5670758","Type":"ContainerStarted","Data":"b6c915f018d7e7a15d3bf5f363484fb004438067e42ea735a38f631fff743612"}
Oct 31 00:55:02.948618 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:02.948558    4284 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-a-f66180-openshift-default-55cfc8577c-lfq7v" podStartSLOduration=1.948536643 podStartE2EDuration="1.948536643s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 00:55:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:55:02.945848573 +0000 UTC m=+1617.090855490" watchObservedRunningTime="2026-10-31 00:55:02.948536643 +0000 UTC m=+1617.093543559"
Oct 31 00:55:03.804535 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:03.804496    4284 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-f66180-openshift-default-55cfc8577c-lfq7v"
Oct 31 00:55:03.810152 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:03.810121    4284 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-f66180-openshift-default-55cfc8577c-lfq7v"
Oct 31 00:55:03.836035 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:03.836006    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-f66180-openshift-default-55cfc8577c-lfq7v"
Oct 31 00:55:03.836957 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:03.836937    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-f66180-openshift-default-55cfc8577c-lfq7v"
Oct 31 00:55:05.293101 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:05.293064    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-lc4wb"
Oct 31 00:55:05.364377 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:05.364350    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-6njss"
Oct 31 00:55:05.368489 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:05.368465    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-knm5g"
Oct 31 00:55:05.376649 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:05.376621    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cdded00d-879b-491a-aa0b-8736512b0907-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cdded00d-879b-491a-aa0b-8736512b0907-istio-data\") pod \"cdded00d-879b-491a-aa0b-8736512b0907\" (UID: \"cdded00d-879b-491a-aa0b-8736512b0907\") "
Oct 31 00:55:05.376795 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:05.376756    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-4s7kn"
Oct 31 00:55:05.376795 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:05.376773    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/cdded00d-879b-491a-aa0b-8736512b0907-istio-token\" (UniqueName: \"kubernetes.io/projected/cdded00d-879b-491a-aa0b-8736512b0907-istio-token\") pod \"cdded00d-879b-491a-aa0b-8736512b0907\" (UID: \"cdded00d-879b-491a-aa0b-8736512b0907\") "
Oct 31 00:55:05.376876 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:05.376796    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cdded00d-879b-491a-aa0b-8736512b0907-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cdded00d-879b-491a-aa0b-8736512b0907-workload-socket\") pod \"cdded00d-879b-491a-aa0b-8736512b0907\" (UID: \"cdded00d-879b-491a-aa0b-8736512b0907\") "
Oct 31 00:55:05.376876 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:05.376831    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/cdded00d-879b-491a-aa0b-8736512b0907-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cdded00d-879b-491a-aa0b-8736512b0907-istio-podinfo\") pod \"cdded00d-879b-491a-aa0b-8736512b0907\" (UID: \"cdded00d-879b-491a-aa0b-8736512b0907\") "
Oct 31 00:55:05.376876 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:05.376848    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cdded00d-879b-491a-aa0b-8736512b0907-istio-data" pod "cdded00d-879b-491a-aa0b-8736512b0907" (UID: "cdded00d-879b-491a-aa0b-8736512b0907"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:55:05.377023 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:05.376956    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cdded00d-879b-491a-aa0b-8736512b0907-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cdded00d-879b-491a-aa0b-8736512b0907-workload-certs\") pod \"cdded00d-879b-491a-aa0b-8736512b0907\" (UID: \"cdded00d-879b-491a-aa0b-8736512b0907\") "
Oct 31 00:55:05.377023 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:05.376998    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cdded00d-879b-491a-aa0b-8736512b0907-workload-socket" pod "cdded00d-879b-491a-aa0b-8736512b0907" (UID: "cdded00d-879b-491a-aa0b-8736512b0907"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:55:05.377146 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:05.377130    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/cdded00d-879b-491a-aa0b-8736512b0907-kube-api-access-4r9sf\" (UniqueName: \"kubernetes.io/projected/cdded00d-879b-491a-aa0b-8736512b0907-kube-api-access-4r9sf\") pod \"cdded00d-879b-491a-aa0b-8736512b0907\" (UID: \"cdded00d-879b-491a-aa0b-8736512b0907\") "
Oct 31 00:55:05.377201 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:05.377154    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cdded00d-879b-491a-aa0b-8736512b0907-workload-certs" pod "cdded00d-879b-491a-aa0b-8736512b0907" (UID: "cdded00d-879b-491a-aa0b-8736512b0907"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:55:05.377275 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:05.377252    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cdded00d-879b-491a-aa0b-8736512b0907-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cdded00d-879b-491a-aa0b-8736512b0907-credential-socket\") pod \"cdded00d-879b-491a-aa0b-8736512b0907\" (UID: \"cdded00d-879b-491a-aa0b-8736512b0907\") "
Oct 31 00:55:05.377333 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:05.377300    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/cdded00d-879b-491a-aa0b-8736512b0907-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cdded00d-879b-491a-aa0b-8736512b0907-istiod-ca-cert\") pod \"cdded00d-879b-491a-aa0b-8736512b0907\" (UID: \"cdded00d-879b-491a-aa0b-8736512b0907\") "
Oct 31 00:55:05.377383 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:05.377354    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cdded00d-879b-491a-aa0b-8736512b0907-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cdded00d-879b-491a-aa0b-8736512b0907-istio-envoy\") pod \"cdded00d-879b-491a-aa0b-8736512b0907\" (UID: \"cdded00d-879b-491a-aa0b-8736512b0907\") "
Oct 31 00:55:05.377433 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:05.377399    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cdded00d-879b-491a-aa0b-8736512b0907-credential-socket" pod "cdded00d-879b-491a-aa0b-8736512b0907" (UID: "cdded00d-879b-491a-aa0b-8736512b0907"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:55:05.377679 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:05.377658    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cdded00d-879b-491a-aa0b-8736512b0907-istiod-ca-cert" pod "cdded00d-879b-491a-aa0b-8736512b0907" (UID: "cdded00d-879b-491a-aa0b-8736512b0907"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 00:55:05.377944 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:05.377930    4284 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cdded00d-879b-491a-aa0b-8736512b0907-credential-socket\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:55:05.377994 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:05.377947    4284 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cdded00d-879b-491a-aa0b-8736512b0907-istiod-ca-cert\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:55:05.377994 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:05.377956    4284 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cdded00d-879b-491a-aa0b-8736512b0907-istio-data\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:55:05.377994 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:05.377966    4284 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cdded00d-879b-491a-aa0b-8736512b0907-workload-socket\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:55:05.377994 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:05.377974    4284 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cdded00d-879b-491a-aa0b-8736512b0907-workload-certs\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:55:05.379757 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:05.379737    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/cdded00d-879b-491a-aa0b-8736512b0907-istio-podinfo" pod "cdded00d-879b-491a-aa0b-8736512b0907" (UID: "cdded00d-879b-491a-aa0b-8736512b0907"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 00:55:05.379862 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:05.379820    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cdded00d-879b-491a-aa0b-8736512b0907-istio-token" pod "cdded00d-879b-491a-aa0b-8736512b0907" (UID: "cdded00d-879b-491a-aa0b-8736512b0907"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 00:55:05.379993 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:05.379963    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cdded00d-879b-491a-aa0b-8736512b0907-istio-envoy" pod "cdded00d-879b-491a-aa0b-8736512b0907" (UID: "cdded00d-879b-491a-aa0b-8736512b0907"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:55:05.380096 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:05.379987    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cdded00d-879b-491a-aa0b-8736512b0907-kube-api-access-4r9sf" pod "cdded00d-879b-491a-aa0b-8736512b0907" (UID: "cdded00d-879b-491a-aa0b-8736512b0907"). InnerVolumeSpecName "kube-api-access-4r9sf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 00:55:05.478566 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:05.478524    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7b453508-9670-49c5-8a23-164a9a59e814-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7b453508-9670-49c5-8a23-164a9a59e814-workload-socket\") pod \"7b453508-9670-49c5-8a23-164a9a59e814\" (UID: \"7b453508-9670-49c5-8a23-164a9a59e814\") "
Oct 31 00:55:05.478758 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:05.478580    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7b453508-9670-49c5-8a23-164a9a59e814-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7b453508-9670-49c5-8a23-164a9a59e814-credential-socket\") pod \"7b453508-9670-49c5-8a23-164a9a59e814\" (UID: \"7b453508-9670-49c5-8a23-164a9a59e814\") "
Oct 31 00:55:05.478758 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:05.478616    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/a75a6e60-7634-4ae3-ac95-f5cdd33c1af8-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a75a6e60-7634-4ae3-ac95-f5cdd33c1af8-istio-podinfo\") pod \"a75a6e60-7634-4ae3-ac95-f5cdd33c1af8\" (UID: \"a75a6e60-7634-4ae3-ac95-f5cdd33c1af8\") "
Oct 31 00:55:05.478758 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:05.478647    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7b453508-9670-49c5-8a23-164a9a59e814-kube-api-access-vj94b\" (UniqueName: \"kubernetes.io/projected/7b453508-9670-49c5-8a23-164a9a59e814-kube-api-access-vj94b\") pod \"7b453508-9670-49c5-8a23-164a9a59e814\" (UID: \"7b453508-9670-49c5-8a23-164a9a59e814\") "
Oct 31 00:55:05.478758 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:05.478682    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a75a6e60-7634-4ae3-ac95-f5cdd33c1af8-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a75a6e60-7634-4ae3-ac95-f5cdd33c1af8-workload-certs\") pod \"a75a6e60-7634-4ae3-ac95-f5cdd33c1af8\" (UID: \"a75a6e60-7634-4ae3-ac95-f5cdd33c1af8\") "
Oct 31 00:55:05.478758 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:05.478718    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/7b453508-9670-49c5-8a23-164a9a59e814-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7b453508-9670-49c5-8a23-164a9a59e814-istio-podinfo\") pod \"7b453508-9670-49c5-8a23-164a9a59e814\" (UID: \"7b453508-9670-49c5-8a23-164a9a59e814\") "
Oct 31 00:55:05.479126 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:05.478768    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a75a6e60-7634-4ae3-ac95-f5cdd33c1af8-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a75a6e60-7634-4ae3-ac95-f5cdd33c1af8-istio-envoy\") pod \"a75a6e60-7634-4ae3-ac95-f5cdd33c1af8\" (UID: \"a75a6e60-7634-4ae3-ac95-f5cdd33c1af8\") "
Oct 31 00:55:05.479126 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:05.478818    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a75a6e60-7634-4ae3-ac95-f5cdd33c1af8-istio-token\" (UniqueName: \"kubernetes.io/projected/a75a6e60-7634-4ae3-ac95-f5cdd33c1af8-istio-token\") pod \"a75a6e60-7634-4ae3-ac95-f5cdd33c1af8\" (UID: \"a75a6e60-7634-4ae3-ac95-f5cdd33c1af8\") "
Oct 31 00:55:05.479126 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:05.478827    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7b453508-9670-49c5-8a23-164a9a59e814-workload-socket" pod "7b453508-9670-49c5-8a23-164a9a59e814" (UID: "7b453508-9670-49c5-8a23-164a9a59e814"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:55:05.479126 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:05.478941    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7b453508-9670-49c5-8a23-164a9a59e814-credential-socket" pod "7b453508-9670-49c5-8a23-164a9a59e814" (UID: "7b453508-9670-49c5-8a23-164a9a59e814"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:55:05.479126 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:05.479002    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/7b453508-9670-49c5-8a23-164a9a59e814-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7b453508-9670-49c5-8a23-164a9a59e814-istiod-ca-cert\") pod \"7b453508-9670-49c5-8a23-164a9a59e814\" (UID: \"7b453508-9670-49c5-8a23-164a9a59e814\") "
Oct 31 00:55:05.479126 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:05.479069    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2cba1971-72c7-4516-9ea2-83e31bc9fc5a-istio-token\" (UniqueName: \"kubernetes.io/projected/2cba1971-72c7-4516-9ea2-83e31bc9fc5a-istio-token\") pod \"2cba1971-72c7-4516-9ea2-83e31bc9fc5a\" (UID: \"2cba1971-72c7-4516-9ea2-83e31bc9fc5a\") "
Oct 31 00:55:05.479126 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:05.479106    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7b453508-9670-49c5-8a23-164a9a59e814-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7b453508-9670-49c5-8a23-164a9a59e814-workload-certs\") pod \"7b453508-9670-49c5-8a23-164a9a59e814\" (UID: \"7b453508-9670-49c5-8a23-164a9a59e814\") "
Oct 31 00:55:05.479641 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:05.479183    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a75a6e60-7634-4ae3-ac95-f5cdd33c1af8-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a75a6e60-7634-4ae3-ac95-f5cdd33c1af8-workload-socket\") pod \"a75a6e60-7634-4ae3-ac95-f5cdd33c1af8\" (UID: \"a75a6e60-7634-4ae3-ac95-f5cdd33c1af8\") "
Oct 31 00:55:05.479641 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:05.479250    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2cba1971-72c7-4516-9ea2-83e31bc9fc5a-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2cba1971-72c7-4516-9ea2-83e31bc9fc5a-istio-data\") pod \"2cba1971-72c7-4516-9ea2-83e31bc9fc5a\" (UID: \"2cba1971-72c7-4516-9ea2-83e31bc9fc5a\") "
Oct 31 00:55:05.479641 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:05.479268    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a75a6e60-7634-4ae3-ac95-f5cdd33c1af8-workload-certs" pod "a75a6e60-7634-4ae3-ac95-f5cdd33c1af8" (UID: "a75a6e60-7634-4ae3-ac95-f5cdd33c1af8"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:55:05.479641 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:05.479287    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/2cba1971-72c7-4516-9ea2-83e31bc9fc5a-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2cba1971-72c7-4516-9ea2-83e31bc9fc5a-istiod-ca-cert\") pod \"2cba1971-72c7-4516-9ea2-83e31bc9fc5a\" (UID: \"2cba1971-72c7-4516-9ea2-83e31bc9fc5a\") "
Oct 31 00:55:05.479641 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:05.479329    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2cba1971-72c7-4516-9ea2-83e31bc9fc5a-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2cba1971-72c7-4516-9ea2-83e31bc9fc5a-istio-envoy\") pod \"2cba1971-72c7-4516-9ea2-83e31bc9fc5a\" (UID: \"2cba1971-72c7-4516-9ea2-83e31bc9fc5a\") "
Oct 31 00:55:05.479641 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:05.479361    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2cba1971-72c7-4516-9ea2-83e31bc9fc5a-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2cba1971-72c7-4516-9ea2-83e31bc9fc5a-workload-certs\") pod \"2cba1971-72c7-4516-9ea2-83e31bc9fc5a\" (UID: \"2cba1971-72c7-4516-9ea2-83e31bc9fc5a\") "
Oct 31 00:55:05.479641 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:05.479416    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/2cba1971-72c7-4516-9ea2-83e31bc9fc5a-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2cba1971-72c7-4516-9ea2-83e31bc9fc5a-istio-podinfo\") pod \"2cba1971-72c7-4516-9ea2-83e31bc9fc5a\" (UID: \"2cba1971-72c7-4516-9ea2-83e31bc9fc5a\") "
Oct 31 00:55:05.479641 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:05.479485    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2cba1971-72c7-4516-9ea2-83e31bc9fc5a-kube-api-access-6ptrj\" (UniqueName: \"kubernetes.io/projected/2cba1971-72c7-4516-9ea2-83e31bc9fc5a-kube-api-access-6ptrj\") pod \"2cba1971-72c7-4516-9ea2-83e31bc9fc5a\" (UID: \"2cba1971-72c7-4516-9ea2-83e31bc9fc5a\") "
Oct 31 00:55:05.479641 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:05.479492    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7b453508-9670-49c5-8a23-164a9a59e814-workload-certs" pod "7b453508-9670-49c5-8a23-164a9a59e814" (UID: "7b453508-9670-49c5-8a23-164a9a59e814"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:55:05.479641 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:05.479529    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2cba1971-72c7-4516-9ea2-83e31bc9fc5a-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2cba1971-72c7-4516-9ea2-83e31bc9fc5a-credential-socket\") pod \"2cba1971-72c7-4516-9ea2-83e31bc9fc5a\" (UID: \"2cba1971-72c7-4516-9ea2-83e31bc9fc5a\") "
Oct 31 00:55:05.479641 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:05.479559    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a75a6e60-7634-4ae3-ac95-f5cdd33c1af8-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a75a6e60-7634-4ae3-ac95-f5cdd33c1af8-istio-data\") pod \"a75a6e60-7634-4ae3-ac95-f5cdd33c1af8\" (UID: \"a75a6e60-7634-4ae3-ac95-f5cdd33c1af8\") "
Oct 31 00:55:05.479641 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:05.479617    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7b453508-9670-49c5-8a23-164a9a59e814-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7b453508-9670-49c5-8a23-164a9a59e814-istio-envoy\") pod \"7b453508-9670-49c5-8a23-164a9a59e814\" (UID: \"7b453508-9670-49c5-8a23-164a9a59e814\") "
Oct 31 00:55:05.480293 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:05.479664    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2cba1971-72c7-4516-9ea2-83e31bc9fc5a-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2cba1971-72c7-4516-9ea2-83e31bc9fc5a-workload-socket\") pod \"2cba1971-72c7-4516-9ea2-83e31bc9fc5a\" (UID: \"2cba1971-72c7-4516-9ea2-83e31bc9fc5a\") "
Oct 31 00:55:05.480293 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:05.479733    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a75a6e60-7634-4ae3-ac95-f5cdd33c1af8-kube-api-access-zdm5g\" (UniqueName: \"kubernetes.io/projected/a75a6e60-7634-4ae3-ac95-f5cdd33c1af8-kube-api-access-zdm5g\") pod \"a75a6e60-7634-4ae3-ac95-f5cdd33c1af8\" (UID: \"a75a6e60-7634-4ae3-ac95-f5cdd33c1af8\") "
Oct 31 00:55:05.480293 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:05.479770    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7b453508-9670-49c5-8a23-164a9a59e814-istiod-ca-cert" pod "7b453508-9670-49c5-8a23-164a9a59e814" (UID: "7b453508-9670-49c5-8a23-164a9a59e814"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 00:55:05.480293 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:05.479792    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a75a6e60-7634-4ae3-ac95-f5cdd33c1af8-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a75a6e60-7634-4ae3-ac95-f5cdd33c1af8-credential-socket\") pod \"a75a6e60-7634-4ae3-ac95-f5cdd33c1af8\" (UID: \"a75a6e60-7634-4ae3-ac95-f5cdd33c1af8\") "
Oct 31 00:55:05.480293 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:05.479843    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7b453508-9670-49c5-8a23-164a9a59e814-istio-token\" (UniqueName: \"kubernetes.io/projected/7b453508-9670-49c5-8a23-164a9a59e814-istio-token\") pod \"7b453508-9670-49c5-8a23-164a9a59e814\" (UID: \"7b453508-9670-49c5-8a23-164a9a59e814\") "
Oct 31 00:55:05.480293 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:05.480079    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a75a6e60-7634-4ae3-ac95-f5cdd33c1af8-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a75a6e60-7634-4ae3-ac95-f5cdd33c1af8-istiod-ca-cert\") pod \"a75a6e60-7634-4ae3-ac95-f5cdd33c1af8\" (UID: \"a75a6e60-7634-4ae3-ac95-f5cdd33c1af8\") "
Oct 31 00:55:05.480293 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:05.480131    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7b453508-9670-49c5-8a23-164a9a59e814-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7b453508-9670-49c5-8a23-164a9a59e814-istio-data\") pod \"7b453508-9670-49c5-8a23-164a9a59e814\" (UID: \"7b453508-9670-49c5-8a23-164a9a59e814\") "
Oct 31 00:55:05.480991 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:05.480965    4284 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cdded00d-879b-491a-aa0b-8736512b0907-istio-envoy\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:55:05.481059 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:05.480998    4284 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7b453508-9670-49c5-8a23-164a9a59e814-workload-socket\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:55:05.481059 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:05.481016    4284 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7b453508-9670-49c5-8a23-164a9a59e814-credential-socket\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:55:05.481059 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:05.481030    4284 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a75a6e60-7634-4ae3-ac95-f5cdd33c1af8-workload-certs\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:55:05.481059 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:05.481047    4284 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cdded00d-879b-491a-aa0b-8736512b0907-istio-token\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:55:05.481299 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:05.481062    4284 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cdded00d-879b-491a-aa0b-8736512b0907-istio-podinfo\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:55:05.481299 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:05.481076    4284 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7b453508-9670-49c5-8a23-164a9a59e814-istiod-ca-cert\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:55:05.481299 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:05.481090    4284 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7b453508-9670-49c5-8a23-164a9a59e814-workload-certs\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:55:05.481299 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:05.481105    4284 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4r9sf\" (UniqueName: \"kubernetes.io/projected/cdded00d-879b-491a-aa0b-8736512b0907-kube-api-access-4r9sf\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:55:05.481535 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:05.481462    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7b453508-9670-49c5-8a23-164a9a59e814-istio-data" pod "7b453508-9670-49c5-8a23-164a9a59e814" (UID: "7b453508-9670-49c5-8a23-164a9a59e814"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:55:05.481619 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:05.481596    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2cba1971-72c7-4516-9ea2-83e31bc9fc5a-workload-socket" pod "2cba1971-72c7-4516-9ea2-83e31bc9fc5a" (UID: "2cba1971-72c7-4516-9ea2-83e31bc9fc5a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:55:05.481843 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:05.481821    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a75a6e60-7634-4ae3-ac95-f5cdd33c1af8-istio-data" pod "a75a6e60-7634-4ae3-ac95-f5cdd33c1af8" (UID: "a75a6e60-7634-4ae3-ac95-f5cdd33c1af8"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:55:05.481843 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:05.481831    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2cba1971-72c7-4516-9ea2-83e31bc9fc5a-credential-socket" pod "2cba1971-72c7-4516-9ea2-83e31bc9fc5a" (UID: "2cba1971-72c7-4516-9ea2-83e31bc9fc5a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:55:05.483326 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:05.482810    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a75a6e60-7634-4ae3-ac95-f5cdd33c1af8-workload-socket" pod "a75a6e60-7634-4ae3-ac95-f5cdd33c1af8" (UID: "a75a6e60-7634-4ae3-ac95-f5cdd33c1af8"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:55:05.483326 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:05.482810    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2cba1971-72c7-4516-9ea2-83e31bc9fc5a-workload-certs" pod "2cba1971-72c7-4516-9ea2-83e31bc9fc5a" (UID: "2cba1971-72c7-4516-9ea2-83e31bc9fc5a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:55:05.483326 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:05.482841    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2cba1971-72c7-4516-9ea2-83e31bc9fc5a-istiod-ca-cert" pod "2cba1971-72c7-4516-9ea2-83e31bc9fc5a" (UID: "2cba1971-72c7-4516-9ea2-83e31bc9fc5a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 00:55:05.483326 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:05.483016    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2cba1971-72c7-4516-9ea2-83e31bc9fc5a-istio-data" pod "2cba1971-72c7-4516-9ea2-83e31bc9fc5a" (UID: "2cba1971-72c7-4516-9ea2-83e31bc9fc5a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:55:05.483326 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:05.483031    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a75a6e60-7634-4ae3-ac95-f5cdd33c1af8-credential-socket" pod "a75a6e60-7634-4ae3-ac95-f5cdd33c1af8" (UID: "a75a6e60-7634-4ae3-ac95-f5cdd33c1af8"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:55:05.483326 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:05.483257    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a75a6e60-7634-4ae3-ac95-f5cdd33c1af8-istiod-ca-cert" pod "a75a6e60-7634-4ae3-ac95-f5cdd33c1af8" (UID: "a75a6e60-7634-4ae3-ac95-f5cdd33c1af8"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 00:55:05.485444 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:05.485383    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/2cba1971-72c7-4516-9ea2-83e31bc9fc5a-istio-podinfo" pod "2cba1971-72c7-4516-9ea2-83e31bc9fc5a" (UID: "2cba1971-72c7-4516-9ea2-83e31bc9fc5a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 00:55:05.485666 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:05.485466    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/7b453508-9670-49c5-8a23-164a9a59e814-istio-podinfo" pod "7b453508-9670-49c5-8a23-164a9a59e814" (UID: "7b453508-9670-49c5-8a23-164a9a59e814"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 00:55:05.485666 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:05.485493    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a75a6e60-7634-4ae3-ac95-f5cdd33c1af8-istio-token" pod "a75a6e60-7634-4ae3-ac95-f5cdd33c1af8" (UID: "a75a6e60-7634-4ae3-ac95-f5cdd33c1af8"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 00:55:05.486532 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:05.486508    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7b453508-9670-49c5-8a23-164a9a59e814-istio-token" pod "7b453508-9670-49c5-8a23-164a9a59e814" (UID: "7b453508-9670-49c5-8a23-164a9a59e814"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 00:55:05.487975 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:05.487948    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a75a6e60-7634-4ae3-ac95-f5cdd33c1af8-kube-api-access-zdm5g" pod "a75a6e60-7634-4ae3-ac95-f5cdd33c1af8" (UID: "a75a6e60-7634-4ae3-ac95-f5cdd33c1af8"). InnerVolumeSpecName "kube-api-access-zdm5g". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 00:55:05.488611 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:05.488508    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2cba1971-72c7-4516-9ea2-83e31bc9fc5a-istio-envoy" pod "2cba1971-72c7-4516-9ea2-83e31bc9fc5a" (UID: "2cba1971-72c7-4516-9ea2-83e31bc9fc5a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:55:05.488611 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:05.488525    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7b453508-9670-49c5-8a23-164a9a59e814-istio-envoy" pod "7b453508-9670-49c5-8a23-164a9a59e814" (UID: "7b453508-9670-49c5-8a23-164a9a59e814"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:55:05.488611 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:05.488555    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7b453508-9670-49c5-8a23-164a9a59e814-kube-api-access-vj94b" pod "7b453508-9670-49c5-8a23-164a9a59e814" (UID: "7b453508-9670-49c5-8a23-164a9a59e814"). InnerVolumeSpecName "kube-api-access-vj94b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 00:55:05.488611 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:05.488561    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a75a6e60-7634-4ae3-ac95-f5cdd33c1af8-istio-podinfo" pod "a75a6e60-7634-4ae3-ac95-f5cdd33c1af8" (UID: "a75a6e60-7634-4ae3-ac95-f5cdd33c1af8"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 00:55:05.488611 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:05.488578    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2cba1971-72c7-4516-9ea2-83e31bc9fc5a-istio-token" pod "2cba1971-72c7-4516-9ea2-83e31bc9fc5a" (UID: "2cba1971-72c7-4516-9ea2-83e31bc9fc5a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 00:55:05.489928 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:05.489908    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2cba1971-72c7-4516-9ea2-83e31bc9fc5a-kube-api-access-6ptrj" pod "2cba1971-72c7-4516-9ea2-83e31bc9fc5a" (UID: "2cba1971-72c7-4516-9ea2-83e31bc9fc5a"). InnerVolumeSpecName "kube-api-access-6ptrj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 00:55:05.493578 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:05.493558    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a75a6e60-7634-4ae3-ac95-f5cdd33c1af8-istio-envoy" pod "a75a6e60-7634-4ae3-ac95-f5cdd33c1af8" (UID: "a75a6e60-7634-4ae3-ac95-f5cdd33c1af8"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:55:05.582562 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:05.582485    4284 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a75a6e60-7634-4ae3-ac95-f5cdd33c1af8-istio-podinfo\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:55:05.582562 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:05.582514    4284 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vj94b\" (UniqueName: \"kubernetes.io/projected/7b453508-9670-49c5-8a23-164a9a59e814-kube-api-access-vj94b\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:55:05.582562 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:05.582525    4284 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7b453508-9670-49c5-8a23-164a9a59e814-istio-podinfo\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:55:05.582562 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:05.582535    4284 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a75a6e60-7634-4ae3-ac95-f5cdd33c1af8-istio-envoy\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:55:05.582562 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:05.582544    4284 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a75a6e60-7634-4ae3-ac95-f5cdd33c1af8-istio-token\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:55:05.582562 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:05.582553    4284 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2cba1971-72c7-4516-9ea2-83e31bc9fc5a-istio-token\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:55:05.582562 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:05.582563    4284 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a75a6e60-7634-4ae3-ac95-f5cdd33c1af8-workload-socket\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:55:05.583012 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:05.582571    4284 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2cba1971-72c7-4516-9ea2-83e31bc9fc5a-istio-data\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:55:05.583012 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:05.582612    4284 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2cba1971-72c7-4516-9ea2-83e31bc9fc5a-istiod-ca-cert\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:55:05.583012 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:05.582621    4284 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2cba1971-72c7-4516-9ea2-83e31bc9fc5a-istio-envoy\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:55:05.583012 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:05.582635    4284 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2cba1971-72c7-4516-9ea2-83e31bc9fc5a-workload-certs\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:55:05.583012 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:05.582648    4284 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2cba1971-72c7-4516-9ea2-83e31bc9fc5a-istio-podinfo\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:55:05.583012 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:05.582656    4284 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6ptrj\" (UniqueName: \"kubernetes.io/projected/2cba1971-72c7-4516-9ea2-83e31bc9fc5a-kube-api-access-6ptrj\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:55:05.583012 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:05.582665    4284 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2cba1971-72c7-4516-9ea2-83e31bc9fc5a-credential-socket\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:55:05.583012 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:05.582673    4284 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a75a6e60-7634-4ae3-ac95-f5cdd33c1af8-istio-data\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:55:05.583012 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:05.582681    4284 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7b453508-9670-49c5-8a23-164a9a59e814-istio-envoy\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:55:05.583012 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:05.582689    4284 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2cba1971-72c7-4516-9ea2-83e31bc9fc5a-workload-socket\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:55:05.583012 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:05.582697    4284 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zdm5g\" (UniqueName: \"kubernetes.io/projected/a75a6e60-7634-4ae3-ac95-f5cdd33c1af8-kube-api-access-zdm5g\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:55:05.583012 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:05.582706    4284 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a75a6e60-7634-4ae3-ac95-f5cdd33c1af8-credential-socket\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:55:05.583012 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:05.582722    4284 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7b453508-9670-49c5-8a23-164a9a59e814-istio-token\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:55:05.583012 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:05.582734    4284 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a75a6e60-7634-4ae3-ac95-f5cdd33c1af8-istiod-ca-cert\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:55:05.583012 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:05.582743    4284 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7b453508-9670-49c5-8a23-164a9a59e814-istio-data\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:55:05.843711 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:05.843606    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:55:05.852577 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:05.852546    4284 generic.go:356] "Generic (PLEG): container finished" podID="7b453508-9670-49c5-8a23-164a9a59e814" containerID="31e60692429111a3e0a59c25d6fc1c088ba778b121d3749c1b31e3dcb0dc1017" exitCode=0
Oct 31 00:55:05.852750 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:05.852599    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-6njss"
Oct 31 00:55:05.852750 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:05.852629    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-6njss" event={"ID":"7b453508-9670-49c5-8a23-164a9a59e814","Type":"ContainerDied","Data":"31e60692429111a3e0a59c25d6fc1c088ba778b121d3749c1b31e3dcb0dc1017"}
Oct 31 00:55:05.852750 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:05.852690    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-6njss" event={"ID":"7b453508-9670-49c5-8a23-164a9a59e814","Type":"ContainerDied","Data":"709fee708e8366ca319a3889cdbdddc0ff909d76ffdb1fcc09b4613261d7633a"}
Oct 31 00:55:05.852750 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:05.852723    4284 scope.go:122] "RemoveContainer" containerID="31e60692429111a3e0a59c25d6fc1c088ba778b121d3749c1b31e3dcb0dc1017"
Oct 31 00:55:05.854901 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:05.854874    4284 generic.go:356] "Generic (PLEG): container finished" podID="2cba1971-72c7-4516-9ea2-83e31bc9fc5a" containerID="15d2b54f9369b27c98788b30d52997a999475b5874b1faad5c8fe2b56b839ccb" exitCode=0
Oct 31 00:55:05.854991 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:05.854936    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-4s7kn"
Oct 31 00:55:05.855053 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:05.854985    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-4s7kn" event={"ID":"2cba1971-72c7-4516-9ea2-83e31bc9fc5a","Type":"ContainerDied","Data":"15d2b54f9369b27c98788b30d52997a999475b5874b1faad5c8fe2b56b839ccb"}
Oct 31 00:55:05.855053 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:05.855027    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-4s7kn" event={"ID":"2cba1971-72c7-4516-9ea2-83e31bc9fc5a","Type":"ContainerDied","Data":"178e4456c35b125b26b2323bee8615bc41094646b619cefe313ce831bb5271d0"}
Oct 31 00:55:05.861914 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:05.861882    4284 generic.go:356] "Generic (PLEG): container finished" podID="a75a6e60-7634-4ae3-ac95-f5cdd33c1af8" containerID="2470536176c418a1f51ecc2c20cfd05d60004bc21cb16997d01a3a87cfa9c85a" exitCode=0
Oct 31 00:55:05.862075 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:05.862057    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-knm5g"
Oct 31 00:55:05.863289 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:05.863264    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-knm5g" event={"ID":"a75a6e60-7634-4ae3-ac95-f5cdd33c1af8","Type":"ContainerDied","Data":"2470536176c418a1f51ecc2c20cfd05d60004bc21cb16997d01a3a87cfa9c85a"}
Oct 31 00:55:05.863391 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:05.863334    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-knm5g" event={"ID":"a75a6e60-7634-4ae3-ac95-f5cdd33c1af8","Type":"ContainerDied","Data":"88653e6d71a2ea9159d842eee29c3371d9b07c8fc7d46b87a229e56ccb7d898a"}
Oct 31 00:55:05.865289 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:05.865062    4284 generic.go:356] "Generic (PLEG): container finished" podID="cdded00d-879b-491a-aa0b-8736512b0907" containerID="efc8d05c1157193d3ce97f031f5d0169b18d48d7b4d96cc0c774f418ec241af4" exitCode=0
Oct 31 00:55:05.865289 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:05.865114    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-lc4wb"
Oct 31 00:55:05.865289 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:05.865188    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-lc4wb" event={"ID":"cdded00d-879b-491a-aa0b-8736512b0907","Type":"ContainerDied","Data":"efc8d05c1157193d3ce97f031f5d0169b18d48d7b4d96cc0c774f418ec241af4"}
Oct 31 00:55:05.865289 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:05.865248    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-lc4wb" event={"ID":"cdded00d-879b-491a-aa0b-8736512b0907","Type":"ContainerDied","Data":"40ba7090893d3aab137c1b4c4cff6ae61f6c2c256ee08ac72f0a590d307ab481"}
Oct 31 00:55:05.869551 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:05.869532    4284 scope.go:122] "RemoveContainer" containerID="31e60692429111a3e0a59c25d6fc1c088ba778b121d3749c1b31e3dcb0dc1017"
Oct 31 00:55:05.870811 ip-10-0-132-173 kubenswrapper[4284]: E1031 00:55:05.870786    4284 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"31e60692429111a3e0a59c25d6fc1c088ba778b121d3749c1b31e3dcb0dc1017\": container with ID starting with 31e60692429111a3e0a59c25d6fc1c088ba778b121d3749c1b31e3dcb0dc1017 not found: ID does not exist" containerID="31e60692429111a3e0a59c25d6fc1c088ba778b121d3749c1b31e3dcb0dc1017"
Oct 31 00:55:05.870916 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:05.870822    4284 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"31e60692429111a3e0a59c25d6fc1c088ba778b121d3749c1b31e3dcb0dc1017"} err="failed to get container status \"31e60692429111a3e0a59c25d6fc1c088ba778b121d3749c1b31e3dcb0dc1017\": rpc error: code = NotFound desc = could not find container \"31e60692429111a3e0a59c25d6fc1c088ba778b121d3749c1b31e3dcb0dc1017\": container with ID starting with 31e60692429111a3e0a59c25d6fc1c088ba778b121d3749c1b31e3dcb0dc1017 not found: ID does not exist"
Oct 31 00:55:05.870916 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:05.870849    4284 scope.go:122] "RemoveContainer" containerID="15d2b54f9369b27c98788b30d52997a999475b5874b1faad5c8fe2b56b839ccb"
Oct 31 00:55:05.883907 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:05.883883    4284 scope.go:122] "RemoveContainer" containerID="15d2b54f9369b27c98788b30d52997a999475b5874b1faad5c8fe2b56b839ccb"
Oct 31 00:55:05.884186 ip-10-0-132-173 kubenswrapper[4284]: E1031 00:55:05.884170    4284 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"15d2b54f9369b27c98788b30d52997a999475b5874b1faad5c8fe2b56b839ccb\": container with ID starting with 15d2b54f9369b27c98788b30d52997a999475b5874b1faad5c8fe2b56b839ccb not found: ID does not exist" containerID="15d2b54f9369b27c98788b30d52997a999475b5874b1faad5c8fe2b56b839ccb"
Oct 31 00:55:05.884276 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:05.884196    4284 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"15d2b54f9369b27c98788b30d52997a999475b5874b1faad5c8fe2b56b839ccb"} err="failed to get container status \"15d2b54f9369b27c98788b30d52997a999475b5874b1faad5c8fe2b56b839ccb\": rpc error: code = NotFound desc = could not find container \"15d2b54f9369b27c98788b30d52997a999475b5874b1faad5c8fe2b56b839ccb\": container with ID starting with 15d2b54f9369b27c98788b30d52997a999475b5874b1faad5c8fe2b56b839ccb not found: ID does not exist"
Oct 31 00:55:05.884276 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:05.884239    4284 scope.go:122] "RemoveContainer" containerID="2470536176c418a1f51ecc2c20cfd05d60004bc21cb16997d01a3a87cfa9c85a"
Oct 31 00:55:05.893361 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:05.893340    4284 scope.go:122] "RemoveContainer" containerID="c285f65ea39e06cf577b1d0d137fe831a23be8ad8243cd2b79bb8e8b9a1b7c23"
Oct 31 00:55:05.903023 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:05.902996    4284 scope.go:122] "RemoveContainer" containerID="2470536176c418a1f51ecc2c20cfd05d60004bc21cb16997d01a3a87cfa9c85a"
Oct 31 00:55:05.903377 ip-10-0-132-173 kubenswrapper[4284]: E1031 00:55:05.903355    4284 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2470536176c418a1f51ecc2c20cfd05d60004bc21cb16997d01a3a87cfa9c85a\": container with ID starting with 2470536176c418a1f51ecc2c20cfd05d60004bc21cb16997d01a3a87cfa9c85a not found: ID does not exist" containerID="2470536176c418a1f51ecc2c20cfd05d60004bc21cb16997d01a3a87cfa9c85a"
Oct 31 00:55:05.903464 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:05.903390    4284 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2470536176c418a1f51ecc2c20cfd05d60004bc21cb16997d01a3a87cfa9c85a"} err="failed to get container status \"2470536176c418a1f51ecc2c20cfd05d60004bc21cb16997d01a3a87cfa9c85a\": rpc error: code = NotFound desc = could not find container \"2470536176c418a1f51ecc2c20cfd05d60004bc21cb16997d01a3a87cfa9c85a\": container with ID starting with 2470536176c418a1f51ecc2c20cfd05d60004bc21cb16997d01a3a87cfa9c85a not found: ID does not exist"
Oct 31 00:55:05.903464 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:05.903414    4284 scope.go:122] "RemoveContainer" containerID="c285f65ea39e06cf577b1d0d137fe831a23be8ad8243cd2b79bb8e8b9a1b7c23"
Oct 31 00:55:05.903679 ip-10-0-132-173 kubenswrapper[4284]: E1031 00:55:05.903660    4284 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c285f65ea39e06cf577b1d0d137fe831a23be8ad8243cd2b79bb8e8b9a1b7c23\": container with ID starting with c285f65ea39e06cf577b1d0d137fe831a23be8ad8243cd2b79bb8e8b9a1b7c23 not found: ID does not exist" containerID="c285f65ea39e06cf577b1d0d137fe831a23be8ad8243cd2b79bb8e8b9a1b7c23"
Oct 31 00:55:05.903730 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:05.903686    4284 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c285f65ea39e06cf577b1d0d137fe831a23be8ad8243cd2b79bb8e8b9a1b7c23"} err="failed to get container status \"c285f65ea39e06cf577b1d0d137fe831a23be8ad8243cd2b79bb8e8b9a1b7c23\": rpc error: code = NotFound desc = could not find container \"c285f65ea39e06cf577b1d0d137fe831a23be8ad8243cd2b79bb8e8b9a1b7c23\": container with ID starting with c285f65ea39e06cf577b1d0d137fe831a23be8ad8243cd2b79bb8e8b9a1b7c23 not found: ID does not exist"
Oct 31 00:55:05.903730 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:05.903702    4284 scope.go:122] "RemoveContainer" containerID="efc8d05c1157193d3ce97f031f5d0169b18d48d7b4d96cc0c774f418ec241af4"
Oct 31 00:55:05.915743 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:05.915716    4284 scope.go:122] "RemoveContainer" containerID="efc8d05c1157193d3ce97f031f5d0169b18d48d7b4d96cc0c774f418ec241af4"
Oct 31 00:55:05.916412 ip-10-0-132-173 kubenswrapper[4284]: E1031 00:55:05.916388    4284 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"efc8d05c1157193d3ce97f031f5d0169b18d48d7b4d96cc0c774f418ec241af4\": container with ID starting with efc8d05c1157193d3ce97f031f5d0169b18d48d7b4d96cc0c774f418ec241af4 not found: ID does not exist" containerID="efc8d05c1157193d3ce97f031f5d0169b18d48d7b4d96cc0c774f418ec241af4"
Oct 31 00:55:05.916509 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:05.916427    4284 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"efc8d05c1157193d3ce97f031f5d0169b18d48d7b4d96cc0c774f418ec241af4"} err="failed to get container status \"efc8d05c1157193d3ce97f031f5d0169b18d48d7b4d96cc0c774f418ec241af4\": rpc error: code = NotFound desc = could not find container \"efc8d05c1157193d3ce97f031f5d0169b18d48d7b4d96cc0c774f418ec241af4\": container with ID starting with efc8d05c1157193d3ce97f031f5d0169b18d48d7b4d96cc0c774f418ec241af4 not found: ID does not exist"
Oct 31 00:55:06.016273 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:06.016231    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-lc4wb"]
Oct 31 00:55:06.021921 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:06.021895    4284 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-lc4wb"]
Oct 31 00:55:06.025642 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:06.025612    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:55:06.129068 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:06.128986    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-4s7kn"]
Oct 31 00:55:06.132972 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:06.132947    4284 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-4s7kn"]
Oct 31 00:55:06.182703 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:06.182629    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:55:06.241288 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:06.241259    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-6njss"]
Oct 31 00:55:06.246878 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:06.246852    4284 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-6njss"]
Oct 31 00:55:06.368732 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:06.368699    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-knm5g"]
Oct 31 00:55:06.372078 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:06.372053    4284 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-knm5g"]
Oct 31 00:55:07.924085 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:07.924045    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:55:07.966411 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:07.966379    4284 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2cba1971-72c7-4516-9ea2-83e31bc9fc5a" path="/var/lib/kubelet/pods/2cba1971-72c7-4516-9ea2-83e31bc9fc5a/volumes"
Oct 31 00:55:07.967087 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:07.967070    4284 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="7b453508-9670-49c5-8a23-164a9a59e814" path="/var/lib/kubelet/pods/7b453508-9670-49c5-8a23-164a9a59e814/volumes"
Oct 31 00:55:07.967718 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:07.967699    4284 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a75a6e60-7634-4ae3-ac95-f5cdd33c1af8" path="/var/lib/kubelet/pods/a75a6e60-7634-4ae3-ac95-f5cdd33c1af8/volumes"
Oct 31 00:55:07.968756 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:07.968739    4284 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="cdded00d-879b-491a-aa0b-8736512b0907" path="/var/lib/kubelet/pods/cdded00d-879b-491a-aa0b-8736512b0907/volumes"
Oct 31 00:55:07.979780 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:07.979752    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-3c636f-6948699f5-hhkc2"]
Oct 31 00:55:08.082152 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:08.082112    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-3c636f-5fd8bf89c5-bdr88"]
Oct 31 00:55:08.335311 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:08.333282    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:55:08.479713 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:08.479672    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:55:09.902323 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:09.900942    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-2sg7k"]
Oct 31 00:55:09.902323 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:09.901288    4284 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-2sg7k" podUID="a973c5a8-c25c-41ef-b2d7-7289fdc66b42" containerName="istio-proxy" containerID="cri-o://e9a2438f11440d23d8677436ffa1b3b02d67819e6942a1ea71860848383d9b95" gracePeriod=30
Oct 31 00:55:09.919668 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:09.919629    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-k89gx"]
Oct 31 00:55:09.919924 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:09.919881    4284 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-k89gx" podUID="41a8f8d6-dbbb-48db-a957-5a3bfe8aa418" containerName="istio-proxy" containerID="cri-o://06ab3e269d09bcaeed61b677bdc73a2b6b116cd3b10be0c1c7721ae11d7127d0" gracePeriod=30
Oct 31 00:55:09.922874 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:09.922844    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-522s8"]
Oct 31 00:55:09.923107 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:09.923033    4284 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-522s8" podUID="7cfa573e-ec98-4eaf-b12f-01f723e5e878" containerName="istio-proxy" containerID="cri-o://e115a6c8d0cb6e96dec8c647554181f54149fda01c2429d3ea743bbf1be49c9f" gracePeriod=30
Oct 31 00:55:09.965850 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:09.965816    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-wqns9"]
Oct 31 00:55:09.966056 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:09.966031    4284 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-wqns9" podUID="fc082395-6c5f-4a62-8d0a-1e7d5d562c12" containerName="istio-proxy" containerID="cri-o://57366ca8c190c348c75806d2546bdbc28b788c48c95551a1dde53188378ad7a4" gracePeriod=30
Oct 31 00:55:11.791967 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:11.791925    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:55:12.020236 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:12.017861    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:55:12.182892 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:12.180798    4284 patch_prober.go:29] interesting pod/e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-wqns9 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 31 00:55:12.182892 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:12.180857    4284 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-wqns9" podUID="fc082395-6c5f-4a62-8d0a-1e7d5d562c12" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 31 00:55:12.249330 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:12.249287    4284 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-f66180-97cbb6f6c-t5b4n"]
Oct 31 00:55:12.250390 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:12.250363    4284 cpu_manager.go:414] "containerMap: removing container" podUID="7b453508-9670-49c5-8a23-164a9a59e814" containerName="istio-proxy"
Oct 31 00:55:12.250390 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:12.250388    4284 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7b453508-9670-49c5-8a23-164a9a59e814" containerName="istio-proxy"
Oct 31 00:55:12.250643 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:12.250403    4284 cpu_manager.go:414] "containerMap: removing container" podUID="2cba1971-72c7-4516-9ea2-83e31bc9fc5a" containerName="istio-proxy"
Oct 31 00:55:12.250643 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:12.250412    4284 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2cba1971-72c7-4516-9ea2-83e31bc9fc5a" containerName="istio-proxy"
Oct 31 00:55:12.250643 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:12.250432    4284 cpu_manager.go:414] "containerMap: removing container" podUID="cdded00d-879b-491a-aa0b-8736512b0907" containerName="istio-proxy"
Oct 31 00:55:12.250643 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:12.250440    4284 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="cdded00d-879b-491a-aa0b-8736512b0907" containerName="istio-proxy"
Oct 31 00:55:12.250643 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:12.250452    4284 cpu_manager.go:414] "containerMap: removing container" podUID="a75a6e60-7634-4ae3-ac95-f5cdd33c1af8" containerName="istio-proxy"
Oct 31 00:55:12.250643 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:12.250458    4284 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a75a6e60-7634-4ae3-ac95-f5cdd33c1af8" containerName="istio-proxy"
Oct 31 00:55:12.250643 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:12.250479    4284 cpu_manager.go:414] "containerMap: removing container" podUID="a75a6e60-7634-4ae3-ac95-f5cdd33c1af8" containerName="istio-proxy"
Oct 31 00:55:12.250643 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:12.250486    4284 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a75a6e60-7634-4ae3-ac95-f5cdd33c1af8" containerName="istio-proxy"
Oct 31 00:55:12.251073 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:12.250651    4284 memory_manager.go:365] "RemoveStaleState removing state" podUID="7b453508-9670-49c5-8a23-164a9a59e814" containerName="istio-proxy"
Oct 31 00:55:12.251073 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:12.250664    4284 memory_manager.go:365] "RemoveStaleState removing state" podUID="2cba1971-72c7-4516-9ea2-83e31bc9fc5a" containerName="istio-proxy"
Oct 31 00:55:12.251073 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:12.250676    4284 memory_manager.go:365] "RemoveStaleState removing state" podUID="cdded00d-879b-491a-aa0b-8736512b0907" containerName="istio-proxy"
Oct 31 00:55:12.251073 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:12.250684    4284 memory_manager.go:365] "RemoveStaleState removing state" podUID="a75a6e60-7634-4ae3-ac95-f5cdd33c1af8" containerName="istio-proxy"
Oct 31 00:55:12.251073 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:12.250698    4284 memory_manager.go:365] "RemoveStaleState removing state" podUID="a75a6e60-7634-4ae3-ac95-f5cdd33c1af8" containerName="istio-proxy"
Oct 31 00:55:12.273247 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:12.267290    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-f66180-97cbb6f6c-t5b4n"]
Oct 31 00:55:12.273247 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:12.267455    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-f66180-97cbb6f6c-t5b4n"
Oct 31 00:55:12.344237 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:12.342559    4284 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-a-f66180\""
Oct 31 00:55:12.344237 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:12.342761    4284 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-a-f66180-dockercfg-f47hl\""
Oct 31 00:55:12.420235 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:12.417009    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8hbpk\" (UniqueName: \"kubernetes.io/projected/804be1f8-9d63-4f09-9199-b90409a88db5-kube-api-access-8hbpk\") pod \"payload-pre-processing-e2e-ipp-a-f66180-97cbb6f6c-t5b4n\" (UID: \"804be1f8-9d63-4f09-9199-b90409a88db5\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-f66180-97cbb6f6c-t5b4n"
Oct 31 00:55:12.420235 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:12.417155    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/804be1f8-9d63-4f09-9199-b90409a88db5-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-f66180-97cbb6f6c-t5b4n\" (UID: \"804be1f8-9d63-4f09-9199-b90409a88db5\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-f66180-97cbb6f6c-t5b4n"
Oct 31 00:55:12.468298 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:12.460423    4284 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-f66180-794b84955-ndwz8"]
Oct 31 00:55:12.520636 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:12.520596    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/804be1f8-9d63-4f09-9199-b90409a88db5-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-f66180-97cbb6f6c-t5b4n\" (UID: \"804be1f8-9d63-4f09-9199-b90409a88db5\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-f66180-97cbb6f6c-t5b4n"
Oct 31 00:55:12.521138 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:12.521105    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8hbpk\" (UniqueName: \"kubernetes.io/projected/804be1f8-9d63-4f09-9199-b90409a88db5-kube-api-access-8hbpk\") pod \"payload-pre-processing-e2e-ipp-a-f66180-97cbb6f6c-t5b4n\" (UID: \"804be1f8-9d63-4f09-9199-b90409a88db5\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-f66180-97cbb6f6c-t5b4n"
Oct 31 00:55:12.522625 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:12.522604    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/804be1f8-9d63-4f09-9199-b90409a88db5-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-f66180-97cbb6f6c-t5b4n\" (UID: \"804be1f8-9d63-4f09-9199-b90409a88db5\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-f66180-97cbb6f6c-t5b4n"
Oct 31 00:55:12.537466 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:12.537441    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-f66180-794b84955-ndwz8"]
Oct 31 00:55:12.537875 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:12.537742    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-f66180-794b84955-ndwz8"
Oct 31 00:55:12.580453 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:12.580412    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8hbpk\" (UniqueName: \"kubernetes.io/projected/804be1f8-9d63-4f09-9199-b90409a88db5-kube-api-access-8hbpk\") pod \"payload-pre-processing-e2e-ipp-a-f66180-97cbb6f6c-t5b4n\" (UID: \"804be1f8-9d63-4f09-9199-b90409a88db5\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-f66180-97cbb6f6c-t5b4n"
Oct 31 00:55:12.623108 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:12.623070    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d7d574bb-fe1e-4566-8f50-bcf9dabba619-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-f66180-794b84955-ndwz8\" (UID: \"d7d574bb-fe1e-4566-8f50-bcf9dabba619\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-f66180-794b84955-ndwz8"
Oct 31 00:55:12.623320 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:12.623148    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ptj2c\" (UniqueName: \"kubernetes.io/projected/d7d574bb-fe1e-4566-8f50-bcf9dabba619-kube-api-access-ptj2c\") pod \"payload-processing-e2e-ipp-a-f66180-794b84955-ndwz8\" (UID: \"d7d574bb-fe1e-4566-8f50-bcf9dabba619\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-f66180-794b84955-ndwz8"
Oct 31 00:55:12.679640 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:12.679378    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-f66180-97cbb6f6c-t5b4n"
Oct 31 00:55:12.742538 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:12.742495    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d7d574bb-fe1e-4566-8f50-bcf9dabba619-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-f66180-794b84955-ndwz8\" (UID: \"d7d574bb-fe1e-4566-8f50-bcf9dabba619\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-f66180-794b84955-ndwz8"
Oct 31 00:55:12.742894 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:12.742871    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ptj2c\" (UniqueName: \"kubernetes.io/projected/d7d574bb-fe1e-4566-8f50-bcf9dabba619-kube-api-access-ptj2c\") pod \"payload-processing-e2e-ipp-a-f66180-794b84955-ndwz8\" (UID: \"d7d574bb-fe1e-4566-8f50-bcf9dabba619\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-f66180-794b84955-ndwz8"
Oct 31 00:55:12.744273 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:12.744246    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d7d574bb-fe1e-4566-8f50-bcf9dabba619-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-f66180-794b84955-ndwz8\" (UID: \"d7d574bb-fe1e-4566-8f50-bcf9dabba619\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-f66180-794b84955-ndwz8"
Oct 31 00:55:12.830260 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:12.829896    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ptj2c\" (UniqueName: \"kubernetes.io/projected/d7d574bb-fe1e-4566-8f50-bcf9dabba619-kube-api-access-ptj2c\") pod \"payload-processing-e2e-ipp-a-f66180-794b84955-ndwz8\" (UID: \"d7d574bb-fe1e-4566-8f50-bcf9dabba619\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-f66180-794b84955-ndwz8"
Oct 31 00:55:12.861835 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:12.861518    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-f66180-794b84955-ndwz8"
Oct 31 00:55:12.981261 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:12.980950    4284 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-b-3c636f-6948699f5-hhkc2" podUID="53dde021-74fb-46c6-8340-6a84bc94a627" containerName="payload-processing" containerID="cri-o://cd13b0606e45f7a97b8505fe06fffa3762158887131beccd349447ec44c17ed8" gracePeriod=25
Oct 31 00:55:13.083234 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:13.083169    4284 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-b-3c636f-5fd8bf89c5-bdr88" podUID="044d4725-d84b-4f94-a7d4-451cb16c2d6b" containerName="payload-pre-processing" containerID="cri-o://d52797a238599a5193274b48145d7f2dc3a9221ebad04c23e2bdaf64646e45c5" gracePeriod=25
Oct 31 00:55:13.254461 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:13.254029    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-3c636f-6948699f5-hhkc2"
Oct 31 00:55:13.256422 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:13.256391    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-f66180-97cbb6f6c-t5b4n"]
Oct 31 00:55:13.256604 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:55:13.256577    4284 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod804be1f8_9d63_4f09_9199_b90409a88db5.slice/crio-2bf12c4c5a62f2607d9647fda8a08839a46bfc685d7fe6f74b033c3df1fc30f4 WatchSource:0}: Error finding container 2bf12c4c5a62f2607d9647fda8a08839a46bfc685d7fe6f74b033c3df1fc30f4: Status 404 returned error can't find the container with id 2bf12c4c5a62f2607d9647fda8a08839a46bfc685d7fe6f74b033c3df1fc30f4
Oct 31 00:55:13.307919 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:13.307838    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-f66180-794b84955-ndwz8"]
Oct 31 00:55:13.346455 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:13.346431    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-3c636f-5fd8bf89c5-bdr88"
Oct 31 00:55:13.361380 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:13.361354    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/53dde021-74fb-46c6-8340-6a84bc94a627-kube-api-access-k2h8d\" (UniqueName: \"kubernetes.io/projected/53dde021-74fb-46c6-8340-6a84bc94a627-kube-api-access-k2h8d\") pod \"53dde021-74fb-46c6-8340-6a84bc94a627\" (UID: \"53dde021-74fb-46c6-8340-6a84bc94a627\") "
Oct 31 00:55:13.361551 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:13.361535    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/53dde021-74fb-46c6-8340-6a84bc94a627-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/53dde021-74fb-46c6-8340-6a84bc94a627-plugins-config-volume\") pod \"53dde021-74fb-46c6-8340-6a84bc94a627\" (UID: \"53dde021-74fb-46c6-8340-6a84bc94a627\") "
Oct 31 00:55:13.361963 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:13.361922    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/53dde021-74fb-46c6-8340-6a84bc94a627-plugins-config-volume" pod "53dde021-74fb-46c6-8340-6a84bc94a627" (UID: "53dde021-74fb-46c6-8340-6a84bc94a627"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 00:55:13.362374 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:13.362352    4284 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/53dde021-74fb-46c6-8340-6a84bc94a627-plugins-config-volume\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:55:13.363481 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:13.363453    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/53dde021-74fb-46c6-8340-6a84bc94a627-kube-api-access-k2h8d" pod "53dde021-74fb-46c6-8340-6a84bc94a627" (UID: "53dde021-74fb-46c6-8340-6a84bc94a627"). InnerVolumeSpecName "kube-api-access-k2h8d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 00:55:13.463460 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:13.463425    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/044d4725-d84b-4f94-a7d4-451cb16c2d6b-kube-api-access-5vlhx\" (UniqueName: \"kubernetes.io/projected/044d4725-d84b-4f94-a7d4-451cb16c2d6b-kube-api-access-5vlhx\") pod \"044d4725-d84b-4f94-a7d4-451cb16c2d6b\" (UID: \"044d4725-d84b-4f94-a7d4-451cb16c2d6b\") "
Oct 31 00:55:13.463685 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:13.463665    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/044d4725-d84b-4f94-a7d4-451cb16c2d6b-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/044d4725-d84b-4f94-a7d4-451cb16c2d6b-plugins-config-volume\") pod \"044d4725-d84b-4f94-a7d4-451cb16c2d6b\" (UID: \"044d4725-d84b-4f94-a7d4-451cb16c2d6b\") "
Oct 31 00:55:13.464079 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:13.464057    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/044d4725-d84b-4f94-a7d4-451cb16c2d6b-plugins-config-volume" pod "044d4725-d84b-4f94-a7d4-451cb16c2d6b" (UID: "044d4725-d84b-4f94-a7d4-451cb16c2d6b"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 00:55:13.464344 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:13.464323    4284 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k2h8d\" (UniqueName: \"kubernetes.io/projected/53dde021-74fb-46c6-8340-6a84bc94a627-kube-api-access-k2h8d\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:55:13.464418 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:13.464348    4284 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/044d4725-d84b-4f94-a7d4-451cb16c2d6b-plugins-config-volume\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:55:13.465772 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:13.465752    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/044d4725-d84b-4f94-a7d4-451cb16c2d6b-kube-api-access-5vlhx" pod "044d4725-d84b-4f94-a7d4-451cb16c2d6b" (UID: "044d4725-d84b-4f94-a7d4-451cb16c2d6b"). InnerVolumeSpecName "kube-api-access-5vlhx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 00:55:13.565809 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:13.565729    4284 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5vlhx\" (UniqueName: \"kubernetes.io/projected/044d4725-d84b-4f94-a7d4-451cb16c2d6b-kube-api-access-5vlhx\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:55:13.722646 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:13.722605    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:55:14.027726 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:14.027691    4284 patch_prober.go:29] interesting pod/e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-k89gx container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 31 00:55:14.028178 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:14.027753    4284 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-k89gx" podUID="41a8f8d6-dbbb-48db-a957-5a3bfe8aa418" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 31 00:55:14.054940 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:14.054903    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:55:14.057063 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:14.057039    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-f66180-97cbb6f6c-t5b4n" event={"ID":"804be1f8-9d63-4f09-9199-b90409a88db5","Type":"ContainerStarted","Data":"35df1344a93032601558b8aa73ce8c7b79f20ee0abccc2e959f7e5214564e846"}
Oct 31 00:55:14.057202 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:14.057085    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-f66180-97cbb6f6c-t5b4n" event={"ID":"804be1f8-9d63-4f09-9199-b90409a88db5","Type":"ContainerStarted","Data":"2bf12c4c5a62f2607d9647fda8a08839a46bfc685d7fe6f74b033c3df1fc30f4"}
Oct 31 00:55:14.057292 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:14.057239    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-f66180-97cbb6f6c-t5b4n"
Oct 31 00:55:14.058747 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:14.058724    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-f66180-794b84955-ndwz8" event={"ID":"d7d574bb-fe1e-4566-8f50-bcf9dabba619","Type":"ContainerStarted","Data":"6d92b26bd9ba4dffd110f46c137aa9da449d2e1020f100e29002c345a58dcbac"}
Oct 31 00:55:14.058870 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:14.058772    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-f66180-794b84955-ndwz8" event={"ID":"d7d574bb-fe1e-4566-8f50-bcf9dabba619","Type":"ContainerStarted","Data":"61f497e07eb3338718d44465870542c729987dd96ca716809aaddeb571958a28"}
Oct 31 00:55:14.058870 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:14.058820    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-f66180-794b84955-ndwz8"
Oct 31 00:55:14.060059 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:14.060037    4284 generic.go:356] "Generic (PLEG): container finished" podID="53dde021-74fb-46c6-8340-6a84bc94a627" containerID="cd13b0606e45f7a97b8505fe06fffa3762158887131beccd349447ec44c17ed8" exitCode=0
Oct 31 00:55:14.060158 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:14.060075    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-3c636f-6948699f5-hhkc2"
Oct 31 00:55:14.060158 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:14.060076    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-3c636f-6948699f5-hhkc2" event={"ID":"53dde021-74fb-46c6-8340-6a84bc94a627","Type":"ContainerDied","Data":"cd13b0606e45f7a97b8505fe06fffa3762158887131beccd349447ec44c17ed8"}
Oct 31 00:55:14.060158 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:14.060116    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-3c636f-6948699f5-hhkc2" event={"ID":"53dde021-74fb-46c6-8340-6a84bc94a627","Type":"ContainerDied","Data":"4d9b99596e9fbb3f1abac8e51b271db613f081242026d9eb14ceb4c40cc756c7"}
Oct 31 00:55:14.060158 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:14.060137    4284 scope.go:122] "RemoveContainer" containerID="cd13b0606e45f7a97b8505fe06fffa3762158887131beccd349447ec44c17ed8"
Oct 31 00:55:14.061582 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:14.061565    4284 generic.go:356] "Generic (PLEG): container finished" podID="044d4725-d84b-4f94-a7d4-451cb16c2d6b" containerID="d52797a238599a5193274b48145d7f2dc3a9221ebad04c23e2bdaf64646e45c5" exitCode=0
Oct 31 00:55:14.061655 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:14.061590    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-3c636f-5fd8bf89c5-bdr88" event={"ID":"044d4725-d84b-4f94-a7d4-451cb16c2d6b","Type":"ContainerDied","Data":"d52797a238599a5193274b48145d7f2dc3a9221ebad04c23e2bdaf64646e45c5"}
Oct 31 00:55:14.061655 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:14.061611    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-3c636f-5fd8bf89c5-bdr88"
Oct 31 00:55:14.061655 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:14.061634    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-3c636f-5fd8bf89c5-bdr88" event={"ID":"044d4725-d84b-4f94-a7d4-451cb16c2d6b","Type":"ContainerDied","Data":"ae64bb430f49fc62f3dff9644bc33cfbf9bf5bbed7417eee8f338cab0a6c65ad"}
Oct 31 00:55:14.070918 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:14.070896    4284 scope.go:122] "RemoveContainer" containerID="cd13b0606e45f7a97b8505fe06fffa3762158887131beccd349447ec44c17ed8"
Oct 31 00:55:14.071277 ip-10-0-132-173 kubenswrapper[4284]: E1031 00:55:14.071251    4284 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cd13b0606e45f7a97b8505fe06fffa3762158887131beccd349447ec44c17ed8\": container with ID starting with cd13b0606e45f7a97b8505fe06fffa3762158887131beccd349447ec44c17ed8 not found: ID does not exist" containerID="cd13b0606e45f7a97b8505fe06fffa3762158887131beccd349447ec44c17ed8"
Oct 31 00:55:14.071381 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:14.071287    4284 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cd13b0606e45f7a97b8505fe06fffa3762158887131beccd349447ec44c17ed8"} err="failed to get container status \"cd13b0606e45f7a97b8505fe06fffa3762158887131beccd349447ec44c17ed8\": rpc error: code = NotFound desc = could not find container \"cd13b0606e45f7a97b8505fe06fffa3762158887131beccd349447ec44c17ed8\": container with ID starting with cd13b0606e45f7a97b8505fe06fffa3762158887131beccd349447ec44c17ed8 not found: ID does not exist"
Oct 31 00:55:14.071381 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:14.071310    4284 scope.go:122] "RemoveContainer" containerID="d52797a238599a5193274b48145d7f2dc3a9221ebad04c23e2bdaf64646e45c5"
Oct 31 00:55:14.081874 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:14.081852    4284 scope.go:122] "RemoveContainer" containerID="d52797a238599a5193274b48145d7f2dc3a9221ebad04c23e2bdaf64646e45c5"
Oct 31 00:55:14.082154 ip-10-0-132-173 kubenswrapper[4284]: E1031 00:55:14.082136    4284 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d52797a238599a5193274b48145d7f2dc3a9221ebad04c23e2bdaf64646e45c5\": container with ID starting with d52797a238599a5193274b48145d7f2dc3a9221ebad04c23e2bdaf64646e45c5 not found: ID does not exist" containerID="d52797a238599a5193274b48145d7f2dc3a9221ebad04c23e2bdaf64646e45c5"
Oct 31 00:55:14.082205 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:14.082162    4284 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d52797a238599a5193274b48145d7f2dc3a9221ebad04c23e2bdaf64646e45c5"} err="failed to get container status \"d52797a238599a5193274b48145d7f2dc3a9221ebad04c23e2bdaf64646e45c5\": rpc error: code = NotFound desc = could not find container \"d52797a238599a5193274b48145d7f2dc3a9221ebad04c23e2bdaf64646e45c5\": container with ID starting with d52797a238599a5193274b48145d7f2dc3a9221ebad04c23e2bdaf64646e45c5 not found: ID does not exist"
Oct 31 00:55:14.185259 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:14.185191    4284 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-f66180-97cbb6f6c-t5b4n" podStartSLOduration=2.185177479 podStartE2EDuration="2.185177479s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 00:55:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:55:14.183032813 +0000 UTC m=+1628.328039729" watchObservedRunningTime="2026-10-31 00:55:14.185177479 +0000 UTC m=+1628.330184394"
Oct 31 00:55:14.447235 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:14.444347    4284 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-a-f66180-794b84955-ndwz8" podStartSLOduration=2.43920082 podStartE2EDuration="2.43920082s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 00:55:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:55:14.322395281 +0000 UTC m=+1628.467402220" watchObservedRunningTime="2026-10-31 00:55:14.43920082 +0000 UTC m=+1628.584207739"
Oct 31 00:55:14.448513 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:14.448485    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-3c636f-5fd8bf89c5-bdr88"]
Oct 31 00:55:14.456453 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:14.456420    4284 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-3c636f-5fd8bf89c5-bdr88"]
Oct 31 00:55:14.560747 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:14.560702    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-3c636f-6948699f5-hhkc2"]
Oct 31 00:55:14.563948 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:14.563919    4284 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-3c636f-6948699f5-hhkc2"]
Oct 31 00:55:15.069657 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:15.069617    4284 generic.go:356] "Generic (PLEG): container finished" podID="7cfa573e-ec98-4eaf-b12f-01f723e5e878" containerID="e115a6c8d0cb6e96dec8c647554181f54149fda01c2429d3ea743bbf1be49c9f" exitCode=0
Oct 31 00:55:15.070115 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:15.069703    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-522s8" event={"ID":"7cfa573e-ec98-4eaf-b12f-01f723e5e878","Type":"ContainerDied","Data":"e115a6c8d0cb6e96dec8c647554181f54149fda01c2429d3ea743bbf1be49c9f"}
Oct 31 00:55:15.071844 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:15.071662    4284 generic.go:356] "Generic (PLEG): container finished" podID="41a8f8d6-dbbb-48db-a957-5a3bfe8aa418" containerID="06ab3e269d09bcaeed61b677bdc73a2b6b116cd3b10be0c1c7721ae11d7127d0" exitCode=0
Oct 31 00:55:15.071844 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:15.071733    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-k89gx" event={"ID":"41a8f8d6-dbbb-48db-a957-5a3bfe8aa418","Type":"ContainerDied","Data":"06ab3e269d09bcaeed61b677bdc73a2b6b116cd3b10be0c1c7721ae11d7127d0"}
Oct 31 00:55:15.075410 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:15.075388    4284 generic.go:356] "Generic (PLEG): container finished" podID="a973c5a8-c25c-41ef-b2d7-7289fdc66b42" containerID="e9a2438f11440d23d8677436ffa1b3b02d67819e6942a1ea71860848383d9b95" exitCode=0
Oct 31 00:55:15.075546 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:15.075465    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-2sg7k" event={"ID":"a973c5a8-c25c-41ef-b2d7-7289fdc66b42","Type":"ContainerDied","Data":"e9a2438f11440d23d8677436ffa1b3b02d67819e6942a1ea71860848383d9b95"}
Oct 31 00:55:15.222935 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:15.222847    4284 patch_prober.go:29] interesting pod/e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-2sg7k container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused" start-of-body=
Oct 31 00:55:15.222935 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:15.222920    4284 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-2sg7k" podUID="a973c5a8-c25c-41ef-b2d7-7289fdc66b42" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused"
Oct 31 00:55:15.271968 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:15.271935    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-522s8"
Oct 31 00:55:15.289096 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:15.289066    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-k89gx"
Oct 31 00:55:15.342547 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:15.342517    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-wqns9"
Oct 31 00:55:15.400130 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:15.399995    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/41a8f8d6-dbbb-48db-a957-5a3bfe8aa418-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/41a8f8d6-dbbb-48db-a957-5a3bfe8aa418-workload-certs\") pod \"41a8f8d6-dbbb-48db-a957-5a3bfe8aa418\" (UID: \"41a8f8d6-dbbb-48db-a957-5a3bfe8aa418\") "
Oct 31 00:55:15.400130 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:15.400069    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/7cfa573e-ec98-4eaf-b12f-01f723e5e878-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7cfa573e-ec98-4eaf-b12f-01f723e5e878-istiod-ca-cert\") pod \"7cfa573e-ec98-4eaf-b12f-01f723e5e878\" (UID: \"7cfa573e-ec98-4eaf-b12f-01f723e5e878\") "
Oct 31 00:55:15.400130 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:15.400129    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7cfa573e-ec98-4eaf-b12f-01f723e5e878-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7cfa573e-ec98-4eaf-b12f-01f723e5e878-workload-certs\") pod \"7cfa573e-ec98-4eaf-b12f-01f723e5e878\" (UID: \"7cfa573e-ec98-4eaf-b12f-01f723e5e878\") "
Oct 31 00:55:15.400515 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:15.400159    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/7cfa573e-ec98-4eaf-b12f-01f723e5e878-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7cfa573e-ec98-4eaf-b12f-01f723e5e878-istio-podinfo\") pod \"7cfa573e-ec98-4eaf-b12f-01f723e5e878\" (UID: \"7cfa573e-ec98-4eaf-b12f-01f723e5e878\") "
Oct 31 00:55:15.400515 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:15.400206    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/41a8f8d6-dbbb-48db-a957-5a3bfe8aa418-kube-api-access-mh69w\" (UniqueName: \"kubernetes.io/projected/41a8f8d6-dbbb-48db-a957-5a3bfe8aa418-kube-api-access-mh69w\") pod \"41a8f8d6-dbbb-48db-a957-5a3bfe8aa418\" (UID: \"41a8f8d6-dbbb-48db-a957-5a3bfe8aa418\") "
Oct 31 00:55:15.400515 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:15.400284    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/41a8f8d6-dbbb-48db-a957-5a3bfe8aa418-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/41a8f8d6-dbbb-48db-a957-5a3bfe8aa418-istio-podinfo\") pod \"41a8f8d6-dbbb-48db-a957-5a3bfe8aa418\" (UID: \"41a8f8d6-dbbb-48db-a957-5a3bfe8aa418\") "
Oct 31 00:55:15.400515 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:15.400318    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/41a8f8d6-dbbb-48db-a957-5a3bfe8aa418-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/41a8f8d6-dbbb-48db-a957-5a3bfe8aa418-credential-socket\") pod \"41a8f8d6-dbbb-48db-a957-5a3bfe8aa418\" (UID: \"41a8f8d6-dbbb-48db-a957-5a3bfe8aa418\") "
Oct 31 00:55:15.400515 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:15.400348    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/41a8f8d6-dbbb-48db-a957-5a3bfe8aa418-workload-certs" pod "41a8f8d6-dbbb-48db-a957-5a3bfe8aa418" (UID: "41a8f8d6-dbbb-48db-a957-5a3bfe8aa418"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:55:15.400515 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:15.400387    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7cfa573e-ec98-4eaf-b12f-01f723e5e878-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7cfa573e-ec98-4eaf-b12f-01f723e5e878-credential-socket\") pod \"7cfa573e-ec98-4eaf-b12f-01f723e5e878\" (UID: \"7cfa573e-ec98-4eaf-b12f-01f723e5e878\") "
Oct 31 00:55:15.400515 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:15.400420    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/41a8f8d6-dbbb-48db-a957-5a3bfe8aa418-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/41a8f8d6-dbbb-48db-a957-5a3bfe8aa418-istio-envoy\") pod \"41a8f8d6-dbbb-48db-a957-5a3bfe8aa418\" (UID: \"41a8f8d6-dbbb-48db-a957-5a3bfe8aa418\") "
Oct 31 00:55:15.401673 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:15.400524    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/41a8f8d6-dbbb-48db-a957-5a3bfe8aa418-istio-token\" (UniqueName: \"kubernetes.io/projected/41a8f8d6-dbbb-48db-a957-5a3bfe8aa418-istio-token\") pod \"41a8f8d6-dbbb-48db-a957-5a3bfe8aa418\" (UID: \"41a8f8d6-dbbb-48db-a957-5a3bfe8aa418\") "
Oct 31 00:55:15.401673 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:15.400592    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/41a8f8d6-dbbb-48db-a957-5a3bfe8aa418-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/41a8f8d6-dbbb-48db-a957-5a3bfe8aa418-istio-data\") pod \"41a8f8d6-dbbb-48db-a957-5a3bfe8aa418\" (UID: \"41a8f8d6-dbbb-48db-a957-5a3bfe8aa418\") "
Oct 31 00:55:15.401673 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:15.400617    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7cfa573e-ec98-4eaf-b12f-01f723e5e878-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7cfa573e-ec98-4eaf-b12f-01f723e5e878-istio-envoy\") pod \"7cfa573e-ec98-4eaf-b12f-01f723e5e878\" (UID: \"7cfa573e-ec98-4eaf-b12f-01f723e5e878\") "
Oct 31 00:55:15.401673 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:15.400662    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7cfa573e-ec98-4eaf-b12f-01f723e5e878-kube-api-access-rd98t\" (UniqueName: \"kubernetes.io/projected/7cfa573e-ec98-4eaf-b12f-01f723e5e878-kube-api-access-rd98t\") pod \"7cfa573e-ec98-4eaf-b12f-01f723e5e878\" (UID: \"7cfa573e-ec98-4eaf-b12f-01f723e5e878\") "
Oct 31 00:55:15.401673 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:15.400706    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7cfa573e-ec98-4eaf-b12f-01f723e5e878-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7cfa573e-ec98-4eaf-b12f-01f723e5e878-istio-data\") pod \"7cfa573e-ec98-4eaf-b12f-01f723e5e878\" (UID: \"7cfa573e-ec98-4eaf-b12f-01f723e5e878\") "
Oct 31 00:55:15.401673 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:15.401001    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7cfa573e-ec98-4eaf-b12f-01f723e5e878-istio-data" pod "7cfa573e-ec98-4eaf-b12f-01f723e5e878" (UID: "7cfa573e-ec98-4eaf-b12f-01f723e5e878"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:55:15.402363 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:15.402003    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7cfa573e-ec98-4eaf-b12f-01f723e5e878-istiod-ca-cert" pod "7cfa573e-ec98-4eaf-b12f-01f723e5e878" (UID: "7cfa573e-ec98-4eaf-b12f-01f723e5e878"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 00:55:15.402546 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:15.402501    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7cfa573e-ec98-4eaf-b12f-01f723e5e878-workload-certs" pod "7cfa573e-ec98-4eaf-b12f-01f723e5e878" (UID: "7cfa573e-ec98-4eaf-b12f-01f723e5e878"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:55:15.402716 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:15.402697    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7cfa573e-ec98-4eaf-b12f-01f723e5e878-credential-socket" pod "7cfa573e-ec98-4eaf-b12f-01f723e5e878" (UID: "7cfa573e-ec98-4eaf-b12f-01f723e5e878"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:55:15.402827 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:15.402789    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/41a8f8d6-dbbb-48db-a957-5a3bfe8aa418-credential-socket" pod "41a8f8d6-dbbb-48db-a957-5a3bfe8aa418" (UID: "41a8f8d6-dbbb-48db-a957-5a3bfe8aa418"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:55:15.403250 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:15.403201    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/41a8f8d6-dbbb-48db-a957-5a3bfe8aa418-istio-data" pod "41a8f8d6-dbbb-48db-a957-5a3bfe8aa418" (UID: "41a8f8d6-dbbb-48db-a957-5a3bfe8aa418"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:55:15.403359 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:15.403339    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/41a8f8d6-dbbb-48db-a957-5a3bfe8aa418-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/41a8f8d6-dbbb-48db-a957-5a3bfe8aa418-workload-socket\") pod \"41a8f8d6-dbbb-48db-a957-5a3bfe8aa418\" (UID: \"41a8f8d6-dbbb-48db-a957-5a3bfe8aa418\") "
Oct 31 00:55:15.403418 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:15.403401    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fc082395-6c5f-4a62-8d0a-1e7d5d562c12-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fc082395-6c5f-4a62-8d0a-1e7d5d562c12-workload-socket\") pod \"fc082395-6c5f-4a62-8d0a-1e7d5d562c12\" (UID: \"fc082395-6c5f-4a62-8d0a-1e7d5d562c12\") "
Oct 31 00:55:15.403489 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:15.403440    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/fc082395-6c5f-4a62-8d0a-1e7d5d562c12-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fc082395-6c5f-4a62-8d0a-1e7d5d562c12-istio-podinfo\") pod \"fc082395-6c5f-4a62-8d0a-1e7d5d562c12\" (UID: \"fc082395-6c5f-4a62-8d0a-1e7d5d562c12\") "
Oct 31 00:55:15.403548 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:15.403492    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7cfa573e-ec98-4eaf-b12f-01f723e5e878-istio-token\" (UniqueName: \"kubernetes.io/projected/7cfa573e-ec98-4eaf-b12f-01f723e5e878-istio-token\") pod \"7cfa573e-ec98-4eaf-b12f-01f723e5e878\" (UID: \"7cfa573e-ec98-4eaf-b12f-01f723e5e878\") "
Oct 31 00:55:15.403548 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:15.403528    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7cfa573e-ec98-4eaf-b12f-01f723e5e878-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7cfa573e-ec98-4eaf-b12f-01f723e5e878-workload-socket\") pod \"7cfa573e-ec98-4eaf-b12f-01f723e5e878\" (UID: \"7cfa573e-ec98-4eaf-b12f-01f723e5e878\") "
Oct 31 00:55:15.403654 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:15.403589    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/41a8f8d6-dbbb-48db-a957-5a3bfe8aa418-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/41a8f8d6-dbbb-48db-a957-5a3bfe8aa418-istiod-ca-cert\") pod \"41a8f8d6-dbbb-48db-a957-5a3bfe8aa418\" (UID: \"41a8f8d6-dbbb-48db-a957-5a3bfe8aa418\") "
Oct 31 00:55:15.404693 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:15.403437    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/41a8f8d6-dbbb-48db-a957-5a3bfe8aa418-istio-token" pod "41a8f8d6-dbbb-48db-a957-5a3bfe8aa418" (UID: "41a8f8d6-dbbb-48db-a957-5a3bfe8aa418"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 00:55:15.404693 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:15.403654    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/41a8f8d6-dbbb-48db-a957-5a3bfe8aa418-workload-socket" pod "41a8f8d6-dbbb-48db-a957-5a3bfe8aa418" (UID: "41a8f8d6-dbbb-48db-a957-5a3bfe8aa418"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:55:15.404693 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:15.404422    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fc082395-6c5f-4a62-8d0a-1e7d5d562c12-workload-socket" pod "fc082395-6c5f-4a62-8d0a-1e7d5d562c12" (UID: "fc082395-6c5f-4a62-8d0a-1e7d5d562c12"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:55:15.404693 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:15.404663    4284 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/41a8f8d6-dbbb-48db-a957-5a3bfe8aa418-workload-certs\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:55:15.404693 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:15.404688    4284 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7cfa573e-ec98-4eaf-b12f-01f723e5e878-istiod-ca-cert\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:55:15.405018 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:15.404705    4284 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7cfa573e-ec98-4eaf-b12f-01f723e5e878-workload-certs\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:55:15.405018 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:15.404720    4284 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/41a8f8d6-dbbb-48db-a957-5a3bfe8aa418-credential-socket\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:55:15.405018 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:15.404734    4284 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7cfa573e-ec98-4eaf-b12f-01f723e5e878-credential-socket\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:55:15.405018 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:15.404751    4284 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/41a8f8d6-dbbb-48db-a957-5a3bfe8aa418-istio-token\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:55:15.405018 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:15.404765    4284 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/41a8f8d6-dbbb-48db-a957-5a3bfe8aa418-istio-data\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:55:15.405018 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:15.404779    4284 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7cfa573e-ec98-4eaf-b12f-01f723e5e878-istio-data\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:55:15.405018 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:15.404795    4284 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/41a8f8d6-dbbb-48db-a957-5a3bfe8aa418-workload-socket\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:55:15.405018 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:15.404810    4284 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fc082395-6c5f-4a62-8d0a-1e7d5d562c12-workload-socket\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:55:15.408110 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:15.406643    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7cfa573e-ec98-4eaf-b12f-01f723e5e878-kube-api-access-rd98t" pod "7cfa573e-ec98-4eaf-b12f-01f723e5e878" (UID: "7cfa573e-ec98-4eaf-b12f-01f723e5e878"). InnerVolumeSpecName "kube-api-access-rd98t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 00:55:15.408110 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:15.406899    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7cfa573e-ec98-4eaf-b12f-01f723e5e878-workload-socket" pod "7cfa573e-ec98-4eaf-b12f-01f723e5e878" (UID: "7cfa573e-ec98-4eaf-b12f-01f723e5e878"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:55:15.408110 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:15.407368    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/41a8f8d6-dbbb-48db-a957-5a3bfe8aa418-istiod-ca-cert" pod "41a8f8d6-dbbb-48db-a957-5a3bfe8aa418" (UID: "41a8f8d6-dbbb-48db-a957-5a3bfe8aa418"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 00:55:15.408681 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:15.408539    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7cfa573e-ec98-4eaf-b12f-01f723e5e878-istio-envoy" pod "7cfa573e-ec98-4eaf-b12f-01f723e5e878" (UID: "7cfa573e-ec98-4eaf-b12f-01f723e5e878"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:55:15.408681 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:15.408655    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/7cfa573e-ec98-4eaf-b12f-01f723e5e878-istio-podinfo" pod "7cfa573e-ec98-4eaf-b12f-01f723e5e878" (UID: "7cfa573e-ec98-4eaf-b12f-01f723e5e878"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 00:55:15.408828 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:15.408730    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/41a8f8d6-dbbb-48db-a957-5a3bfe8aa418-kube-api-access-mh69w" pod "41a8f8d6-dbbb-48db-a957-5a3bfe8aa418" (UID: "41a8f8d6-dbbb-48db-a957-5a3bfe8aa418"). InnerVolumeSpecName "kube-api-access-mh69w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 00:55:15.409949 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:15.409924    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/fc082395-6c5f-4a62-8d0a-1e7d5d562c12-istio-podinfo" pod "fc082395-6c5f-4a62-8d0a-1e7d5d562c12" (UID: "fc082395-6c5f-4a62-8d0a-1e7d5d562c12"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 00:55:15.410066 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:15.410045    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7cfa573e-ec98-4eaf-b12f-01f723e5e878-istio-token" pod "7cfa573e-ec98-4eaf-b12f-01f723e5e878" (UID: "7cfa573e-ec98-4eaf-b12f-01f723e5e878"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 00:55:15.410509 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:15.410489    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/41a8f8d6-dbbb-48db-a957-5a3bfe8aa418-istio-envoy" pod "41a8f8d6-dbbb-48db-a957-5a3bfe8aa418" (UID: "41a8f8d6-dbbb-48db-a957-5a3bfe8aa418"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:55:15.410848 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:15.410827    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/41a8f8d6-dbbb-48db-a957-5a3bfe8aa418-istio-podinfo" pod "41a8f8d6-dbbb-48db-a957-5a3bfe8aa418" (UID: "41a8f8d6-dbbb-48db-a957-5a3bfe8aa418"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 00:55:15.507898 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:15.505434    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/fc082395-6c5f-4a62-8d0a-1e7d5d562c12-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fc082395-6c5f-4a62-8d0a-1e7d5d562c12-istiod-ca-cert\") pod \"fc082395-6c5f-4a62-8d0a-1e7d5d562c12\" (UID: \"fc082395-6c5f-4a62-8d0a-1e7d5d562c12\") "
Oct 31 00:55:15.507898 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:15.505497    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fc082395-6c5f-4a62-8d0a-1e7d5d562c12-kube-api-access-7qzjm\" (UniqueName: \"kubernetes.io/projected/fc082395-6c5f-4a62-8d0a-1e7d5d562c12-kube-api-access-7qzjm\") pod \"fc082395-6c5f-4a62-8d0a-1e7d5d562c12\" (UID: \"fc082395-6c5f-4a62-8d0a-1e7d5d562c12\") "
Oct 31 00:55:15.507898 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:15.505658    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fc082395-6c5f-4a62-8d0a-1e7d5d562c12-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fc082395-6c5f-4a62-8d0a-1e7d5d562c12-credential-socket\") pod \"fc082395-6c5f-4a62-8d0a-1e7d5d562c12\" (UID: \"fc082395-6c5f-4a62-8d0a-1e7d5d562c12\") "
Oct 31 00:55:15.507898 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:15.505753    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fc082395-6c5f-4a62-8d0a-1e7d5d562c12-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fc082395-6c5f-4a62-8d0a-1e7d5d562c12-istio-envoy\") pod \"fc082395-6c5f-4a62-8d0a-1e7d5d562c12\" (UID: \"fc082395-6c5f-4a62-8d0a-1e7d5d562c12\") "
Oct 31 00:55:15.507898 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:15.505921    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fc082395-6c5f-4a62-8d0a-1e7d5d562c12-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fc082395-6c5f-4a62-8d0a-1e7d5d562c12-workload-certs\") pod \"fc082395-6c5f-4a62-8d0a-1e7d5d562c12\" (UID: \"fc082395-6c5f-4a62-8d0a-1e7d5d562c12\") "
Oct 31 00:55:15.507898 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:15.505970    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fc082395-6c5f-4a62-8d0a-1e7d5d562c12-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fc082395-6c5f-4a62-8d0a-1e7d5d562c12-istio-data\") pod \"fc082395-6c5f-4a62-8d0a-1e7d5d562c12\" (UID: \"fc082395-6c5f-4a62-8d0a-1e7d5d562c12\") "
Oct 31 00:55:15.507898 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:15.506013    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fc082395-6c5f-4a62-8d0a-1e7d5d562c12-istio-token\" (UniqueName: \"kubernetes.io/projected/fc082395-6c5f-4a62-8d0a-1e7d5d562c12-istio-token\") pod \"fc082395-6c5f-4a62-8d0a-1e7d5d562c12\" (UID: \"fc082395-6c5f-4a62-8d0a-1e7d5d562c12\") "
Oct 31 00:55:15.511317 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:15.510796    4284 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fc082395-6c5f-4a62-8d0a-1e7d5d562c12-istio-podinfo\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:55:15.511317 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:15.510831    4284 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7cfa573e-ec98-4eaf-b12f-01f723e5e878-istio-token\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:55:15.511317 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:15.510845    4284 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7cfa573e-ec98-4eaf-b12f-01f723e5e878-workload-socket\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:55:15.511317 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:15.510858    4284 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/41a8f8d6-dbbb-48db-a957-5a3bfe8aa418-istiod-ca-cert\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:55:15.511317 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:15.510869    4284 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7cfa573e-ec98-4eaf-b12f-01f723e5e878-istio-podinfo\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:55:15.511317 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:15.510882    4284 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mh69w\" (UniqueName: \"kubernetes.io/projected/41a8f8d6-dbbb-48db-a957-5a3bfe8aa418-kube-api-access-mh69w\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:55:15.511317 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:15.510896    4284 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/41a8f8d6-dbbb-48db-a957-5a3bfe8aa418-istio-podinfo\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:55:15.511317 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:15.510909    4284 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/41a8f8d6-dbbb-48db-a957-5a3bfe8aa418-istio-envoy\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:55:15.511317 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:15.510922    4284 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7cfa573e-ec98-4eaf-b12f-01f723e5e878-istio-envoy\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:55:15.511317 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:15.510937    4284 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rd98t\" (UniqueName: \"kubernetes.io/projected/7cfa573e-ec98-4eaf-b12f-01f723e5e878-kube-api-access-rd98t\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:55:15.513288 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:15.513039    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fc082395-6c5f-4a62-8d0a-1e7d5d562c12-istio-data" pod "fc082395-6c5f-4a62-8d0a-1e7d5d562c12" (UID: "fc082395-6c5f-4a62-8d0a-1e7d5d562c12"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:55:15.513288 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:15.513197    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fc082395-6c5f-4a62-8d0a-1e7d5d562c12-istiod-ca-cert" pod "fc082395-6c5f-4a62-8d0a-1e7d5d562c12" (UID: "fc082395-6c5f-4a62-8d0a-1e7d5d562c12"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 00:55:15.513288 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:15.513267    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fc082395-6c5f-4a62-8d0a-1e7d5d562c12-workload-certs" pod "fc082395-6c5f-4a62-8d0a-1e7d5d562c12" (UID: "fc082395-6c5f-4a62-8d0a-1e7d5d562c12"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:55:15.516825 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:15.516785    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fc082395-6c5f-4a62-8d0a-1e7d5d562c12-credential-socket" pod "fc082395-6c5f-4a62-8d0a-1e7d5d562c12" (UID: "fc082395-6c5f-4a62-8d0a-1e7d5d562c12"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:55:15.521550 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:15.521516    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fc082395-6c5f-4a62-8d0a-1e7d5d562c12-istio-token" pod "fc082395-6c5f-4a62-8d0a-1e7d5d562c12" (UID: "fc082395-6c5f-4a62-8d0a-1e7d5d562c12"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 00:55:15.524468 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:15.524438    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fc082395-6c5f-4a62-8d0a-1e7d5d562c12-kube-api-access-7qzjm" pod "fc082395-6c5f-4a62-8d0a-1e7d5d562c12" (UID: "fc082395-6c5f-4a62-8d0a-1e7d5d562c12"). InnerVolumeSpecName "kube-api-access-7qzjm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 00:55:15.524920 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:15.524890    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fc082395-6c5f-4a62-8d0a-1e7d5d562c12-istio-envoy" pod "fc082395-6c5f-4a62-8d0a-1e7d5d562c12" (UID: "fc082395-6c5f-4a62-8d0a-1e7d5d562c12"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:55:15.571091 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:15.571062    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-2sg7k"
Oct 31 00:55:15.612578 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:15.612543    4284 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fc082395-6c5f-4a62-8d0a-1e7d5d562c12-workload-certs\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:55:15.612578 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:15.612573    4284 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fc082395-6c5f-4a62-8d0a-1e7d5d562c12-istio-data\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:55:15.612578 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:15.612582    4284 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fc082395-6c5f-4a62-8d0a-1e7d5d562c12-istio-token\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:55:15.612823 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:15.612592    4284 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fc082395-6c5f-4a62-8d0a-1e7d5d562c12-istiod-ca-cert\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:55:15.612823 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:15.612601    4284 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7qzjm\" (UniqueName: \"kubernetes.io/projected/fc082395-6c5f-4a62-8d0a-1e7d5d562c12-kube-api-access-7qzjm\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:55:15.612823 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:15.612611    4284 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fc082395-6c5f-4a62-8d0a-1e7d5d562c12-credential-socket\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:55:15.612823 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:15.612619    4284 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fc082395-6c5f-4a62-8d0a-1e7d5d562c12-istio-envoy\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:55:15.713313 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:15.713273    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a973c5a8-c25c-41ef-b2d7-7289fdc66b42-istio-token\" (UniqueName: \"kubernetes.io/projected/a973c5a8-c25c-41ef-b2d7-7289fdc66b42-istio-token\") pod \"a973c5a8-c25c-41ef-b2d7-7289fdc66b42\" (UID: \"a973c5a8-c25c-41ef-b2d7-7289fdc66b42\") "
Oct 31 00:55:15.713962 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:15.713398    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a973c5a8-c25c-41ef-b2d7-7289fdc66b42-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a973c5a8-c25c-41ef-b2d7-7289fdc66b42-workload-certs\") pod \"a973c5a8-c25c-41ef-b2d7-7289fdc66b42\" (UID: \"a973c5a8-c25c-41ef-b2d7-7289fdc66b42\") "
Oct 31 00:55:15.713962 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:15.713461    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a973c5a8-c25c-41ef-b2d7-7289fdc66b42-kube-api-access-4z647\" (UniqueName: \"kubernetes.io/projected/a973c5a8-c25c-41ef-b2d7-7289fdc66b42-kube-api-access-4z647\") pod \"a973c5a8-c25c-41ef-b2d7-7289fdc66b42\" (UID: \"a973c5a8-c25c-41ef-b2d7-7289fdc66b42\") "
Oct 31 00:55:15.713962 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:15.713530    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/a973c5a8-c25c-41ef-b2d7-7289fdc66b42-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a973c5a8-c25c-41ef-b2d7-7289fdc66b42-istio-podinfo\") pod \"a973c5a8-c25c-41ef-b2d7-7289fdc66b42\" (UID: \"a973c5a8-c25c-41ef-b2d7-7289fdc66b42\") "
Oct 31 00:55:15.713962 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:15.713555    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a973c5a8-c25c-41ef-b2d7-7289fdc66b42-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a973c5a8-c25c-41ef-b2d7-7289fdc66b42-istio-envoy\") pod \"a973c5a8-c25c-41ef-b2d7-7289fdc66b42\" (UID: \"a973c5a8-c25c-41ef-b2d7-7289fdc66b42\") "
Oct 31 00:55:15.713962 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:15.713608    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a973c5a8-c25c-41ef-b2d7-7289fdc66b42-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a973c5a8-c25c-41ef-b2d7-7289fdc66b42-credential-socket\") pod \"a973c5a8-c25c-41ef-b2d7-7289fdc66b42\" (UID: \"a973c5a8-c25c-41ef-b2d7-7289fdc66b42\") "
Oct 31 00:55:15.713962 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:15.713660    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a973c5a8-c25c-41ef-b2d7-7289fdc66b42-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a973c5a8-c25c-41ef-b2d7-7289fdc66b42-workload-socket\") pod \"a973c5a8-c25c-41ef-b2d7-7289fdc66b42\" (UID: \"a973c5a8-c25c-41ef-b2d7-7289fdc66b42\") "
Oct 31 00:55:15.713962 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:15.713734    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a973c5a8-c25c-41ef-b2d7-7289fdc66b42-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a973c5a8-c25c-41ef-b2d7-7289fdc66b42-istio-data\") pod \"a973c5a8-c25c-41ef-b2d7-7289fdc66b42\" (UID: \"a973c5a8-c25c-41ef-b2d7-7289fdc66b42\") "
Oct 31 00:55:15.713962 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:15.713752    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a973c5a8-c25c-41ef-b2d7-7289fdc66b42-workload-certs" pod "a973c5a8-c25c-41ef-b2d7-7289fdc66b42" (UID: "a973c5a8-c25c-41ef-b2d7-7289fdc66b42"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:55:15.713962 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:15.713812    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a973c5a8-c25c-41ef-b2d7-7289fdc66b42-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a973c5a8-c25c-41ef-b2d7-7289fdc66b42-istiod-ca-cert\") pod \"a973c5a8-c25c-41ef-b2d7-7289fdc66b42\" (UID: \"a973c5a8-c25c-41ef-b2d7-7289fdc66b42\") "
Oct 31 00:55:15.714546 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:15.713977    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a973c5a8-c25c-41ef-b2d7-7289fdc66b42-credential-socket" pod "a973c5a8-c25c-41ef-b2d7-7289fdc66b42" (UID: "a973c5a8-c25c-41ef-b2d7-7289fdc66b42"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:55:15.714546 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:15.713995    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a973c5a8-c25c-41ef-b2d7-7289fdc66b42-workload-socket" pod "a973c5a8-c25c-41ef-b2d7-7289fdc66b42" (UID: "a973c5a8-c25c-41ef-b2d7-7289fdc66b42"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:55:15.714546 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:15.714079    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a973c5a8-c25c-41ef-b2d7-7289fdc66b42-istio-data" pod "a973c5a8-c25c-41ef-b2d7-7289fdc66b42" (UID: "a973c5a8-c25c-41ef-b2d7-7289fdc66b42"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:55:15.714546 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:15.714364    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a973c5a8-c25c-41ef-b2d7-7289fdc66b42-istiod-ca-cert" pod "a973c5a8-c25c-41ef-b2d7-7289fdc66b42" (UID: "a973c5a8-c25c-41ef-b2d7-7289fdc66b42"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 00:55:15.714731 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:15.714596    4284 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a973c5a8-c25c-41ef-b2d7-7289fdc66b42-credential-socket\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:55:15.714731 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:15.714617    4284 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a973c5a8-c25c-41ef-b2d7-7289fdc66b42-workload-socket\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:55:15.714731 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:15.714631    4284 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a973c5a8-c25c-41ef-b2d7-7289fdc66b42-istio-data\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:55:15.714731 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:15.714645    4284 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a973c5a8-c25c-41ef-b2d7-7289fdc66b42-istiod-ca-cert\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:55:15.714731 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:15.714659    4284 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a973c5a8-c25c-41ef-b2d7-7289fdc66b42-workload-certs\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:55:15.715748 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:15.715725    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a973c5a8-c25c-41ef-b2d7-7289fdc66b42-istio-token" pod "a973c5a8-c25c-41ef-b2d7-7289fdc66b42" (UID: "a973c5a8-c25c-41ef-b2d7-7289fdc66b42"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 00:55:15.715945 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:15.715929    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a973c5a8-c25c-41ef-b2d7-7289fdc66b42-istio-podinfo" pod "a973c5a8-c25c-41ef-b2d7-7289fdc66b42" (UID: "a973c5a8-c25c-41ef-b2d7-7289fdc66b42"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 00:55:15.716372 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:15.716347    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a973c5a8-c25c-41ef-b2d7-7289fdc66b42-istio-envoy" pod "a973c5a8-c25c-41ef-b2d7-7289fdc66b42" (UID: "a973c5a8-c25c-41ef-b2d7-7289fdc66b42"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:55:15.716463 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:15.716375    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a973c5a8-c25c-41ef-b2d7-7289fdc66b42-kube-api-access-4z647" pod "a973c5a8-c25c-41ef-b2d7-7289fdc66b42" (UID: "a973c5a8-c25c-41ef-b2d7-7289fdc66b42"). InnerVolumeSpecName "kube-api-access-4z647". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 00:55:15.815754 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:15.815673    4284 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a973c5a8-c25c-41ef-b2d7-7289fdc66b42-istio-podinfo\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:55:15.815754 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:15.815703    4284 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a973c5a8-c25c-41ef-b2d7-7289fdc66b42-istio-envoy\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:55:15.815754 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:15.815712    4284 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a973c5a8-c25c-41ef-b2d7-7289fdc66b42-istio-token\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:55:15.815754 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:15.815721    4284 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4z647\" (UniqueName: \"kubernetes.io/projected/a973c5a8-c25c-41ef-b2d7-7289fdc66b42-kube-api-access-4z647\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:55:15.964107 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:15.964076    4284 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="044d4725-d84b-4f94-a7d4-451cb16c2d6b" path="/var/lib/kubelet/pods/044d4725-d84b-4f94-a7d4-451cb16c2d6b/volumes"
Oct 31 00:55:15.964541 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:15.964526    4284 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="53dde021-74fb-46c6-8340-6a84bc94a627" path="/var/lib/kubelet/pods/53dde021-74fb-46c6-8340-6a84bc94a627/volumes"
Oct 31 00:55:16.084519 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:16.084436    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-2sg7k"
Oct 31 00:55:16.084997 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:16.084425    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-2sg7k" event={"ID":"a973c5a8-c25c-41ef-b2d7-7289fdc66b42","Type":"ContainerDied","Data":"d4c34fdfa4a7e52d2b3d4fc2513b7a8dfb863fe6a9c31ae86c9361458c0ae369"}
Oct 31 00:55:16.084997 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:16.084594    4284 scope.go:122] "RemoveContainer" containerID="e9a2438f11440d23d8677436ffa1b3b02d67819e6942a1ea71860848383d9b95"
Oct 31 00:55:16.085989 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:16.085963    4284 generic.go:356] "Generic (PLEG): container finished" podID="fc082395-6c5f-4a62-8d0a-1e7d5d562c12" containerID="57366ca8c190c348c75806d2546bdbc28b788c48c95551a1dde53188378ad7a4" exitCode=0
Oct 31 00:55:16.086111 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:16.086042    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-wqns9"
Oct 31 00:55:16.086111 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:16.086049    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-wqns9" event={"ID":"fc082395-6c5f-4a62-8d0a-1e7d5d562c12","Type":"ContainerDied","Data":"57366ca8c190c348c75806d2546bdbc28b788c48c95551a1dde53188378ad7a4"}
Oct 31 00:55:16.086111 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:16.086090    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-wqns9" event={"ID":"fc082395-6c5f-4a62-8d0a-1e7d5d562c12","Type":"ContainerDied","Data":"cfb589e2fd6d38dc979463ee7c57705800f85a52a08fa33b526f3a5861e84e40"}
Oct 31 00:55:16.088021 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:16.087995    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-522s8" event={"ID":"7cfa573e-ec98-4eaf-b12f-01f723e5e878","Type":"ContainerDied","Data":"71dad47ca09eeffbde6ad4f681b754036361d03ce85ff68d1826f9df1b9ea634"}
Oct 31 00:55:16.088154 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:16.088003    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-522s8"
Oct 31 00:55:16.089981 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:16.089966    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-k89gx"
Oct 31 00:55:16.090052 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:16.090006    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-k89gx" event={"ID":"41a8f8d6-dbbb-48db-a957-5a3bfe8aa418","Type":"ContainerDied","Data":"f83e5d6047d314c4b0dbaba0af53c69e761d0e8b62c1558cc48a098a4910813b"}
Oct 31 00:55:16.096592 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:16.096575    4284 scope.go:122] "RemoveContainer" containerID="57366ca8c190c348c75806d2546bdbc28b788c48c95551a1dde53188378ad7a4"
Oct 31 00:55:16.105492 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:16.105471    4284 scope.go:122] "RemoveContainer" containerID="57366ca8c190c348c75806d2546bdbc28b788c48c95551a1dde53188378ad7a4"
Oct 31 00:55:16.105766 ip-10-0-132-173 kubenswrapper[4284]: E1031 00:55:16.105749    4284 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"57366ca8c190c348c75806d2546bdbc28b788c48c95551a1dde53188378ad7a4\": container with ID starting with 57366ca8c190c348c75806d2546bdbc28b788c48c95551a1dde53188378ad7a4 not found: ID does not exist" containerID="57366ca8c190c348c75806d2546bdbc28b788c48c95551a1dde53188378ad7a4"
Oct 31 00:55:16.105849 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:16.105778    4284 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"57366ca8c190c348c75806d2546bdbc28b788c48c95551a1dde53188378ad7a4"} err="failed to get container status \"57366ca8c190c348c75806d2546bdbc28b788c48c95551a1dde53188378ad7a4\": rpc error: code = NotFound desc = could not find container \"57366ca8c190c348c75806d2546bdbc28b788c48c95551a1dde53188378ad7a4\": container with ID starting with 57366ca8c190c348c75806d2546bdbc28b788c48c95551a1dde53188378ad7a4 not found: ID does not exist"
Oct 31 00:55:16.105849 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:16.105802    4284 scope.go:122] "RemoveContainer" containerID="e115a6c8d0cb6e96dec8c647554181f54149fda01c2429d3ea743bbf1be49c9f"
Oct 31 00:55:16.114222 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:16.114192    4284 scope.go:122] "RemoveContainer" containerID="06ab3e269d09bcaeed61b677bdc73a2b6b116cd3b10be0c1c7721ae11d7127d0"
Oct 31 00:55:16.245530 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:16.245497    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-k89gx"]
Oct 31 00:55:16.292169 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:16.292135    4284 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-k89gx"]
Oct 31 00:55:16.358978 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:16.358893    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-522s8"]
Oct 31 00:55:16.363985 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:16.363960    4284 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-522s8"]
Oct 31 00:55:16.468874 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:16.468839    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-2sg7k"]
Oct 31 00:55:16.476397 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:16.476374    4284 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-2sg7k"]
Oct 31 00:55:16.598824 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:16.598788    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-wqns9"]
Oct 31 00:55:16.603532 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:16.603506    4284 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w3-dd98f0-openshift-default-55954fcc8f-wqns9"]
Oct 31 00:55:17.013472 ip-10-0-132-173 kubenswrapper[4284]: E1031 00:55:17.013439    4284 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fd8sp" containerName="extractor"
Oct 31 00:55:17.965071 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:17.965041    4284 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="41a8f8d6-dbbb-48db-a957-5a3bfe8aa418" path="/var/lib/kubelet/pods/41a8f8d6-dbbb-48db-a957-5a3bfe8aa418/volumes"
Oct 31 00:55:17.965587 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:17.965571    4284 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="7cfa573e-ec98-4eaf-b12f-01f723e5e878" path="/var/lib/kubelet/pods/7cfa573e-ec98-4eaf-b12f-01f723e5e878/volumes"
Oct 31 00:55:17.965977 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:17.965966    4284 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a973c5a8-c25c-41ef-b2d7-7289fdc66b42" path="/var/lib/kubelet/pods/a973c5a8-c25c-41ef-b2d7-7289fdc66b42/volumes"
Oct 31 00:55:17.966658 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:17.966642    4284 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="fc082395-6c5f-4a62-8d0a-1e7d5d562c12" path="/var/lib/kubelet/pods/fc082395-6c5f-4a62-8d0a-1e7d5d562c12/volumes"
Oct 31 00:55:18.981711 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:18.981676    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:55:23.847364 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:23.847332    4284 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-ac952a-7795444fc5-jxsgs"]
Oct 31 00:55:23.847961 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:23.847946    4284 cpu_manager.go:414] "containerMap: removing container" podUID="41a8f8d6-dbbb-48db-a957-5a3bfe8aa418" containerName="istio-proxy"
Oct 31 00:55:23.848014 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:23.847962    4284 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="41a8f8d6-dbbb-48db-a957-5a3bfe8aa418" containerName="istio-proxy"
Oct 31 00:55:23.848014 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:23.847980    4284 cpu_manager.go:414] "containerMap: removing container" podUID="7cfa573e-ec98-4eaf-b12f-01f723e5e878" containerName="istio-proxy"
Oct 31 00:55:23.848014 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:23.847985    4284 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7cfa573e-ec98-4eaf-b12f-01f723e5e878" containerName="istio-proxy"
Oct 31 00:55:23.848014 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:23.847998    4284 cpu_manager.go:414] "containerMap: removing container" podUID="fc082395-6c5f-4a62-8d0a-1e7d5d562c12" containerName="istio-proxy"
Oct 31 00:55:23.848014 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:23.848003    4284 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="fc082395-6c5f-4a62-8d0a-1e7d5d562c12" containerName="istio-proxy"
Oct 31 00:55:23.848014 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:23.848010    4284 cpu_manager.go:414] "containerMap: removing container" podUID="53dde021-74fb-46c6-8340-6a84bc94a627" containerName="payload-processing"
Oct 31 00:55:23.848014 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:23.848015    4284 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="53dde021-74fb-46c6-8340-6a84bc94a627" containerName="payload-processing"
Oct 31 00:55:23.848242 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:23.848033    4284 cpu_manager.go:414] "containerMap: removing container" podUID="044d4725-d84b-4f94-a7d4-451cb16c2d6b" containerName="payload-pre-processing"
Oct 31 00:55:23.848242 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:23.848037    4284 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="044d4725-d84b-4f94-a7d4-451cb16c2d6b" containerName="payload-pre-processing"
Oct 31 00:55:23.848242 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:23.848048    4284 cpu_manager.go:414] "containerMap: removing container" podUID="a973c5a8-c25c-41ef-b2d7-7289fdc66b42" containerName="istio-proxy"
Oct 31 00:55:23.848242 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:23.848053    4284 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a973c5a8-c25c-41ef-b2d7-7289fdc66b42" containerName="istio-proxy"
Oct 31 00:55:23.848242 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:23.848135    4284 memory_manager.go:365] "RemoveStaleState removing state" podUID="41a8f8d6-dbbb-48db-a957-5a3bfe8aa418" containerName="istio-proxy"
Oct 31 00:55:23.848242 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:23.848143    4284 memory_manager.go:365] "RemoveStaleState removing state" podUID="a973c5a8-c25c-41ef-b2d7-7289fdc66b42" containerName="istio-proxy"
Oct 31 00:55:23.848242 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:23.848148    4284 memory_manager.go:365] "RemoveStaleState removing state" podUID="7cfa573e-ec98-4eaf-b12f-01f723e5e878" containerName="istio-proxy"
Oct 31 00:55:23.848242 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:23.848156    4284 memory_manager.go:365] "RemoveStaleState removing state" podUID="fc082395-6c5f-4a62-8d0a-1e7d5d562c12" containerName="istio-proxy"
Oct 31 00:55:23.848242 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:23.848160    4284 memory_manager.go:365] "RemoveStaleState removing state" podUID="53dde021-74fb-46c6-8340-6a84bc94a627" containerName="payload-processing"
Oct 31 00:55:23.848242 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:23.848171    4284 memory_manager.go:365] "RemoveStaleState removing state" podUID="044d4725-d84b-4f94-a7d4-451cb16c2d6b" containerName="payload-pre-processing"
Oct 31 00:55:23.852272 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:23.852252    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-ac952a-7795444fc5-jxsgs"
Oct 31 00:55:23.914511 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:23.914479    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/221ea88d-8906-49c5-9e00-c94af9002ea8-metrics-certs\") pod \"maas-api-e2e-ipp-a-ac952a-7795444fc5-jxsgs\" (UID: \"221ea88d-8906-49c5-9e00-c94af9002ea8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-ac952a-7795444fc5-jxsgs"
Oct 31 00:55:23.914677 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:23.914532    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/221ea88d-8906-49c5-9e00-c94af9002ea8-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-ac952a-7795444fc5-jxsgs\" (UID: \"221ea88d-8906-49c5-9e00-c94af9002ea8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-ac952a-7795444fc5-jxsgs"
Oct 31 00:55:23.914677 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:23.914622    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4rbzz\" (UniqueName: \"kubernetes.io/projected/221ea88d-8906-49c5-9e00-c94af9002ea8-kube-api-access-4rbzz\") pod \"maas-api-e2e-ipp-a-ac952a-7795444fc5-jxsgs\" (UID: \"221ea88d-8906-49c5-9e00-c94af9002ea8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-ac952a-7795444fc5-jxsgs"
Oct 31 00:55:23.914677 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:23.914665    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/221ea88d-8906-49c5-9e00-c94af9002ea8-maas-api-tls\") pod \"maas-api-e2e-ipp-a-ac952a-7795444fc5-jxsgs\" (UID: \"221ea88d-8906-49c5-9e00-c94af9002ea8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-ac952a-7795444fc5-jxsgs"
Oct 31 00:55:23.951168 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:23.951136    4284 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-a-ac952a-serving-cert\""
Oct 31 00:55:23.954491 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:23.954469    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-ac952a-7795444fc5-jxsgs"]
Oct 31 00:55:24.016259 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:24.016202    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/221ea88d-8906-49c5-9e00-c94af9002ea8-metrics-certs\") pod \"maas-api-e2e-ipp-a-ac952a-7795444fc5-jxsgs\" (UID: \"221ea88d-8906-49c5-9e00-c94af9002ea8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-ac952a-7795444fc5-jxsgs"
Oct 31 00:55:24.016449 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:24.016292    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/221ea88d-8906-49c5-9e00-c94af9002ea8-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-ac952a-7795444fc5-jxsgs\" (UID: \"221ea88d-8906-49c5-9e00-c94af9002ea8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-ac952a-7795444fc5-jxsgs"
Oct 31 00:55:24.016449 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:24.016323    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4rbzz\" (UniqueName: \"kubernetes.io/projected/221ea88d-8906-49c5-9e00-c94af9002ea8-kube-api-access-4rbzz\") pod \"maas-api-e2e-ipp-a-ac952a-7795444fc5-jxsgs\" (UID: \"221ea88d-8906-49c5-9e00-c94af9002ea8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-ac952a-7795444fc5-jxsgs"
Oct 31 00:55:24.016449 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:24.016372    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/221ea88d-8906-49c5-9e00-c94af9002ea8-maas-api-tls\") pod \"maas-api-e2e-ipp-a-ac952a-7795444fc5-jxsgs\" (UID: \"221ea88d-8906-49c5-9e00-c94af9002ea8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-ac952a-7795444fc5-jxsgs"
Oct 31 00:55:24.016607 ip-10-0-132-173 kubenswrapper[4284]: E1031 00:55:24.016578    4284 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-a-ac952a-serving-cert: secret "maas-api-e2e-ipp-a-ac952a-serving-cert" not found
Oct 31 00:55:24.016660 ip-10-0-132-173 kubenswrapper[4284]: E1031 00:55:24.016647    4284 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/221ea88d-8906-49c5-9e00-c94af9002ea8-maas-api-tls podName:221ea88d-8906-49c5-9e00-c94af9002ea8 nodeName:}" failed. No retries permitted until 2026-10-31 00:55:24.516626433 +0000 UTC m=+1638.661633340 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/221ea88d-8906-49c5-9e00-c94af9002ea8-maas-api-tls") pod "maas-api-e2e-ipp-a-ac952a-7795444fc5-jxsgs" (UID: "221ea88d-8906-49c5-9e00-c94af9002ea8") : secret "maas-api-e2e-ipp-a-ac952a-serving-cert" not found
Oct 31 00:55:24.017052 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:24.017020    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/221ea88d-8906-49c5-9e00-c94af9002ea8-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-ac952a-7795444fc5-jxsgs\" (UID: \"221ea88d-8906-49c5-9e00-c94af9002ea8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-ac952a-7795444fc5-jxsgs"
Oct 31 00:55:24.018879 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:24.018858    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/221ea88d-8906-49c5-9e00-c94af9002ea8-metrics-certs\") pod \"maas-api-e2e-ipp-a-ac952a-7795444fc5-jxsgs\" (UID: \"221ea88d-8906-49c5-9e00-c94af9002ea8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-ac952a-7795444fc5-jxsgs"
Oct 31 00:55:24.038958 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:24.038923    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:55:24.075642 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:24.075613    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4rbzz\" (UniqueName: \"kubernetes.io/projected/221ea88d-8906-49c5-9e00-c94af9002ea8-kube-api-access-4rbzz\") pod \"maas-api-e2e-ipp-a-ac952a-7795444fc5-jxsgs\" (UID: \"221ea88d-8906-49c5-9e00-c94af9002ea8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-ac952a-7795444fc5-jxsgs"
Oct 31 00:55:24.227185 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:24.227149    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:55:24.374581 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:24.374543    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:55:24.523065 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:24.522988    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/221ea88d-8906-49c5-9e00-c94af9002ea8-maas-api-tls\") pod \"maas-api-e2e-ipp-a-ac952a-7795444fc5-jxsgs\" (UID: \"221ea88d-8906-49c5-9e00-c94af9002ea8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-ac952a-7795444fc5-jxsgs"
Oct 31 00:55:24.525613 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:24.525587    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/221ea88d-8906-49c5-9e00-c94af9002ea8-maas-api-tls\") pod \"maas-api-e2e-ipp-a-ac952a-7795444fc5-jxsgs\" (UID: \"221ea88d-8906-49c5-9e00-c94af9002ea8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-ac952a-7795444fc5-jxsgs"
Oct 31 00:55:24.765268 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:24.765233    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-ac952a-7795444fc5-jxsgs"
Oct 31 00:55:25.080133 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:25.080055    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-f66180-97cbb6f6c-t5b4n"
Oct 31 00:55:25.080563 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:25.080137    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-f66180-794b84955-ndwz8"
Oct 31 00:55:25.137272 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:55:25.137238    4284 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod221ea88d_8906_49c5_9e00_c94af9002ea8.slice/crio-63035fb902cf7d0f75568b28546abd2135a56515b1733aac89d9add249d76976 WatchSource:0}: Error finding container 63035fb902cf7d0f75568b28546abd2135a56515b1733aac89d9add249d76976: Status 404 returned error can't find the container with id 63035fb902cf7d0f75568b28546abd2135a56515b1733aac89d9add249d76976
Oct 31 00:55:25.151401 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:25.151369    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-ac952a-7795444fc5-jxsgs" event={"ID":"221ea88d-8906-49c5-9e00-c94af9002ea8","Type":"ContainerStarted","Data":"63035fb902cf7d0f75568b28546abd2135a56515b1733aac89d9add249d76976"}
Oct 31 00:55:25.181066 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:25.181033    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-ac952a-7795444fc5-jxsgs"]
Oct 31 00:55:27.167993 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:27.167950    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-ac952a-7795444fc5-jxsgs" event={"ID":"221ea88d-8906-49c5-9e00-c94af9002ea8","Type":"ContainerStarted","Data":"6e80a29e13171240ad67f7ddd983535cf487e4f68fd4c289bf499aaa7c5523c0"}
Oct 31 00:55:27.168499 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:27.168054    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-ac952a-7795444fc5-jxsgs"
Oct 31 00:55:27.281941 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:27.281891    4284 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-ac952a-7795444fc5-jxsgs" podStartSLOduration=3.145601813 podStartE2EDuration="4.28187566s" totalImagesPullingTime="1.136273847s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 00:55:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:55:27.281593747 +0000 UTC m=+1641.426600664" watchObservedRunningTime="2026-10-31 00:55:27.28187566 +0000 UTC m=+1641.426882575"
Oct 31 00:55:28.293882 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:28.293849    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-69a4bc-69465f7c7-kvvq4"]
Oct 31 00:55:28.396748 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:28.396717    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-69a4bc-854d4ddb7-5gwjd"]
Oct 31 00:55:28.661999 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:28.661970    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:55:28.885460 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:28.885399    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:55:29.434925 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:29.434876    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:55:33.179585 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:33.179554    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-ac952a-7795444fc5-jxsgs"
Oct 31 00:55:33.294233 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:33.294166    4284 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-a-69a4bc-69465f7c7-kvvq4" podUID="79898810-a2f5-42bb-ba69-e9a0bd57b0de" containerName="payload-processing" containerID="cri-o://4c1e7b95e5e485d5a7a158ba9fbeecc54dba0b184083e3d900396e49ba5c3507" gracePeriod=25
Oct 31 00:55:33.397996 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:33.397930    4284 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-a-69a4bc-854d4ddb7-5gwjd" podUID="fa5fde86-708a-4f57-9e4b-3eb7e311e09c" containerName="payload-pre-processing" containerID="cri-o://53004a879cdd829a49f534c3bc6b222750a09e3ab8fa06b9a0f97b950f440728" gracePeriod=25
Oct 31 00:55:33.564839 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:33.564806    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-69a4bc-69465f7c7-kvvq4"
Oct 31 00:55:33.679239 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:33.679176    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/79898810-a2f5-42bb-ba69-e9a0bd57b0de-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/79898810-a2f5-42bb-ba69-e9a0bd57b0de-plugins-config-volume\") pod \"79898810-a2f5-42bb-ba69-e9a0bd57b0de\" (UID: \"79898810-a2f5-42bb-ba69-e9a0bd57b0de\") "
Oct 31 00:55:33.679420 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:33.679402    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/79898810-a2f5-42bb-ba69-e9a0bd57b0de-kube-api-access-flnfr\" (UniqueName: \"kubernetes.io/projected/79898810-a2f5-42bb-ba69-e9a0bd57b0de-kube-api-access-flnfr\") pod \"79898810-a2f5-42bb-ba69-e9a0bd57b0de\" (UID: \"79898810-a2f5-42bb-ba69-e9a0bd57b0de\") "
Oct 31 00:55:33.679616 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:33.679586    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/79898810-a2f5-42bb-ba69-e9a0bd57b0de-plugins-config-volume" pod "79898810-a2f5-42bb-ba69-e9a0bd57b0de" (UID: "79898810-a2f5-42bb-ba69-e9a0bd57b0de"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 00:55:33.679872 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:33.679851    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-69a4bc-854d4ddb7-5gwjd"
Oct 31 00:55:33.679974 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:33.679948    4284 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/79898810-a2f5-42bb-ba69-e9a0bd57b0de-plugins-config-volume\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:55:33.681850 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:33.681824    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/79898810-a2f5-42bb-ba69-e9a0bd57b0de-kube-api-access-flnfr" pod "79898810-a2f5-42bb-ba69-e9a0bd57b0de" (UID: "79898810-a2f5-42bb-ba69-e9a0bd57b0de"). InnerVolumeSpecName "kube-api-access-flnfr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 00:55:33.781255 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:33.781128    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/fa5fde86-708a-4f57-9e4b-3eb7e311e09c-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fa5fde86-708a-4f57-9e4b-3eb7e311e09c-plugins-config-volume\") pod \"fa5fde86-708a-4f57-9e4b-3eb7e311e09c\" (UID: \"fa5fde86-708a-4f57-9e4b-3eb7e311e09c\") "
Oct 31 00:55:33.781422 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:33.781303    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fa5fde86-708a-4f57-9e4b-3eb7e311e09c-kube-api-access-742lx\" (UniqueName: \"kubernetes.io/projected/fa5fde86-708a-4f57-9e4b-3eb7e311e09c-kube-api-access-742lx\") pod \"fa5fde86-708a-4f57-9e4b-3eb7e311e09c\" (UID: \"fa5fde86-708a-4f57-9e4b-3eb7e311e09c\") "
Oct 31 00:55:33.781572 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:33.781537    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fa5fde86-708a-4f57-9e4b-3eb7e311e09c-plugins-config-volume" pod "fa5fde86-708a-4f57-9e4b-3eb7e311e09c" (UID: "fa5fde86-708a-4f57-9e4b-3eb7e311e09c"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 00:55:33.781830 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:33.781814    4284 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fa5fde86-708a-4f57-9e4b-3eb7e311e09c-plugins-config-volume\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:55:33.781900 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:33.781834    4284 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-flnfr\" (UniqueName: \"kubernetes.io/projected/79898810-a2f5-42bb-ba69-e9a0bd57b0de-kube-api-access-flnfr\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:55:33.783549 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:33.783525    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fa5fde86-708a-4f57-9e4b-3eb7e311e09c-kube-api-access-742lx" pod "fa5fde86-708a-4f57-9e4b-3eb7e311e09c" (UID: "fa5fde86-708a-4f57-9e4b-3eb7e311e09c"). InnerVolumeSpecName "kube-api-access-742lx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 00:55:33.882760 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:33.882729    4284 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-742lx\" (UniqueName: \"kubernetes.io/projected/fa5fde86-708a-4f57-9e4b-3eb7e311e09c-kube-api-access-742lx\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:55:34.049803 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:34.049733    4284 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-7e6fe88b-55755c67c7-rct6j"]
Oct 31 00:55:34.050335 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:34.050322    4284 cpu_manager.go:414] "containerMap: removing container" podUID="79898810-a2f5-42bb-ba69-e9a0bd57b0de" containerName="payload-processing"
Oct 31 00:55:34.050389 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:34.050336    4284 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="79898810-a2f5-42bb-ba69-e9a0bd57b0de" containerName="payload-processing"
Oct 31 00:55:34.050389 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:34.050358    4284 cpu_manager.go:414] "containerMap: removing container" podUID="fa5fde86-708a-4f57-9e4b-3eb7e311e09c" containerName="payload-pre-processing"
Oct 31 00:55:34.050389 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:34.050365    4284 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="fa5fde86-708a-4f57-9e4b-3eb7e311e09c" containerName="payload-pre-processing"
Oct 31 00:55:34.050482 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:34.050449    4284 memory_manager.go:365] "RemoveStaleState removing state" podUID="79898810-a2f5-42bb-ba69-e9a0bd57b0de" containerName="payload-processing"
Oct 31 00:55:34.050482 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:34.050462    4284 memory_manager.go:365] "RemoveStaleState removing state" podUID="fa5fde86-708a-4f57-9e4b-3eb7e311e09c" containerName="payload-pre-processing"
Oct 31 00:55:34.054144 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:34.054126    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-7e6fe88b-55755c67c7-rct6j"
Oct 31 00:55:34.105730 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:34.105693    4284 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-7e6fe88b-serving-cert\""
Oct 31 00:55:34.109680 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:34.109646    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-7e6fe88b-55755c67c7-rct6j"]
Oct 31 00:55:34.187234 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:34.187187    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/376421f4-5e4b-457f-bf37-a8f794171def-maas-api-tls\") pod \"maas-api-e2e-mt-7e6fe88b-55755c67c7-rct6j\" (UID: \"376421f4-5e4b-457f-bf37-a8f794171def\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-7e6fe88b-55755c67c7-rct6j"
Oct 31 00:55:34.187729 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:34.187339    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/376421f4-5e4b-457f-bf37-a8f794171def-metrics-certs\") pod \"maas-api-e2e-mt-7e6fe88b-55755c67c7-rct6j\" (UID: \"376421f4-5e4b-457f-bf37-a8f794171def\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-7e6fe88b-55755c67c7-rct6j"
Oct 31 00:55:34.187729 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:34.187490    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/376421f4-5e4b-457f-bf37-a8f794171def-openshift-service-ca\") pod \"maas-api-e2e-mt-7e6fe88b-55755c67c7-rct6j\" (UID: \"376421f4-5e4b-457f-bf37-a8f794171def\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-7e6fe88b-55755c67c7-rct6j"
Oct 31 00:55:34.187729 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:34.187543    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q6hbp\" (UniqueName: \"kubernetes.io/projected/376421f4-5e4b-457f-bf37-a8f794171def-kube-api-access-q6hbp\") pod \"maas-api-e2e-mt-7e6fe88b-55755c67c7-rct6j\" (UID: \"376421f4-5e4b-457f-bf37-a8f794171def\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-7e6fe88b-55755c67c7-rct6j"
Oct 31 00:55:34.211636 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:34.211611    4284 generic.go:356] "Generic (PLEG): container finished" podID="fa5fde86-708a-4f57-9e4b-3eb7e311e09c" containerID="53004a879cdd829a49f534c3bc6b222750a09e3ab8fa06b9a0f97b950f440728" exitCode=0
Oct 31 00:55:34.211776 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:34.211652    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-69a4bc-854d4ddb7-5gwjd"
Oct 31 00:55:34.211776 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:34.211690    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-69a4bc-854d4ddb7-5gwjd" event={"ID":"fa5fde86-708a-4f57-9e4b-3eb7e311e09c","Type":"ContainerDied","Data":"53004a879cdd829a49f534c3bc6b222750a09e3ab8fa06b9a0f97b950f440728"}
Oct 31 00:55:34.211776 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:34.211738    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-69a4bc-854d4ddb7-5gwjd" event={"ID":"fa5fde86-708a-4f57-9e4b-3eb7e311e09c","Type":"ContainerDied","Data":"5b5fd32e78dbd4dcadd04e9ae07410c30f5d2bbff439cf536bb6549f2dfd71e8"}
Oct 31 00:55:34.211776 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:34.211761    4284 scope.go:122] "RemoveContainer" containerID="53004a879cdd829a49f534c3bc6b222750a09e3ab8fa06b9a0f97b950f440728"
Oct 31 00:55:34.213108 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:34.213077    4284 generic.go:356] "Generic (PLEG): container finished" podID="79898810-a2f5-42bb-ba69-e9a0bd57b0de" containerID="4c1e7b95e5e485d5a7a158ba9fbeecc54dba0b184083e3d900396e49ba5c3507" exitCode=0
Oct 31 00:55:34.213317 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:34.213106    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-69a4bc-69465f7c7-kvvq4" event={"ID":"79898810-a2f5-42bb-ba69-e9a0bd57b0de","Type":"ContainerDied","Data":"4c1e7b95e5e485d5a7a158ba9fbeecc54dba0b184083e3d900396e49ba5c3507"}
Oct 31 00:55:34.213317 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:34.213126    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-69a4bc-69465f7c7-kvvq4"
Oct 31 00:55:34.213317 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:34.213149    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-69a4bc-69465f7c7-kvvq4" event={"ID":"79898810-a2f5-42bb-ba69-e9a0bd57b0de","Type":"ContainerDied","Data":"cd346096d27b53882b386cb67cc53954693aa1063db437b1c9dd8e0198a2352a"}
Oct 31 00:55:34.222435 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:34.222411    4284 scope.go:122] "RemoveContainer" containerID="53004a879cdd829a49f534c3bc6b222750a09e3ab8fa06b9a0f97b950f440728"
Oct 31 00:55:34.222688 ip-10-0-132-173 kubenswrapper[4284]: E1031 00:55:34.222671    4284 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"53004a879cdd829a49f534c3bc6b222750a09e3ab8fa06b9a0f97b950f440728\": container with ID starting with 53004a879cdd829a49f534c3bc6b222750a09e3ab8fa06b9a0f97b950f440728 not found: ID does not exist" containerID="53004a879cdd829a49f534c3bc6b222750a09e3ab8fa06b9a0f97b950f440728"
Oct 31 00:55:34.222738 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:34.222695    4284 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"53004a879cdd829a49f534c3bc6b222750a09e3ab8fa06b9a0f97b950f440728"} err="failed to get container status \"53004a879cdd829a49f534c3bc6b222750a09e3ab8fa06b9a0f97b950f440728\": rpc error: code = NotFound desc = could not find container \"53004a879cdd829a49f534c3bc6b222750a09e3ab8fa06b9a0f97b950f440728\": container with ID starting with 53004a879cdd829a49f534c3bc6b222750a09e3ab8fa06b9a0f97b950f440728 not found: ID does not exist"
Oct 31 00:55:34.222738 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:34.222711    4284 scope.go:122] "RemoveContainer" containerID="4c1e7b95e5e485d5a7a158ba9fbeecc54dba0b184083e3d900396e49ba5c3507"
Oct 31 00:55:34.230921 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:34.230908    4284 scope.go:122] "RemoveContainer" containerID="4c1e7b95e5e485d5a7a158ba9fbeecc54dba0b184083e3d900396e49ba5c3507"
Oct 31 00:55:34.231191 ip-10-0-132-173 kubenswrapper[4284]: E1031 00:55:34.231174    4284 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4c1e7b95e5e485d5a7a158ba9fbeecc54dba0b184083e3d900396e49ba5c3507\": container with ID starting with 4c1e7b95e5e485d5a7a158ba9fbeecc54dba0b184083e3d900396e49ba5c3507 not found: ID does not exist" containerID="4c1e7b95e5e485d5a7a158ba9fbeecc54dba0b184083e3d900396e49ba5c3507"
Oct 31 00:55:34.231274 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:34.231197    4284 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4c1e7b95e5e485d5a7a158ba9fbeecc54dba0b184083e3d900396e49ba5c3507"} err="failed to get container status \"4c1e7b95e5e485d5a7a158ba9fbeecc54dba0b184083e3d900396e49ba5c3507\": rpc error: code = NotFound desc = could not find container \"4c1e7b95e5e485d5a7a158ba9fbeecc54dba0b184083e3d900396e49ba5c3507\": container with ID starting with 4c1e7b95e5e485d5a7a158ba9fbeecc54dba0b184083e3d900396e49ba5c3507 not found: ID does not exist"
Oct 31 00:55:34.251405 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:34.251384    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:55:34.289273 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:34.289249    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/376421f4-5e4b-457f-bf37-a8f794171def-metrics-certs\") pod \"maas-api-e2e-mt-7e6fe88b-55755c67c7-rct6j\" (UID: \"376421f4-5e4b-457f-bf37-a8f794171def\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-7e6fe88b-55755c67c7-rct6j"
Oct 31 00:55:34.289411 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:34.289323    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/376421f4-5e4b-457f-bf37-a8f794171def-openshift-service-ca\") pod \"maas-api-e2e-mt-7e6fe88b-55755c67c7-rct6j\" (UID: \"376421f4-5e4b-457f-bf37-a8f794171def\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-7e6fe88b-55755c67c7-rct6j"
Oct 31 00:55:34.289411 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:34.289352    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q6hbp\" (UniqueName: \"kubernetes.io/projected/376421f4-5e4b-457f-bf37-a8f794171def-kube-api-access-q6hbp\") pod \"maas-api-e2e-mt-7e6fe88b-55755c67c7-rct6j\" (UID: \"376421f4-5e4b-457f-bf37-a8f794171def\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-7e6fe88b-55755c67c7-rct6j"
Oct 31 00:55:34.289521 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:34.289470    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/376421f4-5e4b-457f-bf37-a8f794171def-maas-api-tls\") pod \"maas-api-e2e-mt-7e6fe88b-55755c67c7-rct6j\" (UID: \"376421f4-5e4b-457f-bf37-a8f794171def\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-7e6fe88b-55755c67c7-rct6j"
Oct 31 00:55:34.289966 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:34.289944    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/376421f4-5e4b-457f-bf37-a8f794171def-openshift-service-ca\") pod \"maas-api-e2e-mt-7e6fe88b-55755c67c7-rct6j\" (UID: \"376421f4-5e4b-457f-bf37-a8f794171def\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-7e6fe88b-55755c67c7-rct6j"
Oct 31 00:55:34.291741 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:34.291724    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/376421f4-5e4b-457f-bf37-a8f794171def-metrics-certs\") pod \"maas-api-e2e-mt-7e6fe88b-55755c67c7-rct6j\" (UID: \"376421f4-5e4b-457f-bf37-a8f794171def\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-7e6fe88b-55755c67c7-rct6j"
Oct 31 00:55:34.291801 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:34.291769    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/376421f4-5e4b-457f-bf37-a8f794171def-maas-api-tls\") pod \"maas-api-e2e-mt-7e6fe88b-55755c67c7-rct6j\" (UID: \"376421f4-5e4b-457f-bf37-a8f794171def\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-7e6fe88b-55755c67c7-rct6j"
Oct 31 00:55:34.348513 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:34.348485    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q6hbp\" (UniqueName: \"kubernetes.io/projected/376421f4-5e4b-457f-bf37-a8f794171def-kube-api-access-q6hbp\") pod \"maas-api-e2e-mt-7e6fe88b-55755c67c7-rct6j\" (UID: \"376421f4-5e4b-457f-bf37-a8f794171def\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-7e6fe88b-55755c67c7-rct6j"
Oct 31 00:55:34.367454 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:34.367422    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-7e6fe88b-55755c67c7-rct6j"
Oct 31 00:55:34.370749 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:34.370729    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-69a4bc-69465f7c7-kvvq4"]
Oct 31 00:55:34.376756 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:34.376726    4284 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-69a4bc-69465f7c7-kvvq4"]
Oct 31 00:55:34.505237 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:34.504893    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:55:34.565273 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:34.565183    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-69a4bc-854d4ddb7-5gwjd"]
Oct 31 00:55:34.573070 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:34.572946    4284 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-69a4bc-854d4ddb7-5gwjd"]
Oct 31 00:55:34.743720 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:34.743690    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-7e6fe88b-55755c67c7-rct6j"]
Oct 31 00:55:34.744284 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:55:34.744256    4284 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod376421f4_5e4b_457f_bf37_a8f794171def.slice/crio-c3668db7c120a1c31eb0df7fcd1abe0f80f68d807f712072a1c67497d12966fe WatchSource:0}: Error finding container c3668db7c120a1c31eb0df7fcd1abe0f80f68d807f712072a1c67497d12966fe: Status 404 returned error can't find the container with id c3668db7c120a1c31eb0df7fcd1abe0f80f68d807f712072a1c67497d12966fe
Oct 31 00:55:35.224546 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:35.224516    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-7e6fe88b-55755c67c7-rct6j" event={"ID":"376421f4-5e4b-457f-bf37-a8f794171def","Type":"ContainerStarted","Data":"c3668db7c120a1c31eb0df7fcd1abe0f80f68d807f712072a1c67497d12966fe"}
Oct 31 00:55:35.967031 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:35.966992    4284 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="79898810-a2f5-42bb-ba69-e9a0bd57b0de" path="/var/lib/kubelet/pods/79898810-a2f5-42bb-ba69-e9a0bd57b0de/volumes"
Oct 31 00:55:35.968015 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:35.967990    4284 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="fa5fde86-708a-4f57-9e4b-3eb7e311e09c" path="/var/lib/kubelet/pods/fa5fde86-708a-4f57-9e4b-3eb7e311e09c/volumes"
Oct 31 00:55:36.234483 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:36.234444    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-7e6fe88b-55755c67c7-rct6j" event={"ID":"376421f4-5e4b-457f-bf37-a8f794171def","Type":"ContainerStarted","Data":"ae5b5fa52c6375596d189375d0b77597b8bc17f992ade7ee1940fa52aed83226"}
Oct 31 00:55:36.234883 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:36.234566    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-7e6fe88b-55755c67c7-rct6j"
Oct 31 00:55:36.351243 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:36.348112    4284 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-7e6fe88b-55755c67c7-rct6j" podStartSLOduration=1.352537658 podStartE2EDuration="2.348092831s" totalImagesPullingTime="995.555173ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 00:55:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:55:36.347636457 +0000 UTC m=+1650.492643389" watchObservedRunningTime="2026-10-31 00:55:36.348092831 +0000 UTC m=+1650.493099747"
Oct 31 00:55:37.272013 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:37.271983    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-7n64p"]
Oct 31 00:55:37.272481 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:37.272175    4284 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-7n64p" podUID="b4556139-5a27-42e5-ae33-a85c75ea1209" containerName="istio-proxy" containerID="cri-o://a9db7c1234f6aab814b5b9e8b16f3cce82a0afe028a1a1bc819e2eaa9ac6475d" gracePeriod=30
Oct 31 00:55:37.319642 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:37.319607    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-fnnxh"]
Oct 31 00:55:37.319874 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:37.319847    4284 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-fnnxh" podUID="55d97eb9-77d5-4b56-ab6d-5f38df8d1deb" containerName="istio-proxy" containerID="cri-o://6a9eaf70cfdeeb456778b08b524ea4131bb75313e5c24c3ece3e1836171e1cf1" gracePeriod=30
Oct 31 00:55:39.284389 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:39.284352    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:55:42.246892 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:42.246862    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-7e6fe88b-55755c67c7-rct6j"
Oct 31 00:55:42.596041 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:42.595708    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-7n64p"
Oct 31 00:55:42.633098 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:42.633068    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b4556139-5a27-42e5-ae33-a85c75ea1209-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b4556139-5a27-42e5-ae33-a85c75ea1209-workload-certs\") pod \"b4556139-5a27-42e5-ae33-a85c75ea1209\" (UID: \"b4556139-5a27-42e5-ae33-a85c75ea1209\") "
Oct 31 00:55:42.633322 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:42.633110    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b4556139-5a27-42e5-ae33-a85c75ea1209-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b4556139-5a27-42e5-ae33-a85c75ea1209-credential-socket\") pod \"b4556139-5a27-42e5-ae33-a85c75ea1209\" (UID: \"b4556139-5a27-42e5-ae33-a85c75ea1209\") "
Oct 31 00:55:42.633322 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:42.633143    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/b4556139-5a27-42e5-ae33-a85c75ea1209-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b4556139-5a27-42e5-ae33-a85c75ea1209-istio-podinfo\") pod \"b4556139-5a27-42e5-ae33-a85c75ea1209\" (UID: \"b4556139-5a27-42e5-ae33-a85c75ea1209\") "
Oct 31 00:55:42.633322 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:42.633179    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b4556139-5a27-42e5-ae33-a85c75ea1209-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b4556139-5a27-42e5-ae33-a85c75ea1209-workload-socket\") pod \"b4556139-5a27-42e5-ae33-a85c75ea1209\" (UID: \"b4556139-5a27-42e5-ae33-a85c75ea1209\") "
Oct 31 00:55:42.633322 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:42.633247    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b4556139-5a27-42e5-ae33-a85c75ea1209-kube-api-access-8fm2w\" (UniqueName: \"kubernetes.io/projected/b4556139-5a27-42e5-ae33-a85c75ea1209-kube-api-access-8fm2w\") pod \"b4556139-5a27-42e5-ae33-a85c75ea1209\" (UID: \"b4556139-5a27-42e5-ae33-a85c75ea1209\") "
Oct 31 00:55:42.633322 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:42.633283    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b4556139-5a27-42e5-ae33-a85c75ea1209-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b4556139-5a27-42e5-ae33-a85c75ea1209-istio-data\") pod \"b4556139-5a27-42e5-ae33-a85c75ea1209\" (UID: \"b4556139-5a27-42e5-ae33-a85c75ea1209\") "
Oct 31 00:55:42.633322 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:42.633302    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b4556139-5a27-42e5-ae33-a85c75ea1209-istio-token\" (UniqueName: \"kubernetes.io/projected/b4556139-5a27-42e5-ae33-a85c75ea1209-istio-token\") pod \"b4556139-5a27-42e5-ae33-a85c75ea1209\" (UID: \"b4556139-5a27-42e5-ae33-a85c75ea1209\") "
Oct 31 00:55:42.633652 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:42.633371    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b4556139-5a27-42e5-ae33-a85c75ea1209-workload-certs" pod "b4556139-5a27-42e5-ae33-a85c75ea1209" (UID: "b4556139-5a27-42e5-ae33-a85c75ea1209"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:55:42.633652 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:42.633386    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b4556139-5a27-42e5-ae33-a85c75ea1209-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b4556139-5a27-42e5-ae33-a85c75ea1209-istiod-ca-cert\") pod \"b4556139-5a27-42e5-ae33-a85c75ea1209\" (UID: \"b4556139-5a27-42e5-ae33-a85c75ea1209\") "
Oct 31 00:55:42.633652 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:42.633451    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b4556139-5a27-42e5-ae33-a85c75ea1209-credential-socket" pod "b4556139-5a27-42e5-ae33-a85c75ea1209" (UID: "b4556139-5a27-42e5-ae33-a85c75ea1209"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:55:42.633652 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:42.633588    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b4556139-5a27-42e5-ae33-a85c75ea1209-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b4556139-5a27-42e5-ae33-a85c75ea1209-istio-envoy\") pod \"b4556139-5a27-42e5-ae33-a85c75ea1209\" (UID: \"b4556139-5a27-42e5-ae33-a85c75ea1209\") "
Oct 31 00:55:42.633874 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:42.633658    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b4556139-5a27-42e5-ae33-a85c75ea1209-istiod-ca-cert" pod "b4556139-5a27-42e5-ae33-a85c75ea1209" (UID: "b4556139-5a27-42e5-ae33-a85c75ea1209"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 00:55:42.633929 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:42.633901    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b4556139-5a27-42e5-ae33-a85c75ea1209-istio-data" pod "b4556139-5a27-42e5-ae33-a85c75ea1209" (UID: "b4556139-5a27-42e5-ae33-a85c75ea1209"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:55:42.633985 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:42.633770    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b4556139-5a27-42e5-ae33-a85c75ea1209-workload-socket" pod "b4556139-5a27-42e5-ae33-a85c75ea1209" (UID: "b4556139-5a27-42e5-ae33-a85c75ea1209"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:55:42.634542 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:42.634520    4284 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b4556139-5a27-42e5-ae33-a85c75ea1209-istiod-ca-cert\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:55:42.634667 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:42.634544    4284 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b4556139-5a27-42e5-ae33-a85c75ea1209-workload-certs\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:55:42.634667 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:42.634558    4284 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b4556139-5a27-42e5-ae33-a85c75ea1209-credential-socket\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:55:42.634667 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:42.634569    4284 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b4556139-5a27-42e5-ae33-a85c75ea1209-workload-socket\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:55:42.634667 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:42.634583    4284 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b4556139-5a27-42e5-ae33-a85c75ea1209-istio-data\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:55:42.636166 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:42.636140    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b4556139-5a27-42e5-ae33-a85c75ea1209-istio-token" pod "b4556139-5a27-42e5-ae33-a85c75ea1209" (UID: "b4556139-5a27-42e5-ae33-a85c75ea1209"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 00:55:42.636166 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:42.636151    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b4556139-5a27-42e5-ae33-a85c75ea1209-kube-api-access-8fm2w" pod "b4556139-5a27-42e5-ae33-a85c75ea1209" (UID: "b4556139-5a27-42e5-ae33-a85c75ea1209"). InnerVolumeSpecName "kube-api-access-8fm2w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 00:55:42.636366 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:42.636246    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b4556139-5a27-42e5-ae33-a85c75ea1209-istio-podinfo" pod "b4556139-5a27-42e5-ae33-a85c75ea1209" (UID: "b4556139-5a27-42e5-ae33-a85c75ea1209"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 00:55:42.636442 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:42.636418    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b4556139-5a27-42e5-ae33-a85c75ea1209-istio-envoy" pod "b4556139-5a27-42e5-ae33-a85c75ea1209" (UID: "b4556139-5a27-42e5-ae33-a85c75ea1209"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:55:42.657679 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:42.657656    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-fnnxh"
Oct 31 00:55:42.735197 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:42.735164    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/55d97eb9-77d5-4b56-ab6d-5f38df8d1deb-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/55d97eb9-77d5-4b56-ab6d-5f38df8d1deb-workload-socket\") pod \"55d97eb9-77d5-4b56-ab6d-5f38df8d1deb\" (UID: \"55d97eb9-77d5-4b56-ab6d-5f38df8d1deb\") "
Oct 31 00:55:42.735395 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:42.735258    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/55d97eb9-77d5-4b56-ab6d-5f38df8d1deb-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/55d97eb9-77d5-4b56-ab6d-5f38df8d1deb-workload-certs\") pod \"55d97eb9-77d5-4b56-ab6d-5f38df8d1deb\" (UID: \"55d97eb9-77d5-4b56-ab6d-5f38df8d1deb\") "
Oct 31 00:55:42.735395 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:42.735328    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/55d97eb9-77d5-4b56-ab6d-5f38df8d1deb-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/55d97eb9-77d5-4b56-ab6d-5f38df8d1deb-credential-socket\") pod \"55d97eb9-77d5-4b56-ab6d-5f38df8d1deb\" (UID: \"55d97eb9-77d5-4b56-ab6d-5f38df8d1deb\") "
Oct 31 00:55:42.735395 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:42.735368    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/55d97eb9-77d5-4b56-ab6d-5f38df8d1deb-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/55d97eb9-77d5-4b56-ab6d-5f38df8d1deb-istiod-ca-cert\") pod \"55d97eb9-77d5-4b56-ab6d-5f38df8d1deb\" (UID: \"55d97eb9-77d5-4b56-ab6d-5f38df8d1deb\") "
Oct 31 00:55:42.735563 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:42.735416    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/55d97eb9-77d5-4b56-ab6d-5f38df8d1deb-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/55d97eb9-77d5-4b56-ab6d-5f38df8d1deb-istio-envoy\") pod \"55d97eb9-77d5-4b56-ab6d-5f38df8d1deb\" (UID: \"55d97eb9-77d5-4b56-ab6d-5f38df8d1deb\") "
Oct 31 00:55:42.735563 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:42.735456    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/55d97eb9-77d5-4b56-ab6d-5f38df8d1deb-istio-token\" (UniqueName: \"kubernetes.io/projected/55d97eb9-77d5-4b56-ab6d-5f38df8d1deb-istio-token\") pod \"55d97eb9-77d5-4b56-ab6d-5f38df8d1deb\" (UID: \"55d97eb9-77d5-4b56-ab6d-5f38df8d1deb\") "
Oct 31 00:55:42.735563 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:42.735457    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/55d97eb9-77d5-4b56-ab6d-5f38df8d1deb-workload-socket" pod "55d97eb9-77d5-4b56-ab6d-5f38df8d1deb" (UID: "55d97eb9-77d5-4b56-ab6d-5f38df8d1deb"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:55:42.735563 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:42.735493    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/55d97eb9-77d5-4b56-ab6d-5f38df8d1deb-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/55d97eb9-77d5-4b56-ab6d-5f38df8d1deb-istio-podinfo\") pod \"55d97eb9-77d5-4b56-ab6d-5f38df8d1deb\" (UID: \"55d97eb9-77d5-4b56-ab6d-5f38df8d1deb\") "
Oct 31 00:55:42.735563 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:42.735535    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/55d97eb9-77d5-4b56-ab6d-5f38df8d1deb-workload-certs" pod "55d97eb9-77d5-4b56-ab6d-5f38df8d1deb" (UID: "55d97eb9-77d5-4b56-ab6d-5f38df8d1deb"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:55:42.735824 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:42.735571    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/55d97eb9-77d5-4b56-ab6d-5f38df8d1deb-kube-api-access-tcs2z\" (UniqueName: \"kubernetes.io/projected/55d97eb9-77d5-4b56-ab6d-5f38df8d1deb-kube-api-access-tcs2z\") pod \"55d97eb9-77d5-4b56-ab6d-5f38df8d1deb\" (UID: \"55d97eb9-77d5-4b56-ab6d-5f38df8d1deb\") "
Oct 31 00:55:42.735824 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:42.735622    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/55d97eb9-77d5-4b56-ab6d-5f38df8d1deb-credential-socket" pod "55d97eb9-77d5-4b56-ab6d-5f38df8d1deb" (UID: "55d97eb9-77d5-4b56-ab6d-5f38df8d1deb"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:55:42.735938 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:42.735822    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/55d97eb9-77d5-4b56-ab6d-5f38df8d1deb-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/55d97eb9-77d5-4b56-ab6d-5f38df8d1deb-istio-data\") pod \"55d97eb9-77d5-4b56-ab6d-5f38df8d1deb\" (UID: \"55d97eb9-77d5-4b56-ab6d-5f38df8d1deb\") "
Oct 31 00:55:42.736143 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:42.736118    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/55d97eb9-77d5-4b56-ab6d-5f38df8d1deb-istio-data" pod "55d97eb9-77d5-4b56-ab6d-5f38df8d1deb" (UID: "55d97eb9-77d5-4b56-ab6d-5f38df8d1deb"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:55:42.736264 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:42.736157    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/55d97eb9-77d5-4b56-ab6d-5f38df8d1deb-istiod-ca-cert" pod "55d97eb9-77d5-4b56-ab6d-5f38df8d1deb" (UID: "55d97eb9-77d5-4b56-ab6d-5f38df8d1deb"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 00:55:42.736442 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:42.736421    4284 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/55d97eb9-77d5-4b56-ab6d-5f38df8d1deb-istio-data\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:55:42.736442 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:42.736448    4284 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/55d97eb9-77d5-4b56-ab6d-5f38df8d1deb-workload-socket\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:55:42.736442 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:42.736463    4284 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/55d97eb9-77d5-4b56-ab6d-5f38df8d1deb-workload-certs\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:55:42.736442 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:42.736476    4284 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b4556139-5a27-42e5-ae33-a85c75ea1209-istio-envoy\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:55:42.736842 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:42.736490    4284 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/55d97eb9-77d5-4b56-ab6d-5f38df8d1deb-credential-socket\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:55:42.736842 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:42.736503    4284 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/55d97eb9-77d5-4b56-ab6d-5f38df8d1deb-istiod-ca-cert\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:55:42.736842 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:42.736516    4284 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b4556139-5a27-42e5-ae33-a85c75ea1209-istio-podinfo\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:55:42.736842 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:42.736530    4284 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8fm2w\" (UniqueName: \"kubernetes.io/projected/b4556139-5a27-42e5-ae33-a85c75ea1209-kube-api-access-8fm2w\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:55:42.736842 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:42.736544    4284 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b4556139-5a27-42e5-ae33-a85c75ea1209-istio-token\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:55:42.737949 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:42.737922    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/55d97eb9-77d5-4b56-ab6d-5f38df8d1deb-kube-api-access-tcs2z" pod "55d97eb9-77d5-4b56-ab6d-5f38df8d1deb" (UID: "55d97eb9-77d5-4b56-ab6d-5f38df8d1deb"). InnerVolumeSpecName "kube-api-access-tcs2z". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 00:55:42.738179 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:42.738150    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/55d97eb9-77d5-4b56-ab6d-5f38df8d1deb-istio-envoy" pod "55d97eb9-77d5-4b56-ab6d-5f38df8d1deb" (UID: "55d97eb9-77d5-4b56-ab6d-5f38df8d1deb"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:55:42.738560 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:42.738523    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/55d97eb9-77d5-4b56-ab6d-5f38df8d1deb-istio-token" pod "55d97eb9-77d5-4b56-ab6d-5f38df8d1deb" (UID: "55d97eb9-77d5-4b56-ab6d-5f38df8d1deb"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 00:55:42.739122 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:42.739102    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/55d97eb9-77d5-4b56-ab6d-5f38df8d1deb-istio-podinfo" pod "55d97eb9-77d5-4b56-ab6d-5f38df8d1deb" (UID: "55d97eb9-77d5-4b56-ab6d-5f38df8d1deb"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 00:55:42.843298 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:42.838501    4284 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/55d97eb9-77d5-4b56-ab6d-5f38df8d1deb-istio-envoy\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:55:42.843298 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:42.838538    4284 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/55d97eb9-77d5-4b56-ab6d-5f38df8d1deb-istio-token\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:55:42.843298 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:42.838549    4284 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/55d97eb9-77d5-4b56-ab6d-5f38df8d1deb-istio-podinfo\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:55:42.843298 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:42.838560    4284 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tcs2z\" (UniqueName: \"kubernetes.io/projected/55d97eb9-77d5-4b56-ab6d-5f38df8d1deb-kube-api-access-tcs2z\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:55:43.278143 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:43.278108    4284 generic.go:356] "Generic (PLEG): container finished" podID="55d97eb9-77d5-4b56-ab6d-5f38df8d1deb" containerID="6a9eaf70cfdeeb456778b08b524ea4131bb75313e5c24c3ece3e1836171e1cf1" exitCode=0
Oct 31 00:55:43.278650 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:43.278167    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-fnnxh"
Oct 31 00:55:43.278650 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:43.278196    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-fnnxh" event={"ID":"55d97eb9-77d5-4b56-ab6d-5f38df8d1deb","Type":"ContainerDied","Data":"6a9eaf70cfdeeb456778b08b524ea4131bb75313e5c24c3ece3e1836171e1cf1"}
Oct 31 00:55:43.278650 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:43.278264    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-fnnxh" event={"ID":"55d97eb9-77d5-4b56-ab6d-5f38df8d1deb","Type":"ContainerDied","Data":"e0d370dec52bea0fcdae88cdf18b1c7968191a889668d4c510a58ca2469f7327"}
Oct 31 00:55:43.278650 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:43.278287    4284 scope.go:122] "RemoveContainer" containerID="6a9eaf70cfdeeb456778b08b524ea4131bb75313e5c24c3ece3e1836171e1cf1"
Oct 31 00:55:43.279674 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:43.279649    4284 generic.go:356] "Generic (PLEG): container finished" podID="b4556139-5a27-42e5-ae33-a85c75ea1209" containerID="a9db7c1234f6aab814b5b9e8b16f3cce82a0afe028a1a1bc819e2eaa9ac6475d" exitCode=0
Oct 31 00:55:43.279788 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:43.279702    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-7n64p"
Oct 31 00:55:43.279788 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:43.279741    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-7n64p" event={"ID":"b4556139-5a27-42e5-ae33-a85c75ea1209","Type":"ContainerDied","Data":"a9db7c1234f6aab814b5b9e8b16f3cce82a0afe028a1a1bc819e2eaa9ac6475d"}
Oct 31 00:55:43.279883 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:43.279786    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-7n64p" event={"ID":"b4556139-5a27-42e5-ae33-a85c75ea1209","Type":"ContainerDied","Data":"64ff255d7e887ceeba407a9c89a95a7bee9944b5b1622792f6b24cbeb2cdc87f"}
Oct 31 00:55:43.288956 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:43.288892    4284 scope.go:122] "RemoveContainer" containerID="6a9eaf70cfdeeb456778b08b524ea4131bb75313e5c24c3ece3e1836171e1cf1"
Oct 31 00:55:43.289485 ip-10-0-132-173 kubenswrapper[4284]: E1031 00:55:43.289186    4284 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6a9eaf70cfdeeb456778b08b524ea4131bb75313e5c24c3ece3e1836171e1cf1\": container with ID starting with 6a9eaf70cfdeeb456778b08b524ea4131bb75313e5c24c3ece3e1836171e1cf1 not found: ID does not exist" containerID="6a9eaf70cfdeeb456778b08b524ea4131bb75313e5c24c3ece3e1836171e1cf1"
Oct 31 00:55:43.289485 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:43.289245    4284 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6a9eaf70cfdeeb456778b08b524ea4131bb75313e5c24c3ece3e1836171e1cf1"} err="failed to get container status \"6a9eaf70cfdeeb456778b08b524ea4131bb75313e5c24c3ece3e1836171e1cf1\": rpc error: code = NotFound desc = could not find container \"6a9eaf70cfdeeb456778b08b524ea4131bb75313e5c24c3ece3e1836171e1cf1\": container with ID starting with 6a9eaf70cfdeeb456778b08b524ea4131bb75313e5c24c3ece3e1836171e1cf1 not found: ID does not exist"
Oct 31 00:55:43.289485 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:43.289266    4284 scope.go:122] "RemoveContainer" containerID="a9db7c1234f6aab814b5b9e8b16f3cce82a0afe028a1a1bc819e2eaa9ac6475d"
Oct 31 00:55:43.311972 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:43.311953    4284 scope.go:122] "RemoveContainer" containerID="a9db7c1234f6aab814b5b9e8b16f3cce82a0afe028a1a1bc819e2eaa9ac6475d"
Oct 31 00:55:43.312378 ip-10-0-132-173 kubenswrapper[4284]: E1031 00:55:43.312355    4284 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a9db7c1234f6aab814b5b9e8b16f3cce82a0afe028a1a1bc819e2eaa9ac6475d\": container with ID starting with a9db7c1234f6aab814b5b9e8b16f3cce82a0afe028a1a1bc819e2eaa9ac6475d not found: ID does not exist" containerID="a9db7c1234f6aab814b5b9e8b16f3cce82a0afe028a1a1bc819e2eaa9ac6475d"
Oct 31 00:55:43.312437 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:43.312385    4284 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a9db7c1234f6aab814b5b9e8b16f3cce82a0afe028a1a1bc819e2eaa9ac6475d"} err="failed to get container status \"a9db7c1234f6aab814b5b9e8b16f3cce82a0afe028a1a1bc819e2eaa9ac6475d\": rpc error: code = NotFound desc = could not find container \"a9db7c1234f6aab814b5b9e8b16f3cce82a0afe028a1a1bc819e2eaa9ac6475d\": container with ID starting with a9db7c1234f6aab814b5b9e8b16f3cce82a0afe028a1a1bc819e2eaa9ac6475d not found: ID does not exist"
Oct 31 00:55:43.440088 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:43.440058    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-7n64p"]
Oct 31 00:55:43.487672 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:43.487639    4284 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-7n64p"]
Oct 31 00:55:43.594310 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:43.594228    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-fnnxh"]
Oct 31 00:55:43.599054 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:43.599031    4284 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-fnnxh"]
Oct 31 00:55:43.964010 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:43.963979    4284 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="55d97eb9-77d5-4b56-ab6d-5f38df8d1deb" path="/var/lib/kubelet/pods/55d97eb9-77d5-4b56-ab6d-5f38df8d1deb/volumes"
Oct 31 00:55:43.964465 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:43.964450    4284 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b4556139-5a27-42e5-ae33-a85c75ea1209" path="/var/lib/kubelet/pods/b4556139-5a27-42e5-ae33-a85c75ea1209/volumes"
Oct 31 00:55:44.196967 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:44.196932    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:55:47.014905 ip-10-0-132-173 kubenswrapper[4284]: E1031 00:55:47.014824    4284 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fd8sp" containerName="extractor"
Oct 31 00:55:49.196836 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:49.196803    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:55:53.645659 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:53.645622    4284 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-f66180-5c5d986cbf-7hvkv"]
Oct 31 00:55:53.646262 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:53.646241    4284 cpu_manager.go:414] "containerMap: removing container" podUID="55d97eb9-77d5-4b56-ab6d-5f38df8d1deb" containerName="istio-proxy"
Oct 31 00:55:53.646262 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:53.646261    4284 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="55d97eb9-77d5-4b56-ab6d-5f38df8d1deb" containerName="istio-proxy"
Oct 31 00:55:53.646399 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:53.646294    4284 cpu_manager.go:414] "containerMap: removing container" podUID="b4556139-5a27-42e5-ae33-a85c75ea1209" containerName="istio-proxy"
Oct 31 00:55:53.646399 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:53.646299    4284 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b4556139-5a27-42e5-ae33-a85c75ea1209" containerName="istio-proxy"
Oct 31 00:55:53.646399 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:53.646395    4284 memory_manager.go:365] "RemoveStaleState removing state" podUID="b4556139-5a27-42e5-ae33-a85c75ea1209" containerName="istio-proxy"
Oct 31 00:55:53.646492 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:53.646402    4284 memory_manager.go:365] "RemoveStaleState removing state" podUID="55d97eb9-77d5-4b56-ab6d-5f38df8d1deb" containerName="istio-proxy"
Oct 31 00:55:53.651388 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:53.651371    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-f66180-5c5d986cbf-7hvkv"
Oct 31 00:55:53.702355 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:53.702328    4284 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-a-f66180-serving-cert\""
Oct 31 00:55:53.753318 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:53.753277    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-f66180-5c5d986cbf-7hvkv"]
Oct 31 00:55:53.779287 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:53.779252    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d9d70c1c-9fbd-48cc-b100-24fe6c5836d8-metrics-certs\") pod \"maas-api-e2e-ipp-a-f66180-5c5d986cbf-7hvkv\" (UID: \"d9d70c1c-9fbd-48cc-b100-24fe6c5836d8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-f66180-5c5d986cbf-7hvkv"
Oct 31 00:55:53.779287 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:53.779296    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rgd29\" (UniqueName: \"kubernetes.io/projected/d9d70c1c-9fbd-48cc-b100-24fe6c5836d8-kube-api-access-rgd29\") pod \"maas-api-e2e-ipp-a-f66180-5c5d986cbf-7hvkv\" (UID: \"d9d70c1c-9fbd-48cc-b100-24fe6c5836d8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-f66180-5c5d986cbf-7hvkv"
Oct 31 00:55:53.779555 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:53.779457    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d9d70c1c-9fbd-48cc-b100-24fe6c5836d8-maas-api-tls\") pod \"maas-api-e2e-ipp-a-f66180-5c5d986cbf-7hvkv\" (UID: \"d9d70c1c-9fbd-48cc-b100-24fe6c5836d8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-f66180-5c5d986cbf-7hvkv"
Oct 31 00:55:53.779555 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:53.779531    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d9d70c1c-9fbd-48cc-b100-24fe6c5836d8-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-f66180-5c5d986cbf-7hvkv\" (UID: \"d9d70c1c-9fbd-48cc-b100-24fe6c5836d8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-f66180-5c5d986cbf-7hvkv"
Oct 31 00:55:53.834271 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:53.834198    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:55:53.881306 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:53.881278    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d9d70c1c-9fbd-48cc-b100-24fe6c5836d8-maas-api-tls\") pod \"maas-api-e2e-ipp-a-f66180-5c5d986cbf-7hvkv\" (UID: \"d9d70c1c-9fbd-48cc-b100-24fe6c5836d8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-f66180-5c5d986cbf-7hvkv"
Oct 31 00:55:53.881549 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:53.881356    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d9d70c1c-9fbd-48cc-b100-24fe6c5836d8-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-f66180-5c5d986cbf-7hvkv\" (UID: \"d9d70c1c-9fbd-48cc-b100-24fe6c5836d8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-f66180-5c5d986cbf-7hvkv"
Oct 31 00:55:53.881603 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:53.881555    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d9d70c1c-9fbd-48cc-b100-24fe6c5836d8-metrics-certs\") pod \"maas-api-e2e-ipp-a-f66180-5c5d986cbf-7hvkv\" (UID: \"d9d70c1c-9fbd-48cc-b100-24fe6c5836d8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-f66180-5c5d986cbf-7hvkv"
Oct 31 00:55:53.881603 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:53.881575    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rgd29\" (UniqueName: \"kubernetes.io/projected/d9d70c1c-9fbd-48cc-b100-24fe6c5836d8-kube-api-access-rgd29\") pod \"maas-api-e2e-ipp-a-f66180-5c5d986cbf-7hvkv\" (UID: \"d9d70c1c-9fbd-48cc-b100-24fe6c5836d8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-f66180-5c5d986cbf-7hvkv"
Oct 31 00:55:53.882172 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:53.882099    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d9d70c1c-9fbd-48cc-b100-24fe6c5836d8-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-f66180-5c5d986cbf-7hvkv\" (UID: \"d9d70c1c-9fbd-48cc-b100-24fe6c5836d8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-f66180-5c5d986cbf-7hvkv"
Oct 31 00:55:53.883937 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:53.883914    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d9d70c1c-9fbd-48cc-b100-24fe6c5836d8-maas-api-tls\") pod \"maas-api-e2e-ipp-a-f66180-5c5d986cbf-7hvkv\" (UID: \"d9d70c1c-9fbd-48cc-b100-24fe6c5836d8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-f66180-5c5d986cbf-7hvkv"
Oct 31 00:55:53.884028 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:53.883973    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d9d70c1c-9fbd-48cc-b100-24fe6c5836d8-metrics-certs\") pod \"maas-api-e2e-ipp-a-f66180-5c5d986cbf-7hvkv\" (UID: \"d9d70c1c-9fbd-48cc-b100-24fe6c5836d8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-f66180-5c5d986cbf-7hvkv"
Oct 31 00:55:53.941459 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:53.941426    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rgd29\" (UniqueName: \"kubernetes.io/projected/d9d70c1c-9fbd-48cc-b100-24fe6c5836d8-kube-api-access-rgd29\") pod \"maas-api-e2e-ipp-a-f66180-5c5d986cbf-7hvkv\" (UID: \"d9d70c1c-9fbd-48cc-b100-24fe6c5836d8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-f66180-5c5d986cbf-7hvkv"
Oct 31 00:55:53.964256 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:53.964201    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-f66180-5c5d986cbf-7hvkv"
Oct 31 00:55:54.002663 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:54.002624    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:55:54.157738 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:54.155777    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:55:54.342975 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:54.342948    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-f66180-5c5d986cbf-7hvkv"]
Oct 31 00:55:54.343300 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:55:54.343280    4284 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd9d70c1c_9fbd_48cc_b100_24fe6c5836d8.slice/crio-7d141c3ff2d4ea3725d935322fe39eebb1a9bac95701688826b8900e9d4007ea WatchSource:0}: Error finding container 7d141c3ff2d4ea3725d935322fe39eebb1a9bac95701688826b8900e9d4007ea: Status 404 returned error can't find the container with id 7d141c3ff2d4ea3725d935322fe39eebb1a9bac95701688826b8900e9d4007ea
Oct 31 00:55:54.376552 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:54.376515    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-f66180-5c5d986cbf-7hvkv" event={"ID":"d9d70c1c-9fbd-48cc-b100-24fe6c5836d8","Type":"ContainerStarted","Data":"7d141c3ff2d4ea3725d935322fe39eebb1a9bac95701688826b8900e9d4007ea"}
Oct 31 00:55:56.390141 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:56.390106    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-f66180-5c5d986cbf-7hvkv" event={"ID":"d9d70c1c-9fbd-48cc-b100-24fe6c5836d8","Type":"ContainerStarted","Data":"eaf3f53d12259dc8b6bf69802db0f4fb268754d0af80ebfc94131eb51c101afc"}
Oct 31 00:55:56.390665 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:56.390249    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-f66180-5c5d986cbf-7hvkv"
Oct 31 00:55:56.504008 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:56.503956    4284 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-f66180-5c5d986cbf-7hvkv" podStartSLOduration=2.5310723680000002 podStartE2EDuration="3.503941599s" totalImagesPullingTime="972.869231ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 00:55:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:55:56.503168944 +0000 UTC m=+1670.648175859" watchObservedRunningTime="2026-10-31 00:55:56.503941599 +0000 UTC m=+1670.648948538"
Oct 31 00:55:58.228911 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:58.228877    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:55:58.801205 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:55:58.801163    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:56:02.414345 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:02.414310    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-f66180-5c5d986cbf-7hvkv"
Oct 31 00:56:03.737817 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:03.737775    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:56:08.594985 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:08.594944    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:56:13.368229 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:13.368172    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:56:17.014048 ip-10-0-132-173 kubenswrapper[4284]: E1031 00:56:17.013980    4284 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fd8sp" containerName="extractor"
Oct 31 00:56:18.129088 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:18.129058    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:56:21.844477 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:21.844440    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:56:22.533090 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:22.533055    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-ac952a-676dc686ff-lzbkn"]
Oct 31 00:56:22.649775 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:22.649740    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-ac952a-66d6cb8df5-hdzd6"]
Oct 31 00:56:22.856759 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:22.856663    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:56:23.018851 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:23.018362    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:56:23.195588 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:23.195556    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:56:23.375513 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:23.375482    4284 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-b-cb2b8a-openshift-default-75dd976c57-4jgr7"]
Oct 31 00:56:23.384134 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:23.384110    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-cb2b8a-openshift-default-75dd976c57-4jgr7"
Oct 31 00:56:23.460296 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:23.451152    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-cb2b8a-openshift-default-75dd976c57-4jgr7"]
Oct 31 00:56:23.460296 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:23.452078    4284 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-b-cb2b8a-openshift-default-dockercfg-x2cvw\""
Oct 31 00:56:23.484245 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:23.482898    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:56:23.529921 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:23.529878    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d999d6fd-4e88-49eb-b8ac-f9c1e03baa4c-istio-data\") pod \"e2e-ipp-b-cb2b8a-openshift-default-75dd976c57-4jgr7\" (UID: \"d999d6fd-4e88-49eb-b8ac-f9c1e03baa4c\") " pod="openshift-ingress/e2e-ipp-b-cb2b8a-openshift-default-75dd976c57-4jgr7"
Oct 31 00:56:23.530315 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:23.530290    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d999d6fd-4e88-49eb-b8ac-f9c1e03baa4c-credential-socket\") pod \"e2e-ipp-b-cb2b8a-openshift-default-75dd976c57-4jgr7\" (UID: \"d999d6fd-4e88-49eb-b8ac-f9c1e03baa4c\") " pod="openshift-ingress/e2e-ipp-b-cb2b8a-openshift-default-75dd976c57-4jgr7"
Oct 31 00:56:23.530491 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:23.530472    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d999d6fd-4e88-49eb-b8ac-f9c1e03baa4c-istio-envoy\") pod \"e2e-ipp-b-cb2b8a-openshift-default-75dd976c57-4jgr7\" (UID: \"d999d6fd-4e88-49eb-b8ac-f9c1e03baa4c\") " pod="openshift-ingress/e2e-ipp-b-cb2b8a-openshift-default-75dd976c57-4jgr7"
Oct 31 00:56:23.530720 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:23.530703    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d999d6fd-4e88-49eb-b8ac-f9c1e03baa4c-workload-certs\") pod \"e2e-ipp-b-cb2b8a-openshift-default-75dd976c57-4jgr7\" (UID: \"d999d6fd-4e88-49eb-b8ac-f9c1e03baa4c\") " pod="openshift-ingress/e2e-ipp-b-cb2b8a-openshift-default-75dd976c57-4jgr7"
Oct 31 00:56:23.530838 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:23.530825    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d999d6fd-4e88-49eb-b8ac-f9c1e03baa4c-istio-podinfo\") pod \"e2e-ipp-b-cb2b8a-openshift-default-75dd976c57-4jgr7\" (UID: \"d999d6fd-4e88-49eb-b8ac-f9c1e03baa4c\") " pod="openshift-ingress/e2e-ipp-b-cb2b8a-openshift-default-75dd976c57-4jgr7"
Oct 31 00:56:23.531044 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:23.531025    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mqbkt\" (UniqueName: \"kubernetes.io/projected/d999d6fd-4e88-49eb-b8ac-f9c1e03baa4c-kube-api-access-mqbkt\") pod \"e2e-ipp-b-cb2b8a-openshift-default-75dd976c57-4jgr7\" (UID: \"d999d6fd-4e88-49eb-b8ac-f9c1e03baa4c\") " pod="openshift-ingress/e2e-ipp-b-cb2b8a-openshift-default-75dd976c57-4jgr7"
Oct 31 00:56:23.531202 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:23.531186    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d999d6fd-4e88-49eb-b8ac-f9c1e03baa4c-workload-socket\") pod \"e2e-ipp-b-cb2b8a-openshift-default-75dd976c57-4jgr7\" (UID: \"d999d6fd-4e88-49eb-b8ac-f9c1e03baa4c\") " pod="openshift-ingress/e2e-ipp-b-cb2b8a-openshift-default-75dd976c57-4jgr7"
Oct 31 00:56:23.531389 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:23.531369    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d999d6fd-4e88-49eb-b8ac-f9c1e03baa4c-istio-token\") pod \"e2e-ipp-b-cb2b8a-openshift-default-75dd976c57-4jgr7\" (UID: \"d999d6fd-4e88-49eb-b8ac-f9c1e03baa4c\") " pod="openshift-ingress/e2e-ipp-b-cb2b8a-openshift-default-75dd976c57-4jgr7"
Oct 31 00:56:23.531551 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:23.531524    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d999d6fd-4e88-49eb-b8ac-f9c1e03baa4c-istiod-ca-cert\") pod \"e2e-ipp-b-cb2b8a-openshift-default-75dd976c57-4jgr7\" (UID: \"d999d6fd-4e88-49eb-b8ac-f9c1e03baa4c\") " pod="openshift-ingress/e2e-ipp-b-cb2b8a-openshift-default-75dd976c57-4jgr7"
Oct 31 00:56:23.665528 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:23.635297    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d999d6fd-4e88-49eb-b8ac-f9c1e03baa4c-istio-envoy\") pod \"e2e-ipp-b-cb2b8a-openshift-default-75dd976c57-4jgr7\" (UID: \"d999d6fd-4e88-49eb-b8ac-f9c1e03baa4c\") " pod="openshift-ingress/e2e-ipp-b-cb2b8a-openshift-default-75dd976c57-4jgr7"
Oct 31 00:56:23.665528 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:23.635399    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d999d6fd-4e88-49eb-b8ac-f9c1e03baa4c-workload-certs\") pod \"e2e-ipp-b-cb2b8a-openshift-default-75dd976c57-4jgr7\" (UID: \"d999d6fd-4e88-49eb-b8ac-f9c1e03baa4c\") " pod="openshift-ingress/e2e-ipp-b-cb2b8a-openshift-default-75dd976c57-4jgr7"
Oct 31 00:56:23.665528 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:23.635428    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d999d6fd-4e88-49eb-b8ac-f9c1e03baa4c-istio-podinfo\") pod \"e2e-ipp-b-cb2b8a-openshift-default-75dd976c57-4jgr7\" (UID: \"d999d6fd-4e88-49eb-b8ac-f9c1e03baa4c\") " pod="openshift-ingress/e2e-ipp-b-cb2b8a-openshift-default-75dd976c57-4jgr7"
Oct 31 00:56:23.665528 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:23.635534    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mqbkt\" (UniqueName: \"kubernetes.io/projected/d999d6fd-4e88-49eb-b8ac-f9c1e03baa4c-kube-api-access-mqbkt\") pod \"e2e-ipp-b-cb2b8a-openshift-default-75dd976c57-4jgr7\" (UID: \"d999d6fd-4e88-49eb-b8ac-f9c1e03baa4c\") " pod="openshift-ingress/e2e-ipp-b-cb2b8a-openshift-default-75dd976c57-4jgr7"
Oct 31 00:56:23.665528 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:23.635593    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d999d6fd-4e88-49eb-b8ac-f9c1e03baa4c-workload-socket\") pod \"e2e-ipp-b-cb2b8a-openshift-default-75dd976c57-4jgr7\" (UID: \"d999d6fd-4e88-49eb-b8ac-f9c1e03baa4c\") " pod="openshift-ingress/e2e-ipp-b-cb2b8a-openshift-default-75dd976c57-4jgr7"
Oct 31 00:56:23.665528 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:23.635643    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d999d6fd-4e88-49eb-b8ac-f9c1e03baa4c-istio-token\") pod \"e2e-ipp-b-cb2b8a-openshift-default-75dd976c57-4jgr7\" (UID: \"d999d6fd-4e88-49eb-b8ac-f9c1e03baa4c\") " pod="openshift-ingress/e2e-ipp-b-cb2b8a-openshift-default-75dd976c57-4jgr7"
Oct 31 00:56:23.665528 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:23.635675    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d999d6fd-4e88-49eb-b8ac-f9c1e03baa4c-istiod-ca-cert\") pod \"e2e-ipp-b-cb2b8a-openshift-default-75dd976c57-4jgr7\" (UID: \"d999d6fd-4e88-49eb-b8ac-f9c1e03baa4c\") " pod="openshift-ingress/e2e-ipp-b-cb2b8a-openshift-default-75dd976c57-4jgr7"
Oct 31 00:56:23.665528 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:23.635712    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d999d6fd-4e88-49eb-b8ac-f9c1e03baa4c-istio-data\") pod \"e2e-ipp-b-cb2b8a-openshift-default-75dd976c57-4jgr7\" (UID: \"d999d6fd-4e88-49eb-b8ac-f9c1e03baa4c\") " pod="openshift-ingress/e2e-ipp-b-cb2b8a-openshift-default-75dd976c57-4jgr7"
Oct 31 00:56:23.665528 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:23.635837    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d999d6fd-4e88-49eb-b8ac-f9c1e03baa4c-credential-socket\") pod \"e2e-ipp-b-cb2b8a-openshift-default-75dd976c57-4jgr7\" (UID: \"d999d6fd-4e88-49eb-b8ac-f9c1e03baa4c\") " pod="openshift-ingress/e2e-ipp-b-cb2b8a-openshift-default-75dd976c57-4jgr7"
Oct 31 00:56:23.665528 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:23.636321    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d999d6fd-4e88-49eb-b8ac-f9c1e03baa4c-credential-socket\") pod \"e2e-ipp-b-cb2b8a-openshift-default-75dd976c57-4jgr7\" (UID: \"d999d6fd-4e88-49eb-b8ac-f9c1e03baa4c\") " pod="openshift-ingress/e2e-ipp-b-cb2b8a-openshift-default-75dd976c57-4jgr7"
Oct 31 00:56:23.665528 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:23.657277    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d999d6fd-4e88-49eb-b8ac-f9c1e03baa4c-istio-envoy\") pod \"e2e-ipp-b-cb2b8a-openshift-default-75dd976c57-4jgr7\" (UID: \"d999d6fd-4e88-49eb-b8ac-f9c1e03baa4c\") " pod="openshift-ingress/e2e-ipp-b-cb2b8a-openshift-default-75dd976c57-4jgr7"
Oct 31 00:56:23.665528 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:23.657546    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d999d6fd-4e88-49eb-b8ac-f9c1e03baa4c-workload-certs\") pod \"e2e-ipp-b-cb2b8a-openshift-default-75dd976c57-4jgr7\" (UID: \"d999d6fd-4e88-49eb-b8ac-f9c1e03baa4c\") " pod="openshift-ingress/e2e-ipp-b-cb2b8a-openshift-default-75dd976c57-4jgr7"
Oct 31 00:56:23.673390 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:23.671978    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d999d6fd-4e88-49eb-b8ac-f9c1e03baa4c-workload-socket\") pod \"e2e-ipp-b-cb2b8a-openshift-default-75dd976c57-4jgr7\" (UID: \"d999d6fd-4e88-49eb-b8ac-f9c1e03baa4c\") " pod="openshift-ingress/e2e-ipp-b-cb2b8a-openshift-default-75dd976c57-4jgr7"
Oct 31 00:56:23.673390 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:23.672660    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d999d6fd-4e88-49eb-b8ac-f9c1e03baa4c-istiod-ca-cert\") pod \"e2e-ipp-b-cb2b8a-openshift-default-75dd976c57-4jgr7\" (UID: \"d999d6fd-4e88-49eb-b8ac-f9c1e03baa4c\") " pod="openshift-ingress/e2e-ipp-b-cb2b8a-openshift-default-75dd976c57-4jgr7"
Oct 31 00:56:23.673390 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:23.672918    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d999d6fd-4e88-49eb-b8ac-f9c1e03baa4c-istio-data\") pod \"e2e-ipp-b-cb2b8a-openshift-default-75dd976c57-4jgr7\" (UID: \"d999d6fd-4e88-49eb-b8ac-f9c1e03baa4c\") " pod="openshift-ingress/e2e-ipp-b-cb2b8a-openshift-default-75dd976c57-4jgr7"
Oct 31 00:56:23.699089 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:23.699026    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d999d6fd-4e88-49eb-b8ac-f9c1e03baa4c-istio-podinfo\") pod \"e2e-ipp-b-cb2b8a-openshift-default-75dd976c57-4jgr7\" (UID: \"d999d6fd-4e88-49eb-b8ac-f9c1e03baa4c\") " pod="openshift-ingress/e2e-ipp-b-cb2b8a-openshift-default-75dd976c57-4jgr7"
Oct 31 00:56:23.730084 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:23.730014    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d999d6fd-4e88-49eb-b8ac-f9c1e03baa4c-istio-token\") pod \"e2e-ipp-b-cb2b8a-openshift-default-75dd976c57-4jgr7\" (UID: \"d999d6fd-4e88-49eb-b8ac-f9c1e03baa4c\") " pod="openshift-ingress/e2e-ipp-b-cb2b8a-openshift-default-75dd976c57-4jgr7"
Oct 31 00:56:23.730282 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:23.730256    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mqbkt\" (UniqueName: \"kubernetes.io/projected/d999d6fd-4e88-49eb-b8ac-f9c1e03baa4c-kube-api-access-mqbkt\") pod \"e2e-ipp-b-cb2b8a-openshift-default-75dd976c57-4jgr7\" (UID: \"d999d6fd-4e88-49eb-b8ac-f9c1e03baa4c\") " pod="openshift-ingress/e2e-ipp-b-cb2b8a-openshift-default-75dd976c57-4jgr7"
Oct 31 00:56:23.891541 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:23.891431    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:56:24.017901 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:24.017820    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-cb2b8a-openshift-default-75dd976c57-4jgr7"
Oct 31 00:56:24.469479 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:56:24.460333    4284 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd999d6fd_4e88_49eb_b8ac_f9c1e03baa4c.slice/crio-c994076e13d596bf71e6cc04e075ebce889bdfc41e8914381fe964883d1fb440 WatchSource:0}: Error finding container c994076e13d596bf71e6cc04e075ebce889bdfc41e8914381fe964883d1fb440: Status 404 returned error can't find the container with id c994076e13d596bf71e6cc04e075ebce889bdfc41e8914381fe964883d1fb440
Oct 31 00:56:24.469479 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:24.462152    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-cb2b8a-openshift-default-75dd976c57-4jgr7"]
Oct 31 00:56:24.474401 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:24.474364    4284 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 00:56:24.474516 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:24.474475    4284 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 00:56:24.474579 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:24.474523    4284 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 00:56:24.591249 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:24.589344    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-cb2b8a-openshift-default-75dd976c57-4jgr7" event={"ID":"d999d6fd-4e88-49eb-b8ac-f9c1e03baa4c","Type":"ContainerStarted","Data":"c994076e13d596bf71e6cc04e075ebce889bdfc41e8914381fe964883d1fb440"}
Oct 31 00:56:25.597887 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:25.597852    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-cb2b8a-openshift-default-75dd976c57-4jgr7" event={"ID":"d999d6fd-4e88-49eb-b8ac-f9c1e03baa4c","Type":"ContainerStarted","Data":"37925e0229115d194189405ddd0fef06de3719e8b611f08a2d0a091046aaa467"}
Oct 31 00:56:25.811141 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:25.811091    4284 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-b-cb2b8a-openshift-default-75dd976c57-4jgr7" podStartSLOduration=2.811076029 podStartE2EDuration="2.811076029s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 00:56:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:56:25.762035076 +0000 UTC m=+1699.907041991" watchObservedRunningTime="2026-10-31 00:56:25.811076029 +0000 UTC m=+1699.956082945"
Oct 31 00:56:26.018248 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:26.018190    4284 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-cb2b8a-openshift-default-75dd976c57-4jgr7"
Oct 31 00:56:26.024333 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:26.024307    4284 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-cb2b8a-openshift-default-75dd976c57-4jgr7"
Oct 31 00:56:26.603995 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:26.603963    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-cb2b8a-openshift-default-75dd976c57-4jgr7"
Oct 31 00:56:26.604847 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:26.604829    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-cb2b8a-openshift-default-75dd976c57-4jgr7"
Oct 31 00:56:27.533835 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:27.533792    4284 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-a-ac952a-676dc686ff-lzbkn" podUID="5870504f-57b8-4e98-b62b-e67c472d45b7" containerName="payload-processing" containerID="cri-o://ce5c26b95967b2813f87287af2f9c6c7d612a5fc9df21785590aa875118708f4" gracePeriod=25
Oct 31 00:56:27.650981 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:27.650941    4284 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-ac952a-66d6cb8df5-hdzd6" podUID="dc6cbe1a-0138-4aa3-92fa-6b80803d3abb" containerName="payload-pre-processing" containerID="cri-o://ea123e617c3773f314cd03285d3641bb3e84b442dfb3bf95b562aa0ddcca6062" gracePeriod=25
Oct 31 00:56:28.001394 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:28.001366    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-ac952a-676dc686ff-lzbkn"
Oct 31 00:56:28.053407 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:28.051870    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5870504f-57b8-4e98-b62b-e67c472d45b7-kube-api-access-nhmw5\" (UniqueName: \"kubernetes.io/projected/5870504f-57b8-4e98-b62b-e67c472d45b7-kube-api-access-nhmw5\") pod \"5870504f-57b8-4e98-b62b-e67c472d45b7\" (UID: \"5870504f-57b8-4e98-b62b-e67c472d45b7\") "
Oct 31 00:56:28.053407 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:28.051930    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/5870504f-57b8-4e98-b62b-e67c472d45b7-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5870504f-57b8-4e98-b62b-e67c472d45b7-plugins-config-volume\") pod \"5870504f-57b8-4e98-b62b-e67c472d45b7\" (UID: \"5870504f-57b8-4e98-b62b-e67c472d45b7\") "
Oct 31 00:56:28.053407 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:28.052452    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5870504f-57b8-4e98-b62b-e67c472d45b7-plugins-config-volume" pod "5870504f-57b8-4e98-b62b-e67c472d45b7" (UID: "5870504f-57b8-4e98-b62b-e67c472d45b7"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 00:56:28.055257 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:28.054416    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5870504f-57b8-4e98-b62b-e67c472d45b7-kube-api-access-nhmw5" pod "5870504f-57b8-4e98-b62b-e67c472d45b7" (UID: "5870504f-57b8-4e98-b62b-e67c472d45b7"). InnerVolumeSpecName "kube-api-access-nhmw5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 00:56:28.148398 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:28.148366    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-ac952a-66d6cb8df5-hdzd6"
Oct 31 00:56:28.153363 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:28.153342    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/dc6cbe1a-0138-4aa3-92fa-6b80803d3abb-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dc6cbe1a-0138-4aa3-92fa-6b80803d3abb-plugins-config-volume\") pod \"dc6cbe1a-0138-4aa3-92fa-6b80803d3abb\" (UID: \"dc6cbe1a-0138-4aa3-92fa-6b80803d3abb\") "
Oct 31 00:56:28.153477 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:28.153409    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/dc6cbe1a-0138-4aa3-92fa-6b80803d3abb-kube-api-access-wd47t\" (UniqueName: \"kubernetes.io/projected/dc6cbe1a-0138-4aa3-92fa-6b80803d3abb-kube-api-access-wd47t\") pod \"dc6cbe1a-0138-4aa3-92fa-6b80803d3abb\" (UID: \"dc6cbe1a-0138-4aa3-92fa-6b80803d3abb\") "
Oct 31 00:56:28.153678 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:28.153656    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dc6cbe1a-0138-4aa3-92fa-6b80803d3abb-plugins-config-volume" pod "dc6cbe1a-0138-4aa3-92fa-6b80803d3abb" (UID: "dc6cbe1a-0138-4aa3-92fa-6b80803d3abb"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 00:56:28.153800 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:28.153786    4284 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5870504f-57b8-4e98-b62b-e67c472d45b7-plugins-config-volume\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:56:28.153842 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:28.153803    4284 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dc6cbe1a-0138-4aa3-92fa-6b80803d3abb-plugins-config-volume\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:56:28.153842 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:28.153812    4284 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nhmw5\" (UniqueName: \"kubernetes.io/projected/5870504f-57b8-4e98-b62b-e67c472d45b7-kube-api-access-nhmw5\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:56:28.155493 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:28.155464    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dc6cbe1a-0138-4aa3-92fa-6b80803d3abb-kube-api-access-wd47t" pod "dc6cbe1a-0138-4aa3-92fa-6b80803d3abb" (UID: "dc6cbe1a-0138-4aa3-92fa-6b80803d3abb"). InnerVolumeSpecName "kube-api-access-wd47t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 00:56:28.254622 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:28.254551    4284 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wd47t\" (UniqueName: \"kubernetes.io/projected/dc6cbe1a-0138-4aa3-92fa-6b80803d3abb-kube-api-access-wd47t\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:56:28.617627 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:28.617593    4284 generic.go:356] "Generic (PLEG): container finished" podID="dc6cbe1a-0138-4aa3-92fa-6b80803d3abb" containerID="ea123e617c3773f314cd03285d3641bb3e84b442dfb3bf95b562aa0ddcca6062" exitCode=0
Oct 31 00:56:28.617886 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:28.617640    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-ac952a-66d6cb8df5-hdzd6"
Oct 31 00:56:28.617886 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:28.617678    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-ac952a-66d6cb8df5-hdzd6" event={"ID":"dc6cbe1a-0138-4aa3-92fa-6b80803d3abb","Type":"ContainerDied","Data":"ea123e617c3773f314cd03285d3641bb3e84b442dfb3bf95b562aa0ddcca6062"}
Oct 31 00:56:28.617886 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:28.617757    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-ac952a-66d6cb8df5-hdzd6" event={"ID":"dc6cbe1a-0138-4aa3-92fa-6b80803d3abb","Type":"ContainerDied","Data":"a177de9cc732dfb9a695895f699cb39ce394fd760100fe643f7b5895fe3fbb1b"}
Oct 31 00:56:28.617886 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:28.617787    4284 scope.go:122] "RemoveContainer" containerID="ea123e617c3773f314cd03285d3641bb3e84b442dfb3bf95b562aa0ddcca6062"
Oct 31 00:56:28.619682 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:28.619656    4284 generic.go:356] "Generic (PLEG): container finished" podID="5870504f-57b8-4e98-b62b-e67c472d45b7" containerID="ce5c26b95967b2813f87287af2f9c6c7d612a5fc9df21785590aa875118708f4" exitCode=0
Oct 31 00:56:28.619819 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:28.619717    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-ac952a-676dc686ff-lzbkn"
Oct 31 00:56:28.619819 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:28.619725    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-ac952a-676dc686ff-lzbkn" event={"ID":"5870504f-57b8-4e98-b62b-e67c472d45b7","Type":"ContainerDied","Data":"ce5c26b95967b2813f87287af2f9c6c7d612a5fc9df21785590aa875118708f4"}
Oct 31 00:56:28.619819 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:28.619769    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-ac952a-676dc686ff-lzbkn" event={"ID":"5870504f-57b8-4e98-b62b-e67c472d45b7","Type":"ContainerDied","Data":"798778a4066cd9fe4982b2daefaa522add737b8e5b6d2e09b1bef7dfce1532ab"}
Oct 31 00:56:28.627989 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:28.627973    4284 scope.go:122] "RemoveContainer" containerID="ea123e617c3773f314cd03285d3641bb3e84b442dfb3bf95b562aa0ddcca6062"
Oct 31 00:56:28.628338 ip-10-0-132-173 kubenswrapper[4284]: E1031 00:56:28.628315    4284 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ea123e617c3773f314cd03285d3641bb3e84b442dfb3bf95b562aa0ddcca6062\": container with ID starting with ea123e617c3773f314cd03285d3641bb3e84b442dfb3bf95b562aa0ddcca6062 not found: ID does not exist" containerID="ea123e617c3773f314cd03285d3641bb3e84b442dfb3bf95b562aa0ddcca6062"
Oct 31 00:56:28.628407 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:28.628347    4284 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ea123e617c3773f314cd03285d3641bb3e84b442dfb3bf95b562aa0ddcca6062"} err="failed to get container status \"ea123e617c3773f314cd03285d3641bb3e84b442dfb3bf95b562aa0ddcca6062\": rpc error: code = NotFound desc = could not find container \"ea123e617c3773f314cd03285d3641bb3e84b442dfb3bf95b562aa0ddcca6062\": container with ID starting with ea123e617c3773f314cd03285d3641bb3e84b442dfb3bf95b562aa0ddcca6062 not found: ID does not exist"
Oct 31 00:56:28.628407 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:28.628369    4284 scope.go:122] "RemoveContainer" containerID="ce5c26b95967b2813f87287af2f9c6c7d612a5fc9df21785590aa875118708f4"
Oct 31 00:56:28.637609 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:28.637591    4284 scope.go:122] "RemoveContainer" containerID="ce5c26b95967b2813f87287af2f9c6c7d612a5fc9df21785590aa875118708f4"
Oct 31 00:56:28.637863 ip-10-0-132-173 kubenswrapper[4284]: E1031 00:56:28.637848    4284 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ce5c26b95967b2813f87287af2f9c6c7d612a5fc9df21785590aa875118708f4\": container with ID starting with ce5c26b95967b2813f87287af2f9c6c7d612a5fc9df21785590aa875118708f4 not found: ID does not exist" containerID="ce5c26b95967b2813f87287af2f9c6c7d612a5fc9df21785590aa875118708f4"
Oct 31 00:56:28.637902 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:28.637871    4284 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ce5c26b95967b2813f87287af2f9c6c7d612a5fc9df21785590aa875118708f4"} err="failed to get container status \"ce5c26b95967b2813f87287af2f9c6c7d612a5fc9df21785590aa875118708f4\": rpc error: code = NotFound desc = could not find container \"ce5c26b95967b2813f87287af2f9c6c7d612a5fc9df21785590aa875118708f4\": container with ID starting with ce5c26b95967b2813f87287af2f9c6c7d612a5fc9df21785590aa875118708f4 not found: ID does not exist"
Oct 31 00:56:28.647698 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:28.647676    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:56:28.798690 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:28.797884    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-ac952a-676dc686ff-lzbkn"]
Oct 31 00:56:28.804532 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:28.804035    4284 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-ac952a-676dc686ff-lzbkn"]
Oct 31 00:56:28.898846 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:28.898813    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-ac952a-66d6cb8df5-hdzd6"]
Oct 31 00:56:28.904739 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:28.904712    4284 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-ac952a-66d6cb8df5-hdzd6"]
Oct 31 00:56:29.004037 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:29.004000    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:56:29.201723 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:29.201685    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:56:29.963410 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:29.963381    4284 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5870504f-57b8-4e98-b62b-e67c472d45b7" path="/var/lib/kubelet/pods/5870504f-57b8-4e98-b62b-e67c472d45b7/volumes"
Oct 31 00:56:29.963804 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:29.963769    4284 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="dc6cbe1a-0138-4aa3-92fa-6b80803d3abb" path="/var/lib/kubelet/pods/dc6cbe1a-0138-4aa3-92fa-6b80803d3abb/volumes"
Oct 31 00:56:33.104180 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:33.104145    4284 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7289-da8265"]
Oct 31 00:56:33.105776 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:33.105748    4284 cpu_manager.go:414] "containerMap: removing container" podUID="5870504f-57b8-4e98-b62b-e67c472d45b7" containerName="payload-processing"
Oct 31 00:56:33.105938 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:33.105924    4284 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5870504f-57b8-4e98-b62b-e67c472d45b7" containerName="payload-processing"
Oct 31 00:56:33.110285 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:33.110262    4284 cpu_manager.go:414] "containerMap: removing container" podUID="dc6cbe1a-0138-4aa3-92fa-6b80803d3abb" containerName="payload-pre-processing"
Oct 31 00:56:33.110427 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:33.110413    4284 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="dc6cbe1a-0138-4aa3-92fa-6b80803d3abb" containerName="payload-pre-processing"
Oct 31 00:56:33.110667 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:33.110653    4284 memory_manager.go:365] "RemoveStaleState removing state" podUID="5870504f-57b8-4e98-b62b-e67c472d45b7" containerName="payload-processing"
Oct 31 00:56:33.110761 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:33.110749    4284 memory_manager.go:365] "RemoveStaleState removing state" podUID="dc6cbe1a-0138-4aa3-92fa-6b80803d3abb" containerName="payload-pre-processing"
Oct 31 00:56:33.116428 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:33.116413    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7289-da8265"
Oct 31 00:56:33.131644 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:33.131623    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rxhll\" (UniqueName: \"kubernetes.io/projected/b18eddc1-2928-4890-82a5-48116f781941-kube-api-access-rxhll\") pod \"test-curl-7289-da8265\" (UID: \"b18eddc1-2928-4890-82a5-48116f781941\") " pod="opendatahub/test-curl-7289-da8265"
Oct 31 00:56:33.143535 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:33.143511    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7289-da8265"]
Oct 31 00:56:33.167236 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:33.167182    4284 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-8gxx2\""
Oct 31 00:56:33.233096 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:33.233059    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rxhll\" (UniqueName: \"kubernetes.io/projected/b18eddc1-2928-4890-82a5-48116f781941-kube-api-access-rxhll\") pod \"test-curl-7289-da8265\" (UID: \"b18eddc1-2928-4890-82a5-48116f781941\") " pod="opendatahub/test-curl-7289-da8265"
Oct 31 00:56:33.309841 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:33.309808    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rxhll\" (UniqueName: \"kubernetes.io/projected/b18eddc1-2928-4890-82a5-48116f781941-kube-api-access-rxhll\") pod \"test-curl-7289-da8265\" (UID: \"b18eddc1-2928-4890-82a5-48116f781941\") " pod="opendatahub/test-curl-7289-da8265"
Oct 31 00:56:33.429019 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:33.428985    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7289-da8265"
Oct 31 00:56:33.797703 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:33.797552    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:56:33.841235 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:33.834934    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7289-da8265"]
Oct 31 00:56:33.855019 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:56:33.854985    4284 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb18eddc1_2928_4890_82a5_48116f781941.slice/crio-c1289d9a0e534c363e585a72571bbf5a1bdb67f3458907a72faa54fd4f4e4e68 WatchSource:0}: Error finding container c1289d9a0e534c363e585a72571bbf5a1bdb67f3458907a72faa54fd4f4e4e68: Status 404 returned error can't find the container with id c1289d9a0e534c363e585a72571bbf5a1bdb67f3458907a72faa54fd4f4e4e68
Oct 31 00:56:33.857145 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:33.857125    4284 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 31 00:56:34.645134 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:34.645093    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:56:34.659098 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:34.659062    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7289-da8265" event={"ID":"b18eddc1-2928-4890-82a5-48116f781941","Type":"ContainerStarted","Data":"c1289d9a0e534c363e585a72571bbf5a1bdb67f3458907a72faa54fd4f4e4e68"}
Oct 31 00:56:34.904025 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:34.903995    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:56:34.934875 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:34.934842    4284 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-7e6fe88b-openshift-default-5b64b75c89-vtxl5"]
Oct 31 00:56:34.944615 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:34.944587    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-7e6fe88b-openshift-default-5b64b75c89-vtxl5"
Oct 31 00:56:35.060474 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:35.060443    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-7e6fe88b-openshift-default-5b64b75c89-vtxl5"]
Oct 31 00:56:35.060898 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:35.060876    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c381f046-a134-41bd-8e86-ce8a9d580cb9-istio-data\") pod \"e2e-mt-7e6fe88b-openshift-default-5b64b75c89-vtxl5\" (UID: \"c381f046-a134-41bd-8e86-ce8a9d580cb9\") " pod="openshift-ingress/e2e-mt-7e6fe88b-openshift-default-5b64b75c89-vtxl5"
Oct 31 00:56:35.060976 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:35.060962    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c381f046-a134-41bd-8e86-ce8a9d580cb9-istiod-ca-cert\") pod \"e2e-mt-7e6fe88b-openshift-default-5b64b75c89-vtxl5\" (UID: \"c381f046-a134-41bd-8e86-ce8a9d580cb9\") " pod="openshift-ingress/e2e-mt-7e6fe88b-openshift-default-5b64b75c89-vtxl5"
Oct 31 00:56:35.061022 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:35.061011    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c381f046-a134-41bd-8e86-ce8a9d580cb9-workload-socket\") pod \"e2e-mt-7e6fe88b-openshift-default-5b64b75c89-vtxl5\" (UID: \"c381f046-a134-41bd-8e86-ce8a9d580cb9\") " pod="openshift-ingress/e2e-mt-7e6fe88b-openshift-default-5b64b75c89-vtxl5"
Oct 31 00:56:35.061096 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:35.061084    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c381f046-a134-41bd-8e86-ce8a9d580cb9-istio-token\") pod \"e2e-mt-7e6fe88b-openshift-default-5b64b75c89-vtxl5\" (UID: \"c381f046-a134-41bd-8e86-ce8a9d580cb9\") " pod="openshift-ingress/e2e-mt-7e6fe88b-openshift-default-5b64b75c89-vtxl5"
Oct 31 00:56:35.061265 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:35.061250    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c381f046-a134-41bd-8e86-ce8a9d580cb9-credential-socket\") pod \"e2e-mt-7e6fe88b-openshift-default-5b64b75c89-vtxl5\" (UID: \"c381f046-a134-41bd-8e86-ce8a9d580cb9\") " pod="openshift-ingress/e2e-mt-7e6fe88b-openshift-default-5b64b75c89-vtxl5"
Oct 31 00:56:35.061308 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:35.061279    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zxk4p\" (UniqueName: \"kubernetes.io/projected/c381f046-a134-41bd-8e86-ce8a9d580cb9-kube-api-access-zxk4p\") pod \"e2e-mt-7e6fe88b-openshift-default-5b64b75c89-vtxl5\" (UID: \"c381f046-a134-41bd-8e86-ce8a9d580cb9\") " pod="openshift-ingress/e2e-mt-7e6fe88b-openshift-default-5b64b75c89-vtxl5"
Oct 31 00:56:35.061401 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:35.061383    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c381f046-a134-41bd-8e86-ce8a9d580cb9-workload-certs\") pod \"e2e-mt-7e6fe88b-openshift-default-5b64b75c89-vtxl5\" (UID: \"c381f046-a134-41bd-8e86-ce8a9d580cb9\") " pod="openshift-ingress/e2e-mt-7e6fe88b-openshift-default-5b64b75c89-vtxl5"
Oct 31 00:56:35.061451 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:35.061439    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c381f046-a134-41bd-8e86-ce8a9d580cb9-istio-podinfo\") pod \"e2e-mt-7e6fe88b-openshift-default-5b64b75c89-vtxl5\" (UID: \"c381f046-a134-41bd-8e86-ce8a9d580cb9\") " pod="openshift-ingress/e2e-mt-7e6fe88b-openshift-default-5b64b75c89-vtxl5"
Oct 31 00:56:35.061498 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:35.061462    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c381f046-a134-41bd-8e86-ce8a9d580cb9-istio-envoy\") pod \"e2e-mt-7e6fe88b-openshift-default-5b64b75c89-vtxl5\" (UID: \"c381f046-a134-41bd-8e86-ce8a9d580cb9\") " pod="openshift-ingress/e2e-mt-7e6fe88b-openshift-default-5b64b75c89-vtxl5"
Oct 31 00:56:35.163272 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:35.163184    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c381f046-a134-41bd-8e86-ce8a9d580cb9-workload-socket\") pod \"e2e-mt-7e6fe88b-openshift-default-5b64b75c89-vtxl5\" (UID: \"c381f046-a134-41bd-8e86-ce8a9d580cb9\") " pod="openshift-ingress/e2e-mt-7e6fe88b-openshift-default-5b64b75c89-vtxl5"
Oct 31 00:56:35.163272 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:35.163247    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c381f046-a134-41bd-8e86-ce8a9d580cb9-istio-token\") pod \"e2e-mt-7e6fe88b-openshift-default-5b64b75c89-vtxl5\" (UID: \"c381f046-a134-41bd-8e86-ce8a9d580cb9\") " pod="openshift-ingress/e2e-mt-7e6fe88b-openshift-default-5b64b75c89-vtxl5"
Oct 31 00:56:35.163452 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:35.163340    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c381f046-a134-41bd-8e86-ce8a9d580cb9-credential-socket\") pod \"e2e-mt-7e6fe88b-openshift-default-5b64b75c89-vtxl5\" (UID: \"c381f046-a134-41bd-8e86-ce8a9d580cb9\") " pod="openshift-ingress/e2e-mt-7e6fe88b-openshift-default-5b64b75c89-vtxl5"
Oct 31 00:56:35.163452 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:35.163362    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zxk4p\" (UniqueName: \"kubernetes.io/projected/c381f046-a134-41bd-8e86-ce8a9d580cb9-kube-api-access-zxk4p\") pod \"e2e-mt-7e6fe88b-openshift-default-5b64b75c89-vtxl5\" (UID: \"c381f046-a134-41bd-8e86-ce8a9d580cb9\") " pod="openshift-ingress/e2e-mt-7e6fe88b-openshift-default-5b64b75c89-vtxl5"
Oct 31 00:56:35.163452 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:35.163402    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c381f046-a134-41bd-8e86-ce8a9d580cb9-workload-certs\") pod \"e2e-mt-7e6fe88b-openshift-default-5b64b75c89-vtxl5\" (UID: \"c381f046-a134-41bd-8e86-ce8a9d580cb9\") " pod="openshift-ingress/e2e-mt-7e6fe88b-openshift-default-5b64b75c89-vtxl5"
Oct 31 00:56:35.163452 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:35.163442    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c381f046-a134-41bd-8e86-ce8a9d580cb9-istio-podinfo\") pod \"e2e-mt-7e6fe88b-openshift-default-5b64b75c89-vtxl5\" (UID: \"c381f046-a134-41bd-8e86-ce8a9d580cb9\") " pod="openshift-ingress/e2e-mt-7e6fe88b-openshift-default-5b64b75c89-vtxl5"
Oct 31 00:56:35.163611 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:35.163461    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c381f046-a134-41bd-8e86-ce8a9d580cb9-istio-envoy\") pod \"e2e-mt-7e6fe88b-openshift-default-5b64b75c89-vtxl5\" (UID: \"c381f046-a134-41bd-8e86-ce8a9d580cb9\") " pod="openshift-ingress/e2e-mt-7e6fe88b-openshift-default-5b64b75c89-vtxl5"
Oct 31 00:56:35.163611 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:35.163494    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c381f046-a134-41bd-8e86-ce8a9d580cb9-istio-data\") pod \"e2e-mt-7e6fe88b-openshift-default-5b64b75c89-vtxl5\" (UID: \"c381f046-a134-41bd-8e86-ce8a9d580cb9\") " pod="openshift-ingress/e2e-mt-7e6fe88b-openshift-default-5b64b75c89-vtxl5"
Oct 31 00:56:35.163611 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:35.163562    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c381f046-a134-41bd-8e86-ce8a9d580cb9-istiod-ca-cert\") pod \"e2e-mt-7e6fe88b-openshift-default-5b64b75c89-vtxl5\" (UID: \"c381f046-a134-41bd-8e86-ce8a9d580cb9\") " pod="openshift-ingress/e2e-mt-7e6fe88b-openshift-default-5b64b75c89-vtxl5"
Oct 31 00:56:35.163731 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:35.163657    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c381f046-a134-41bd-8e86-ce8a9d580cb9-workload-socket\") pod \"e2e-mt-7e6fe88b-openshift-default-5b64b75c89-vtxl5\" (UID: \"c381f046-a134-41bd-8e86-ce8a9d580cb9\") " pod="openshift-ingress/e2e-mt-7e6fe88b-openshift-default-5b64b75c89-vtxl5"
Oct 31 00:56:35.164293 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:35.164044    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c381f046-a134-41bd-8e86-ce8a9d580cb9-workload-certs\") pod \"e2e-mt-7e6fe88b-openshift-default-5b64b75c89-vtxl5\" (UID: \"c381f046-a134-41bd-8e86-ce8a9d580cb9\") " pod="openshift-ingress/e2e-mt-7e6fe88b-openshift-default-5b64b75c89-vtxl5"
Oct 31 00:56:35.164293 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:35.164122    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c381f046-a134-41bd-8e86-ce8a9d580cb9-istiod-ca-cert\") pod \"e2e-mt-7e6fe88b-openshift-default-5b64b75c89-vtxl5\" (UID: \"c381f046-a134-41bd-8e86-ce8a9d580cb9\") " pod="openshift-ingress/e2e-mt-7e6fe88b-openshift-default-5b64b75c89-vtxl5"
Oct 31 00:56:35.164759 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:35.164654    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c381f046-a134-41bd-8e86-ce8a9d580cb9-credential-socket\") pod \"e2e-mt-7e6fe88b-openshift-default-5b64b75c89-vtxl5\" (UID: \"c381f046-a134-41bd-8e86-ce8a9d580cb9\") " pod="openshift-ingress/e2e-mt-7e6fe88b-openshift-default-5b64b75c89-vtxl5"
Oct 31 00:56:35.164990 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:35.164971    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c381f046-a134-41bd-8e86-ce8a9d580cb9-istio-data\") pod \"e2e-mt-7e6fe88b-openshift-default-5b64b75c89-vtxl5\" (UID: \"c381f046-a134-41bd-8e86-ce8a9d580cb9\") " pod="openshift-ingress/e2e-mt-7e6fe88b-openshift-default-5b64b75c89-vtxl5"
Oct 31 00:56:35.168238 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:35.167001    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c381f046-a134-41bd-8e86-ce8a9d580cb9-istio-podinfo\") pod \"e2e-mt-7e6fe88b-openshift-default-5b64b75c89-vtxl5\" (UID: \"c381f046-a134-41bd-8e86-ce8a9d580cb9\") " pod="openshift-ingress/e2e-mt-7e6fe88b-openshift-default-5b64b75c89-vtxl5"
Oct 31 00:56:35.168238 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:35.167254    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c381f046-a134-41bd-8e86-ce8a9d580cb9-istio-envoy\") pod \"e2e-mt-7e6fe88b-openshift-default-5b64b75c89-vtxl5\" (UID: \"c381f046-a134-41bd-8e86-ce8a9d580cb9\") " pod="openshift-ingress/e2e-mt-7e6fe88b-openshift-default-5b64b75c89-vtxl5"
Oct 31 00:56:35.258826 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:35.258794    4284 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-cb2b8a-7546789495-5wzmh"]
Oct 31 00:56:35.309860 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:35.309833    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-cb2b8a-7546789495-5wzmh"]
Oct 31 00:56:35.310337 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:35.310318    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-cb2b8a-7546789495-5wzmh"
Oct 31 00:56:35.411289 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:35.378064    4284 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-b-cb2b8a-dockercfg-hfc6w\""
Oct 31 00:56:35.411289 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:35.378901    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zxk4p\" (UniqueName: \"kubernetes.io/projected/c381f046-a134-41bd-8e86-ce8a9d580cb9-kube-api-access-zxk4p\") pod \"e2e-mt-7e6fe88b-openshift-default-5b64b75c89-vtxl5\" (UID: \"c381f046-a134-41bd-8e86-ce8a9d580cb9\") " pod="openshift-ingress/e2e-mt-7e6fe88b-openshift-default-5b64b75c89-vtxl5"
Oct 31 00:56:35.411289 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:35.386589    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a8b1b169-c9ee-4dea-91cc-ec434f44b05c-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-cb2b8a-7546789495-5wzmh\" (UID: \"a8b1b169-c9ee-4dea-91cc-ec434f44b05c\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-cb2b8a-7546789495-5wzmh"
Oct 31 00:56:35.411289 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:35.386647    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-68vx9\" (UniqueName: \"kubernetes.io/projected/a8b1b169-c9ee-4dea-91cc-ec434f44b05c-kube-api-access-68vx9\") pod \"payload-processing-e2e-ipp-b-cb2b8a-7546789495-5wzmh\" (UID: \"a8b1b169-c9ee-4dea-91cc-ec434f44b05c\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-cb2b8a-7546789495-5wzmh"
Oct 31 00:56:35.419088 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:35.418331    4284 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-b-cb2b8a\""
Oct 31 00:56:35.444061 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:35.444002    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c381f046-a134-41bd-8e86-ce8a9d580cb9-istio-token\") pod \"e2e-mt-7e6fe88b-openshift-default-5b64b75c89-vtxl5\" (UID: \"c381f046-a134-41bd-8e86-ce8a9d580cb9\") " pod="openshift-ingress/e2e-mt-7e6fe88b-openshift-default-5b64b75c89-vtxl5"
Oct 31 00:56:35.509837 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:35.506177    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a8b1b169-c9ee-4dea-91cc-ec434f44b05c-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-cb2b8a-7546789495-5wzmh\" (UID: \"a8b1b169-c9ee-4dea-91cc-ec434f44b05c\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-cb2b8a-7546789495-5wzmh"
Oct 31 00:56:35.509837 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:35.509339    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-68vx9\" (UniqueName: \"kubernetes.io/projected/a8b1b169-c9ee-4dea-91cc-ec434f44b05c-kube-api-access-68vx9\") pod \"payload-processing-e2e-ipp-b-cb2b8a-7546789495-5wzmh\" (UID: \"a8b1b169-c9ee-4dea-91cc-ec434f44b05c\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-cb2b8a-7546789495-5wzmh"
Oct 31 00:56:35.510162 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:35.510135    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a8b1b169-c9ee-4dea-91cc-ec434f44b05c-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-cb2b8a-7546789495-5wzmh\" (UID: \"a8b1b169-c9ee-4dea-91cc-ec434f44b05c\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-cb2b8a-7546789495-5wzmh"
Oct 31 00:56:35.562057 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:35.562021    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-7e6fe88b-openshift-default-5b64b75c89-vtxl5"
Oct 31 00:56:35.568604 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:35.568577    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-68vx9\" (UniqueName: \"kubernetes.io/projected/a8b1b169-c9ee-4dea-91cc-ec434f44b05c-kube-api-access-68vx9\") pod \"payload-processing-e2e-ipp-b-cb2b8a-7546789495-5wzmh\" (UID: \"a8b1b169-c9ee-4dea-91cc-ec434f44b05c\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-cb2b8a-7546789495-5wzmh"
Oct 31 00:56:35.750378 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:35.750287    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-cb2b8a-7546789495-5wzmh"
Oct 31 00:56:36.024516 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:36.024486    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-7e6fe88b-openshift-default-5b64b75c89-vtxl5"]
Oct 31 00:56:36.026556 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:56:36.026530    4284 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc381f046_a134_41bd_8e86_ce8a9d580cb9.slice/crio-283124e2462a8d0995cc86fa0d94feb636df52249e95a606f458bb7fc9e35e66 WatchSource:0}: Error finding container 283124e2462a8d0995cc86fa0d94feb636df52249e95a606f458bb7fc9e35e66: Status 404 returned error can't find the container with id 283124e2462a8d0995cc86fa0d94feb636df52249e95a606f458bb7fc9e35e66
Oct 31 00:56:36.029670 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:36.029638    4284 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 00:56:36.029775 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:36.029718    4284 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 00:56:36.035046 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:36.030563    4284 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 00:56:36.167005 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:36.166975    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-cb2b8a-7546789495-5wzmh"]
Oct 31 00:56:36.169197 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:56:36.169171    4284 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda8b1b169_c9ee_4dea_91cc_ec434f44b05c.slice/crio-ea11ed7dd33844ad9119f5fb2096907e378311f3235f0b98c6b584bfde3e0358 WatchSource:0}: Error finding container ea11ed7dd33844ad9119f5fb2096907e378311f3235f0b98c6b584bfde3e0358: Status 404 returned error can't find the container with id ea11ed7dd33844ad9119f5fb2096907e378311f3235f0b98c6b584bfde3e0358
Oct 31 00:56:36.681001 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:36.680875    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-cb2b8a-7546789495-5wzmh" event={"ID":"a8b1b169-c9ee-4dea-91cc-ec434f44b05c","Type":"ContainerStarted","Data":"2a1707f7388819a8fdab42f235ef72a25ac9beb07a2742fdc37309ea21a60f33"}
Oct 31 00:56:36.681001 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:36.680953    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-cb2b8a-7546789495-5wzmh" event={"ID":"a8b1b169-c9ee-4dea-91cc-ec434f44b05c","Type":"ContainerStarted","Data":"ea11ed7dd33844ad9119f5fb2096907e378311f3235f0b98c6b584bfde3e0358"}
Oct 31 00:56:36.681207 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:36.681117    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-cb2b8a-7546789495-5wzmh"
Oct 31 00:56:36.683328 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:36.683298    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-7e6fe88b-openshift-default-5b64b75c89-vtxl5" event={"ID":"c381f046-a134-41bd-8e86-ce8a9d580cb9","Type":"ContainerStarted","Data":"62331e7d0778d8c7d56790518744d30374d6bb135b70a871775015fb520a54aa"}
Oct 31 00:56:36.684730 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:36.683351    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-7e6fe88b-openshift-default-5b64b75c89-vtxl5" event={"ID":"c381f046-a134-41bd-8e86-ce8a9d580cb9","Type":"ContainerStarted","Data":"283124e2462a8d0995cc86fa0d94feb636df52249e95a606f458bb7fc9e35e66"}
Oct 31 00:56:36.794790 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:36.794730    4284 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-b-cb2b8a-7546789495-5wzmh" podStartSLOduration=1.794711129 podStartE2EDuration="1.794711129s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 00:56:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:56:36.793374346 +0000 UTC m=+1710.938381261" watchObservedRunningTime="2026-10-31 00:56:36.794711129 +0000 UTC m=+1710.939718048"
Oct 31 00:56:36.963238 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:36.946947    4284 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-7e6fe88b-openshift-default-5b64b75c89-vtxl5" podStartSLOduration=2.9469256919999998 podStartE2EDuration="2.946925692s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 00:56:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:56:36.924547789 +0000 UTC m=+1711.069554705" watchObservedRunningTime="2026-10-31 00:56:36.946925692 +0000 UTC m=+1711.091932611"
Oct 31 00:56:37.562807 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:37.562773    4284 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-7e6fe88b-openshift-default-5b64b75c89-vtxl5"
Oct 31 00:56:37.564848 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:37.564809    4284 patch_prober.go:29] interesting pod/e2e-mt-7e6fe88b-openshift-default-5b64b75c89-vtxl5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.105:15021/healthz/ready\": dial tcp 10.134.0.105:15021: connect: connection refused" start-of-body=
Oct 31 00:56:37.564998 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:37.564894    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-7e6fe88b-openshift-default-5b64b75c89-vtxl5" podUID="c381f046-a134-41bd-8e86-ce8a9d580cb9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.105:15021/healthz/ready\": dial tcp 10.134.0.105:15021: connect: connection refused"
Oct 31 00:56:38.563195 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:38.563158    4284 patch_prober.go:29] interesting pod/e2e-mt-7e6fe88b-openshift-default-5b64b75c89-vtxl5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.105:15021/healthz/ready\": dial tcp 10.134.0.105:15021: connect: connection refused" start-of-body=
Oct 31 00:56:38.563648 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:38.563246    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-7e6fe88b-openshift-default-5b64b75c89-vtxl5" podUID="c381f046-a134-41bd-8e86-ce8a9d580cb9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.105:15021/healthz/ready\": dial tcp 10.134.0.105:15021: connect: connection refused"
Oct 31 00:56:38.678853 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:38.678813    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:56:38.782821 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:38.782759    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7289-da8265" event={"ID":"b18eddc1-2928-4890-82a5-48116f781941","Type":"ContainerStarted","Data":"d2ba4dcda65e336dba5c8cdc30746a9aea3168370c5b64de63b3b61748dd97b3"}
Oct 31 00:56:38.904287 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:38.903693    4284 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/test-curl-7289-da8265" podStartSLOduration=1.69511417 podStartE2EDuration="5.903673455s" totalImagesPullingTime="4.208559285s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 00:56:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:56:38.902518505 +0000 UTC m=+1713.047525422" watchObservedRunningTime="2026-10-31 00:56:38.903673455 +0000 UTC m=+1713.048680371"
Oct 31 00:56:39.562700 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:39.562650    4284 patch_prober.go:29] interesting pod/e2e-mt-7e6fe88b-openshift-default-5b64b75c89-vtxl5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.105:15021/healthz/ready\": dial tcp 10.134.0.105:15021: connect: connection refused" start-of-body=
Oct 31 00:56:39.562700 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:39.562704    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-7e6fe88b-openshift-default-5b64b75c89-vtxl5" podUID="c381f046-a134-41bd-8e86-ce8a9d580cb9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.105:15021/healthz/ready\": dial tcp 10.134.0.105:15021: connect: connection refused"
Oct 31 00:56:40.537033 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:40.536811    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7289-da8265"]
Oct 31 00:56:40.542352 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:40.541957    4284 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7289-da8265"]
Oct 31 00:56:40.563276 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:40.563243    4284 patch_prober.go:29] interesting pod/e2e-mt-7e6fe88b-openshift-default-5b64b75c89-vtxl5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.105:15021/healthz/ready\": dial tcp 10.134.0.105:15021: connect: connection refused" start-of-body=
Oct 31 00:56:40.563418 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:40.563306    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-7e6fe88b-openshift-default-5b64b75c89-vtxl5" podUID="c381f046-a134-41bd-8e86-ce8a9d580cb9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.105:15021/healthz/ready\": dial tcp 10.134.0.105:15021: connect: connection refused"
Oct 31 00:56:40.823575 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:40.823463    4284 kuberuntime_container.go:898] "Killing container with a grace period" pod="opendatahub/test-curl-7289-da8265" podUID="b18eddc1-2928-4890-82a5-48116f781941" containerName="test-curl-7289-da8265" containerID="cri-o://d2ba4dcda65e336dba5c8cdc30746a9aea3168370c5b64de63b3b61748dd97b3" gracePeriod=2
Oct 31 00:56:40.872583 ip-10-0-132-173 kubenswrapper[4284]: E1031 00:56:40.872529    4284 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-7289-da8265\" is forbidden: User \"system:node:ip-10-0-132-173.us-west-2.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-132-173.us-west-2.compute.internal' and this object" podUID="b18eddc1-2928-4890-82a5-48116f781941" pod="opendatahub/test-curl-7289-da8265"
Oct 31 00:56:41.562835 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:41.562801    4284 patch_prober.go:29] interesting pod/e2e-mt-7e6fe88b-openshift-default-5b64b75c89-vtxl5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.105:15021/healthz/ready\": dial tcp 10.134.0.105:15021: connect: connection refused" start-of-body=
Oct 31 00:56:41.563290 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:41.562849    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-7e6fe88b-openshift-default-5b64b75c89-vtxl5" podUID="c381f046-a134-41bd-8e86-ce8a9d580cb9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.105:15021/healthz/ready\": dial tcp 10.134.0.105:15021: connect: connection refused"
Oct 31 00:56:41.830582 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:41.830511    4284 generic.go:356] "Generic (PLEG): container finished" podID="1e029b42-c66c-4ef0-b6e4-0a5ef5670758" containerID="75bee10e954827824a90ad4ba61a1a6327708e401bb1c1759bff7b703d745e5d" exitCode=0
Oct 31 00:56:41.830724 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:41.830586    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-f66180-openshift-default-55cfc8577c-lfq7v" event={"ID":"1e029b42-c66c-4ef0-b6e4-0a5ef5670758","Type":"ContainerDied","Data":"75bee10e954827824a90ad4ba61a1a6327708e401bb1c1759bff7b703d745e5d"}
Oct 31 00:56:41.831032 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:41.831019    4284 scope.go:122] "RemoveContainer" containerID="75bee10e954827824a90ad4ba61a1a6327708e401bb1c1759bff7b703d745e5d"
Oct 31 00:56:41.832567 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:41.832537    4284 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 00:56:41.832705 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:41.832619    4284 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 00:56:41.832705 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:41.832667    4284 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 00:56:41.879699 ip-10-0-132-173 kubenswrapper[4284]: E1031 00:56:41.879667    4284 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-7289-da8265\" is forbidden: User \"system:node:ip-10-0-132-173.us-west-2.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-132-173.us-west-2.compute.internal' and this object" podUID="b18eddc1-2928-4890-82a5-48116f781941" pod="opendatahub/test-curl-7289-da8265"
Oct 31 00:56:42.562869 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:42.562829    4284 patch_prober.go:29] interesting pod/e2e-mt-7e6fe88b-openshift-default-5b64b75c89-vtxl5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.105:15021/healthz/ready\": dial tcp 10.134.0.105:15021: connect: connection refused" start-of-body=
Oct 31 00:56:42.563571 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:42.562893    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-7e6fe88b-openshift-default-5b64b75c89-vtxl5" podUID="c381f046-a134-41bd-8e86-ce8a9d580cb9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.105:15021/healthz/ready\": dial tcp 10.134.0.105:15021: connect: connection refused"
Oct 31 00:56:42.855377 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:42.855339    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-f66180-openshift-default-55cfc8577c-lfq7v" event={"ID":"1e029b42-c66c-4ef0-b6e4-0a5ef5670758","Type":"ContainerStarted","Data":"cd8e6d6347680c0a4421384a11472c5da9bcacbaafb9153b701402aaa2e28f6e"}
Oct 31 00:56:43.078723 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:43.078399    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:56:43.163175 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:43.163153    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7289-da8265"
Oct 31 00:56:43.257237 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:43.257083    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b18eddc1-2928-4890-82a5-48116f781941-kube-api-access-rxhll\" (UniqueName: \"kubernetes.io/projected/b18eddc1-2928-4890-82a5-48116f781941-kube-api-access-rxhll\") pod \"b18eddc1-2928-4890-82a5-48116f781941\" (UID: \"b18eddc1-2928-4890-82a5-48116f781941\") "
Oct 31 00:56:43.302236 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:43.299537    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b18eddc1-2928-4890-82a5-48116f781941-kube-api-access-rxhll" pod "b18eddc1-2928-4890-82a5-48116f781941" (UID: "b18eddc1-2928-4890-82a5-48116f781941"). InnerVolumeSpecName "kube-api-access-rxhll". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 00:56:43.358717 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:43.358644    4284 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rxhll\" (UniqueName: \"kubernetes.io/projected/b18eddc1-2928-4890-82a5-48116f781941-kube-api-access-rxhll\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:56:43.622350 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:43.604532    4284 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-7e6fe88b-openshift-default-5b64b75c89-vtxl5"
Oct 31 00:56:43.622350 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:43.614112    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-7e6fe88b-openshift-default-5b64b75c89-vtxl5"
Oct 31 00:56:43.640653 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:43.640519    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-7e6fe88b-openshift-default-5b64b75c89-vtxl5"
Oct 31 00:56:43.805809 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:43.805675    4284 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-f66180-openshift-default-55cfc8577c-lfq7v"
Oct 31 00:56:43.806137 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:43.806110    4284 patch_prober.go:29] interesting pod/e2e-ipp-a-f66180-openshift-default-55cfc8577c-lfq7v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.97:15021/healthz/ready\": dial tcp 10.134.0.97:15021: connect: connection refused" start-of-body=
Oct 31 00:56:43.806348 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:43.806321    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-f66180-openshift-default-55cfc8577c-lfq7v" podUID="1e029b42-c66c-4ef0-b6e4-0a5ef5670758" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.97:15021/healthz/ready\": dial tcp 10.134.0.97:15021: connect: connection refused"
Oct 31 00:56:43.867237 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:43.866642    4284 generic.go:356] "Generic (PLEG): container finished" podID="b18eddc1-2928-4890-82a5-48116f781941" containerID="d2ba4dcda65e336dba5c8cdc30746a9aea3168370c5b64de63b3b61748dd97b3" exitCode=137
Oct 31 00:56:43.867237 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:43.866704    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7289-da8265"
Oct 31 00:56:43.867237 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:43.866792    4284 scope.go:122] "RemoveContainer" containerID="d2ba4dcda65e336dba5c8cdc30746a9aea3168370c5b64de63b3b61748dd97b3"
Oct 31 00:56:43.893467 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:43.881869    4284 scope.go:122] "RemoveContainer" containerID="d2ba4dcda65e336dba5c8cdc30746a9aea3168370c5b64de63b3b61748dd97b3"
Oct 31 00:56:43.893467 ip-10-0-132-173 kubenswrapper[4284]: E1031 00:56:43.882203    4284 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d2ba4dcda65e336dba5c8cdc30746a9aea3168370c5b64de63b3b61748dd97b3\": container with ID starting with d2ba4dcda65e336dba5c8cdc30746a9aea3168370c5b64de63b3b61748dd97b3 not found: ID does not exist" containerID="d2ba4dcda65e336dba5c8cdc30746a9aea3168370c5b64de63b3b61748dd97b3"
Oct 31 00:56:43.893467 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:43.882280    4284 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d2ba4dcda65e336dba5c8cdc30746a9aea3168370c5b64de63b3b61748dd97b3"} err="failed to get container status \"d2ba4dcda65e336dba5c8cdc30746a9aea3168370c5b64de63b3b61748dd97b3\": rpc error: code = NotFound desc = could not find container \"d2ba4dcda65e336dba5c8cdc30746a9aea3168370c5b64de63b3b61748dd97b3\": container with ID starting with d2ba4dcda65e336dba5c8cdc30746a9aea3168370c5b64de63b3b61748dd97b3 not found: ID does not exist"
Oct 31 00:56:43.927121 ip-10-0-132-173 kubenswrapper[4284]: E1031 00:56:43.927085    4284 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-7289-da8265\" is forbidden: User \"system:node:ip-10-0-132-173.us-west-2.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-132-173.us-west-2.compute.internal' and this object" podUID="b18eddc1-2928-4890-82a5-48116f781941" pod="opendatahub/test-curl-7289-da8265"
Oct 31 00:56:43.965020 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:43.964986    4284 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b18eddc1-2928-4890-82a5-48116f781941" path="/var/lib/kubelet/pods/b18eddc1-2928-4890-82a5-48116f781941/volumes"
Oct 31 00:56:44.245701 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:44.245670    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-3c636f-586fcffd55-lxv6t"]
Oct 31 00:56:44.245880 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:44.245860    4284 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-3c636f-586fcffd55-lxv6t" podUID="250cb265-61b0-4cb7-aac7-1b9a0f46c783" containerName="maas-api" containerID="cri-o://af9cccb3cb693970e7ce6b540b8e456f82d5f520d1e0a2208e9e6cd11001823b" gracePeriod=30
Oct 31 00:56:44.338897 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:44.338861    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:56:44.757419 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:44.753396    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:56:44.810458 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:44.809676    4284 patch_prober.go:29] interesting pod/e2e-ipp-a-f66180-openshift-default-55cfc8577c-lfq7v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.97:15021/healthz/ready\": dial tcp 10.134.0.97:15021: connect: connection refused" start-of-body=
Oct 31 00:56:44.810458 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:44.809750    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-f66180-openshift-default-55cfc8577c-lfq7v" podUID="1e029b42-c66c-4ef0-b6e4-0a5ef5670758" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.97:15021/healthz/ready\": dial tcp 10.134.0.97:15021: connect: connection refused"
Oct 31 00:56:44.817188 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:44.817166    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-3c636f-586fcffd55-lxv6t"
Oct 31 00:56:44.875364 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:44.875323    4284 generic.go:356] "Generic (PLEG): container finished" podID="250cb265-61b0-4cb7-aac7-1b9a0f46c783" containerID="af9cccb3cb693970e7ce6b540b8e456f82d5f520d1e0a2208e9e6cd11001823b" exitCode=0
Oct 31 00:56:44.875573 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:44.875377    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-3c636f-586fcffd55-lxv6t"
Oct 31 00:56:44.875573 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:44.875404    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-3c636f-586fcffd55-lxv6t" event={"ID":"250cb265-61b0-4cb7-aac7-1b9a0f46c783","Type":"ContainerDied","Data":"af9cccb3cb693970e7ce6b540b8e456f82d5f520d1e0a2208e9e6cd11001823b"}
Oct 31 00:56:44.875573 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:44.875455    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-3c636f-586fcffd55-lxv6t" event={"ID":"250cb265-61b0-4cb7-aac7-1b9a0f46c783","Type":"ContainerDied","Data":"74497a73825e5c4db7407d96e45e93ccbe04bfe5748144a05c5e192474be93a7"}
Oct 31 00:56:44.875573 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:44.875485    4284 scope.go:122] "RemoveContainer" containerID="af9cccb3cb693970e7ce6b540b8e456f82d5f520d1e0a2208e9e6cd11001823b"
Oct 31 00:56:44.903286 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:44.884259    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/250cb265-61b0-4cb7-aac7-1b9a0f46c783-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/250cb265-61b0-4cb7-aac7-1b9a0f46c783-maas-api-tls\") pod \"250cb265-61b0-4cb7-aac7-1b9a0f46c783\" (UID: \"250cb265-61b0-4cb7-aac7-1b9a0f46c783\") "
Oct 31 00:56:44.903286 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:44.884352    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/250cb265-61b0-4cb7-aac7-1b9a0f46c783-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/250cb265-61b0-4cb7-aac7-1b9a0f46c783-openshift-service-ca\") pod \"250cb265-61b0-4cb7-aac7-1b9a0f46c783\" (UID: \"250cb265-61b0-4cb7-aac7-1b9a0f46c783\") "
Oct 31 00:56:44.903286 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:44.884486    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/250cb265-61b0-4cb7-aac7-1b9a0f46c783-metrics-certs\" (UniqueName: \"kubernetes.io/secret/250cb265-61b0-4cb7-aac7-1b9a0f46c783-metrics-certs\") pod \"250cb265-61b0-4cb7-aac7-1b9a0f46c783\" (UID: \"250cb265-61b0-4cb7-aac7-1b9a0f46c783\") "
Oct 31 00:56:44.903286 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:44.884535    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/250cb265-61b0-4cb7-aac7-1b9a0f46c783-kube-api-access-wfz5c\" (UniqueName: \"kubernetes.io/projected/250cb265-61b0-4cb7-aac7-1b9a0f46c783-kube-api-access-wfz5c\") pod \"250cb265-61b0-4cb7-aac7-1b9a0f46c783\" (UID: \"250cb265-61b0-4cb7-aac7-1b9a0f46c783\") "
Oct 31 00:56:44.903286 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:44.895714    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/250cb265-61b0-4cb7-aac7-1b9a0f46c783-openshift-service-ca" pod "250cb265-61b0-4cb7-aac7-1b9a0f46c783" (UID: "250cb265-61b0-4cb7-aac7-1b9a0f46c783"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 00:56:44.912712 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:44.911375    4284 scope.go:122] "RemoveContainer" containerID="af9cccb3cb693970e7ce6b540b8e456f82d5f520d1e0a2208e9e6cd11001823b"
Oct 31 00:56:44.912712 ip-10-0-132-173 kubenswrapper[4284]: E1031 00:56:44.911833    4284 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"af9cccb3cb693970e7ce6b540b8e456f82d5f520d1e0a2208e9e6cd11001823b\": container with ID starting with af9cccb3cb693970e7ce6b540b8e456f82d5f520d1e0a2208e9e6cd11001823b not found: ID does not exist" containerID="af9cccb3cb693970e7ce6b540b8e456f82d5f520d1e0a2208e9e6cd11001823b"
Oct 31 00:56:44.912712 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:44.911870    4284 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"af9cccb3cb693970e7ce6b540b8e456f82d5f520d1e0a2208e9e6cd11001823b"} err="failed to get container status \"af9cccb3cb693970e7ce6b540b8e456f82d5f520d1e0a2208e9e6cd11001823b\": rpc error: code = NotFound desc = could not find container \"af9cccb3cb693970e7ce6b540b8e456f82d5f520d1e0a2208e9e6cd11001823b\": container with ID starting with af9cccb3cb693970e7ce6b540b8e456f82d5f520d1e0a2208e9e6cd11001823b not found: ID does not exist"
Oct 31 00:56:44.912978 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:44.912887    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/250cb265-61b0-4cb7-aac7-1b9a0f46c783-metrics-certs" pod "250cb265-61b0-4cb7-aac7-1b9a0f46c783" (UID: "250cb265-61b0-4cb7-aac7-1b9a0f46c783"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 00:56:44.913519 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:44.913498    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/250cb265-61b0-4cb7-aac7-1b9a0f46c783-maas-api-tls" pod "250cb265-61b0-4cb7-aac7-1b9a0f46c783" (UID: "250cb265-61b0-4cb7-aac7-1b9a0f46c783"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 00:56:44.915482 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:44.915453    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/250cb265-61b0-4cb7-aac7-1b9a0f46c783-kube-api-access-wfz5c" pod "250cb265-61b0-4cb7-aac7-1b9a0f46c783" (UID: "250cb265-61b0-4cb7-aac7-1b9a0f46c783"). InnerVolumeSpecName "kube-api-access-wfz5c". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 00:56:45.011737 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:45.011632    4284 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/250cb265-61b0-4cb7-aac7-1b9a0f46c783-maas-api-tls\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:56:45.011737 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:45.011674    4284 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/250cb265-61b0-4cb7-aac7-1b9a0f46c783-openshift-service-ca\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:56:45.011737 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:45.011690    4284 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/250cb265-61b0-4cb7-aac7-1b9a0f46c783-metrics-certs\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:56:45.011737 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:45.011705    4284 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wfz5c\" (UniqueName: \"kubernetes.io/projected/250cb265-61b0-4cb7-aac7-1b9a0f46c783-kube-api-access-wfz5c\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:56:45.070984 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:45.070950    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:56:45.331289 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:45.329268    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-3c636f-586fcffd55-lxv6t"]
Oct 31 00:56:45.362236 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:45.350495    4284 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-3c636f-586fcffd55-lxv6t"]
Oct 31 00:56:45.710600 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:45.710514    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:56:45.805031 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:45.804602    4284 patch_prober.go:29] interesting pod/e2e-ipp-a-f66180-openshift-default-55cfc8577c-lfq7v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.97:15021/healthz/ready\": dial tcp 10.134.0.97:15021: connect: connection refused" start-of-body=
Oct 31 00:56:45.805031 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:45.804675    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-f66180-openshift-default-55cfc8577c-lfq7v" podUID="1e029b42-c66c-4ef0-b6e4-0a5ef5670758" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.97:15021/healthz/ready\": dial tcp 10.134.0.97:15021: connect: connection refused"
Oct 31 00:56:45.963591 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:45.963510    4284 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="250cb265-61b0-4cb7-aac7-1b9a0f46c783" path="/var/lib/kubelet/pods/250cb265-61b0-4cb7-aac7-1b9a0f46c783/volumes"
Oct 31 00:56:46.119702 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:46.119672    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:56:46.441888 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:46.441855    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:56:46.630354 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:46.630319    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:56:46.805293 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:46.804651    4284 patch_prober.go:29] interesting pod/e2e-ipp-a-f66180-openshift-default-55cfc8577c-lfq7v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.97:15021/healthz/ready\": dial tcp 10.134.0.97:15021: connect: connection refused" start-of-body=
Oct 31 00:56:46.805293 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:46.804716    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-f66180-openshift-default-55cfc8577c-lfq7v" podUID="1e029b42-c66c-4ef0-b6e4-0a5ef5670758" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.97:15021/healthz/ready\": dial tcp 10.134.0.97:15021: connect: connection refused"
Oct 31 00:56:47.037194 ip-10-0-132-173 kubenswrapper[4284]: E1031 00:56:47.037165    4284 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fd8sp" containerName="extractor"
Oct 31 00:56:47.144824 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:47.144734    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:56:47.725318 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:47.725283    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-cb2b8a-7546789495-5wzmh"
Oct 31 00:56:47.807235 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:47.806616    4284 patch_prober.go:29] interesting pod/e2e-ipp-a-f66180-openshift-default-55cfc8577c-lfq7v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.97:15021/healthz/ready\": dial tcp 10.134.0.97:15021: connect: connection refused" start-of-body=
Oct 31 00:56:47.807235 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:47.806680    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-f66180-openshift-default-55cfc8577c-lfq7v" podUID="1e029b42-c66c-4ef0-b6e4-0a5ef5670758" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.97:15021/healthz/ready\": dial tcp 10.134.0.97:15021: connect: connection refused"
Oct 31 00:56:48.808728 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:48.808697    4284 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-f66180-openshift-default-55cfc8577c-lfq7v"
Oct 31 00:56:48.809128 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:48.808994    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-f66180-openshift-default-55cfc8577c-lfq7v"
Oct 31 00:56:48.809645 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:48.809626    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-f66180-openshift-default-55cfc8577c-lfq7v"
Oct 31 00:56:50.041577 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:50.041537    4284 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-7e6fe88b-openshift-default-5b64b75c89-l8cw5"]
Oct 31 00:56:50.042344 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:50.042323    4284 cpu_manager.go:414] "containerMap: removing container" podUID="250cb265-61b0-4cb7-aac7-1b9a0f46c783" containerName="maas-api"
Oct 31 00:56:50.042344 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:50.042343    4284 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="250cb265-61b0-4cb7-aac7-1b9a0f46c783" containerName="maas-api"
Oct 31 00:56:50.042531 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:50.042383    4284 cpu_manager.go:414] "containerMap: removing container" podUID="b18eddc1-2928-4890-82a5-48116f781941" containerName="test-curl-7289-da8265"
Oct 31 00:56:50.042531 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:50.042392    4284 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b18eddc1-2928-4890-82a5-48116f781941" containerName="test-curl-7289-da8265"
Oct 31 00:56:50.042634 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:50.042535    4284 memory_manager.go:365] "RemoveStaleState removing state" podUID="b18eddc1-2928-4890-82a5-48116f781941" containerName="test-curl-7289-da8265"
Oct 31 00:56:50.042634 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:50.042551    4284 memory_manager.go:365] "RemoveStaleState removing state" podUID="250cb265-61b0-4cb7-aac7-1b9a0f46c783" containerName="maas-api"
Oct 31 00:56:50.047669 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:50.047648    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-7e6fe88b-openshift-default-5b64b75c89-l8cw5"
Oct 31 00:56:50.158030 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:50.157989    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vx88b\" (UniqueName: \"kubernetes.io/projected/965d1fab-96f5-400b-bcf8-1be86ab61076-kube-api-access-vx88b\") pod \"e2e-mt-7e6fe88b-openshift-default-5b64b75c89-l8cw5\" (UID: \"965d1fab-96f5-400b-bcf8-1be86ab61076\") " pod="openshift-ingress/e2e-mt-7e6fe88b-openshift-default-5b64b75c89-l8cw5"
Oct 31 00:56:50.158188 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:50.158050    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/965d1fab-96f5-400b-bcf8-1be86ab61076-istiod-ca-cert\") pod \"e2e-mt-7e6fe88b-openshift-default-5b64b75c89-l8cw5\" (UID: \"965d1fab-96f5-400b-bcf8-1be86ab61076\") " pod="openshift-ingress/e2e-mt-7e6fe88b-openshift-default-5b64b75c89-l8cw5"
Oct 31 00:56:50.158283 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:50.158257    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/965d1fab-96f5-400b-bcf8-1be86ab61076-workload-socket\") pod \"e2e-mt-7e6fe88b-openshift-default-5b64b75c89-l8cw5\" (UID: \"965d1fab-96f5-400b-bcf8-1be86ab61076\") " pod="openshift-ingress/e2e-mt-7e6fe88b-openshift-default-5b64b75c89-l8cw5"
Oct 31 00:56:50.158338 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:50.158304    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/965d1fab-96f5-400b-bcf8-1be86ab61076-istio-data\") pod \"e2e-mt-7e6fe88b-openshift-default-5b64b75c89-l8cw5\" (UID: \"965d1fab-96f5-400b-bcf8-1be86ab61076\") " pod="openshift-ingress/e2e-mt-7e6fe88b-openshift-default-5b64b75c89-l8cw5"
Oct 31 00:56:50.158457 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:50.158432    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/965d1fab-96f5-400b-bcf8-1be86ab61076-istio-token\") pod \"e2e-mt-7e6fe88b-openshift-default-5b64b75c89-l8cw5\" (UID: \"965d1fab-96f5-400b-bcf8-1be86ab61076\") " pod="openshift-ingress/e2e-mt-7e6fe88b-openshift-default-5b64b75c89-l8cw5"
Oct 31 00:56:50.158598 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:50.158579    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/965d1fab-96f5-400b-bcf8-1be86ab61076-istio-envoy\") pod \"e2e-mt-7e6fe88b-openshift-default-5b64b75c89-l8cw5\" (UID: \"965d1fab-96f5-400b-bcf8-1be86ab61076\") " pod="openshift-ingress/e2e-mt-7e6fe88b-openshift-default-5b64b75c89-l8cw5"
Oct 31 00:56:50.158644 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:50.158629    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/965d1fab-96f5-400b-bcf8-1be86ab61076-credential-socket\") pod \"e2e-mt-7e6fe88b-openshift-default-5b64b75c89-l8cw5\" (UID: \"965d1fab-96f5-400b-bcf8-1be86ab61076\") " pod="openshift-ingress/e2e-mt-7e6fe88b-openshift-default-5b64b75c89-l8cw5"
Oct 31 00:56:50.158697 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:50.158668    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/965d1fab-96f5-400b-bcf8-1be86ab61076-workload-certs\") pod \"e2e-mt-7e6fe88b-openshift-default-5b64b75c89-l8cw5\" (UID: \"965d1fab-96f5-400b-bcf8-1be86ab61076\") " pod="openshift-ingress/e2e-mt-7e6fe88b-openshift-default-5b64b75c89-l8cw5"
Oct 31 00:56:50.158855 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:50.158829    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/965d1fab-96f5-400b-bcf8-1be86ab61076-istio-podinfo\") pod \"e2e-mt-7e6fe88b-openshift-default-5b64b75c89-l8cw5\" (UID: \"965d1fab-96f5-400b-bcf8-1be86ab61076\") " pod="openshift-ingress/e2e-mt-7e6fe88b-openshift-default-5b64b75c89-l8cw5"
Oct 31 00:56:50.260608 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:50.260569    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/965d1fab-96f5-400b-bcf8-1be86ab61076-workload-socket\") pod \"e2e-mt-7e6fe88b-openshift-default-5b64b75c89-l8cw5\" (UID: \"965d1fab-96f5-400b-bcf8-1be86ab61076\") " pod="openshift-ingress/e2e-mt-7e6fe88b-openshift-default-5b64b75c89-l8cw5"
Oct 31 00:56:50.260608 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:50.260607    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/965d1fab-96f5-400b-bcf8-1be86ab61076-istio-data\") pod \"e2e-mt-7e6fe88b-openshift-default-5b64b75c89-l8cw5\" (UID: \"965d1fab-96f5-400b-bcf8-1be86ab61076\") " pod="openshift-ingress/e2e-mt-7e6fe88b-openshift-default-5b64b75c89-l8cw5"
Oct 31 00:56:50.260854 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:50.260643    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/965d1fab-96f5-400b-bcf8-1be86ab61076-istio-token\") pod \"e2e-mt-7e6fe88b-openshift-default-5b64b75c89-l8cw5\" (UID: \"965d1fab-96f5-400b-bcf8-1be86ab61076\") " pod="openshift-ingress/e2e-mt-7e6fe88b-openshift-default-5b64b75c89-l8cw5"
Oct 31 00:56:50.260854 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:50.260698    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/965d1fab-96f5-400b-bcf8-1be86ab61076-istio-envoy\") pod \"e2e-mt-7e6fe88b-openshift-default-5b64b75c89-l8cw5\" (UID: \"965d1fab-96f5-400b-bcf8-1be86ab61076\") " pod="openshift-ingress/e2e-mt-7e6fe88b-openshift-default-5b64b75c89-l8cw5"
Oct 31 00:56:50.260854 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:50.260731    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/965d1fab-96f5-400b-bcf8-1be86ab61076-credential-socket\") pod \"e2e-mt-7e6fe88b-openshift-default-5b64b75c89-l8cw5\" (UID: \"965d1fab-96f5-400b-bcf8-1be86ab61076\") " pod="openshift-ingress/e2e-mt-7e6fe88b-openshift-default-5b64b75c89-l8cw5"
Oct 31 00:56:50.260854 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:50.260765    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/965d1fab-96f5-400b-bcf8-1be86ab61076-workload-certs\") pod \"e2e-mt-7e6fe88b-openshift-default-5b64b75c89-l8cw5\" (UID: \"965d1fab-96f5-400b-bcf8-1be86ab61076\") " pod="openshift-ingress/e2e-mt-7e6fe88b-openshift-default-5b64b75c89-l8cw5"
Oct 31 00:56:50.261052 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:50.260861    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/965d1fab-96f5-400b-bcf8-1be86ab61076-istio-podinfo\") pod \"e2e-mt-7e6fe88b-openshift-default-5b64b75c89-l8cw5\" (UID: \"965d1fab-96f5-400b-bcf8-1be86ab61076\") " pod="openshift-ingress/e2e-mt-7e6fe88b-openshift-default-5b64b75c89-l8cw5"
Oct 31 00:56:50.261052 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:50.260924    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vx88b\" (UniqueName: \"kubernetes.io/projected/965d1fab-96f5-400b-bcf8-1be86ab61076-kube-api-access-vx88b\") pod \"e2e-mt-7e6fe88b-openshift-default-5b64b75c89-l8cw5\" (UID: \"965d1fab-96f5-400b-bcf8-1be86ab61076\") " pod="openshift-ingress/e2e-mt-7e6fe88b-openshift-default-5b64b75c89-l8cw5"
Oct 31 00:56:50.261052 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:50.260955    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/965d1fab-96f5-400b-bcf8-1be86ab61076-istiod-ca-cert\") pod \"e2e-mt-7e6fe88b-openshift-default-5b64b75c89-l8cw5\" (UID: \"965d1fab-96f5-400b-bcf8-1be86ab61076\") " pod="openshift-ingress/e2e-mt-7e6fe88b-openshift-default-5b64b75c89-l8cw5"
Oct 31 00:56:50.261261 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:50.261072    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/965d1fab-96f5-400b-bcf8-1be86ab61076-workload-socket\") pod \"e2e-mt-7e6fe88b-openshift-default-5b64b75c89-l8cw5\" (UID: \"965d1fab-96f5-400b-bcf8-1be86ab61076\") " pod="openshift-ingress/e2e-mt-7e6fe88b-openshift-default-5b64b75c89-l8cw5"
Oct 31 00:56:50.261261 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:50.261073    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/965d1fab-96f5-400b-bcf8-1be86ab61076-istio-data\") pod \"e2e-mt-7e6fe88b-openshift-default-5b64b75c89-l8cw5\" (UID: \"965d1fab-96f5-400b-bcf8-1be86ab61076\") " pod="openshift-ingress/e2e-mt-7e6fe88b-openshift-default-5b64b75c89-l8cw5"
Oct 31 00:56:50.261261 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:50.261177    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/965d1fab-96f5-400b-bcf8-1be86ab61076-credential-socket\") pod \"e2e-mt-7e6fe88b-openshift-default-5b64b75c89-l8cw5\" (UID: \"965d1fab-96f5-400b-bcf8-1be86ab61076\") " pod="openshift-ingress/e2e-mt-7e6fe88b-openshift-default-5b64b75c89-l8cw5"
Oct 31 00:56:50.261421 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:50.261290    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/965d1fab-96f5-400b-bcf8-1be86ab61076-workload-certs\") pod \"e2e-mt-7e6fe88b-openshift-default-5b64b75c89-l8cw5\" (UID: \"965d1fab-96f5-400b-bcf8-1be86ab61076\") " pod="openshift-ingress/e2e-mt-7e6fe88b-openshift-default-5b64b75c89-l8cw5"
Oct 31 00:56:50.261641 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:50.261617    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/965d1fab-96f5-400b-bcf8-1be86ab61076-istiod-ca-cert\") pod \"e2e-mt-7e6fe88b-openshift-default-5b64b75c89-l8cw5\" (UID: \"965d1fab-96f5-400b-bcf8-1be86ab61076\") " pod="openshift-ingress/e2e-mt-7e6fe88b-openshift-default-5b64b75c89-l8cw5"
Oct 31 00:56:50.264292 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:50.263819    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/965d1fab-96f5-400b-bcf8-1be86ab61076-istio-envoy\") pod \"e2e-mt-7e6fe88b-openshift-default-5b64b75c89-l8cw5\" (UID: \"965d1fab-96f5-400b-bcf8-1be86ab61076\") " pod="openshift-ingress/e2e-mt-7e6fe88b-openshift-default-5b64b75c89-l8cw5"
Oct 31 00:56:50.264979 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:50.264946    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/965d1fab-96f5-400b-bcf8-1be86ab61076-istio-podinfo\") pod \"e2e-mt-7e6fe88b-openshift-default-5b64b75c89-l8cw5\" (UID: \"965d1fab-96f5-400b-bcf8-1be86ab61076\") " pod="openshift-ingress/e2e-mt-7e6fe88b-openshift-default-5b64b75c89-l8cw5"
Oct 31 00:56:50.269244 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:50.269205    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-7e6fe88b-openshift-default-5b64b75c89-l8cw5"]
Oct 31 00:56:50.320039 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:50.319968    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/965d1fab-96f5-400b-bcf8-1be86ab61076-istio-token\") pod \"e2e-mt-7e6fe88b-openshift-default-5b64b75c89-l8cw5\" (UID: \"965d1fab-96f5-400b-bcf8-1be86ab61076\") " pod="openshift-ingress/e2e-mt-7e6fe88b-openshift-default-5b64b75c89-l8cw5"
Oct 31 00:56:50.320176 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:50.320109    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vx88b\" (UniqueName: \"kubernetes.io/projected/965d1fab-96f5-400b-bcf8-1be86ab61076-kube-api-access-vx88b\") pod \"e2e-mt-7e6fe88b-openshift-default-5b64b75c89-l8cw5\" (UID: \"965d1fab-96f5-400b-bcf8-1be86ab61076\") " pod="openshift-ingress/e2e-mt-7e6fe88b-openshift-default-5b64b75c89-l8cw5"
Oct 31 00:56:50.360826 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:50.360793    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-7e6fe88b-openshift-default-5b64b75c89-l8cw5"
Oct 31 00:56:50.787083 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:50.787054    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-7e6fe88b-openshift-default-5b64b75c89-l8cw5"]
Oct 31 00:56:50.787772 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:56:50.787748    4284 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod965d1fab_96f5_400b_bcf8_1be86ab61076.slice/crio-ddc619ca6301459166b9d81e23e03dc2a534dc83334662e44d1528a81931c6ee WatchSource:0}: Error finding container ddc619ca6301459166b9d81e23e03dc2a534dc83334662e44d1528a81931c6ee: Status 404 returned error can't find the container with id ddc619ca6301459166b9d81e23e03dc2a534dc83334662e44d1528a81931c6ee
Oct 31 00:56:50.790540 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:50.790512    4284 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 00:56:50.790611 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:50.790577    4284 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 00:56:50.790611 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:50.790606    4284 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 00:56:51.046612 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:51.045685    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-7e6fe88b-openshift-default-5b64b75c89-l8cw5" event={"ID":"965d1fab-96f5-400b-bcf8-1be86ab61076","Type":"ContainerStarted","Data":"77b99edf1a3273033b494ff3534649c142c507990d3758bb7b6ababbef95d2e8"}
Oct 31 00:56:51.046612 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:51.045755    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-7e6fe88b-openshift-default-5b64b75c89-l8cw5" event={"ID":"965d1fab-96f5-400b-bcf8-1be86ab61076","Type":"ContainerStarted","Data":"ddc619ca6301459166b9d81e23e03dc2a534dc83334662e44d1528a81931c6ee"}
Oct 31 00:56:51.165908 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:51.165845    4284 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-7e6fe88b-openshift-default-5b64b75c89-l8cw5" podStartSLOduration=2.165824116 podStartE2EDuration="2.165824116s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 00:56:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:56:51.164327522 +0000 UTC m=+1725.309334438" watchObservedRunningTime="2026-10-31 00:56:51.165824116 +0000 UTC m=+1725.310831042"
Oct 31 00:56:51.361028 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:51.360942    4284 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-7e6fe88b-openshift-default-5b64b75c89-l8cw5"
Oct 31 00:56:51.362252 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:51.362228    4284 patch_prober.go:29] interesting pod/e2e-mt-7e6fe88b-openshift-default-5b64b75c89-l8cw5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.107:15021/healthz/ready\": dial tcp 10.134.0.107:15021: connect: connection refused" start-of-body=
Oct 31 00:56:51.362390 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:51.362297    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-7e6fe88b-openshift-default-5b64b75c89-l8cw5" podUID="965d1fab-96f5-400b-bcf8-1be86ab61076" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.107:15021/healthz/ready\": dial tcp 10.134.0.107:15021: connect: connection refused"
Oct 31 00:56:51.937559 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:51.937518    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:56:52.361830 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:52.361747    4284 patch_prober.go:29] interesting pod/e2e-mt-7e6fe88b-openshift-default-5b64b75c89-l8cw5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.107:15021/healthz/ready\": dial tcp 10.134.0.107:15021: connect: connection refused" start-of-body=
Oct 31 00:56:52.361830 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:52.361806    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-7e6fe88b-openshift-default-5b64b75c89-l8cw5" podUID="965d1fab-96f5-400b-bcf8-1be86ab61076" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.107:15021/healthz/ready\": dial tcp 10.134.0.107:15021: connect: connection refused"
Oct 31 00:56:53.029350 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:53.029300    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:56:53.224871 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:53.224835    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:56:53.363012 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:53.362934    4284 patch_prober.go:29] interesting pod/e2e-mt-7e6fe88b-openshift-default-5b64b75c89-l8cw5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.107:15021/healthz/ready\": dial tcp 10.134.0.107:15021: connect: connection refused" start-of-body=
Oct 31 00:56:53.363886 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:53.363725    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-7e6fe88b-openshift-default-5b64b75c89-l8cw5" podUID="965d1fab-96f5-400b-bcf8-1be86ab61076" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.107:15021/healthz/ready\": dial tcp 10.134.0.107:15021: connect: connection refused"
Oct 31 00:56:53.418187 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:53.418148    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-69a4bc-openshift-default-797b9d8659-rgs5q"]
Oct 31 00:56:53.419047 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:53.419014    4284 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-69a4bc-openshift-default-797b9d8659-rgs5q" podUID="66210261-9baa-4961-855e-b92e2c4e154e" containerName="istio-proxy" containerID="cri-o://b19053f555a381b9431742ed6595a28a718339359ee7db011f0ede450776068a" gracePeriod=30
Oct 31 00:56:53.443403 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:53.443374    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:56:53.678826 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:53.678770    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:56:54.310369 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:54.310335    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:56:54.363909 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:54.363873    4284 patch_prober.go:29] interesting pod/e2e-mt-7e6fe88b-openshift-default-5b64b75c89-l8cw5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.107:15021/healthz/ready\": dial tcp 10.134.0.107:15021: connect: connection refused" start-of-body=
Oct 31 00:56:54.364614 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:54.364576    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-7e6fe88b-openshift-default-5b64b75c89-l8cw5" podUID="965d1fab-96f5-400b-bcf8-1be86ab61076" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.107:15021/healthz/ready\": dial tcp 10.134.0.107:15021: connect: connection refused"
Oct 31 00:56:54.571067 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:54.570976    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:56:54.740654 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:54.740623    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:56:54.975017 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:54.974983    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:56:55.145743 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:55.145709    4284 patch_prober.go:29] interesting pod/e2e-api-a-69a4bc-openshift-default-797b9d8659-rgs5q container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 31 00:56:55.145901 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:55.145763    4284 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-69a4bc-openshift-default-797b9d8659-rgs5q" podUID="66210261-9baa-4961-855e-b92e2c4e154e" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 31 00:56:55.180177 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:55.180144    4284 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-b-f6979f-openshift-default-569f6f8dff-4k4gk"]
Oct 31 00:56:55.200532 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:55.200501    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-f6979f-openshift-default-569f6f8dff-4k4gk"
Oct 31 00:56:55.248390 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:55.248180    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-f6979f-openshift-default-569f6f8dff-4k4gk"]
Oct 31 00:56:55.265418 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:55.265385    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:56:55.269362 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:55.269338    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c0f54831-726b-41bd-adfb-445f8b9a39f9-workload-socket\") pod \"e2e-ipp-b-f6979f-openshift-default-569f6f8dff-4k4gk\" (UID: \"c0f54831-726b-41bd-adfb-445f8b9a39f9\") " pod="openshift-ingress/e2e-ipp-b-f6979f-openshift-default-569f6f8dff-4k4gk"
Oct 31 00:56:55.269549 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:55.269536    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c0f54831-726b-41bd-adfb-445f8b9a39f9-workload-certs\") pod \"e2e-ipp-b-f6979f-openshift-default-569f6f8dff-4k4gk\" (UID: \"c0f54831-726b-41bd-adfb-445f8b9a39f9\") " pod="openshift-ingress/e2e-ipp-b-f6979f-openshift-default-569f6f8dff-4k4gk"
Oct 31 00:56:55.269651 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:55.269638    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c0f54831-726b-41bd-adfb-445f8b9a39f9-istio-envoy\") pod \"e2e-ipp-b-f6979f-openshift-default-569f6f8dff-4k4gk\" (UID: \"c0f54831-726b-41bd-adfb-445f8b9a39f9\") " pod="openshift-ingress/e2e-ipp-b-f6979f-openshift-default-569f6f8dff-4k4gk"
Oct 31 00:56:55.269786 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:55.269770    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c0f54831-726b-41bd-adfb-445f8b9a39f9-istio-podinfo\") pod \"e2e-ipp-b-f6979f-openshift-default-569f6f8dff-4k4gk\" (UID: \"c0f54831-726b-41bd-adfb-445f8b9a39f9\") " pod="openshift-ingress/e2e-ipp-b-f6979f-openshift-default-569f6f8dff-4k4gk"
Oct 31 00:56:55.269858 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:55.269814    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c0f54831-726b-41bd-adfb-445f8b9a39f9-credential-socket\") pod \"e2e-ipp-b-f6979f-openshift-default-569f6f8dff-4k4gk\" (UID: \"c0f54831-726b-41bd-adfb-445f8b9a39f9\") " pod="openshift-ingress/e2e-ipp-b-f6979f-openshift-default-569f6f8dff-4k4gk"
Oct 31 00:56:55.269858 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:55.269847    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c0f54831-726b-41bd-adfb-445f8b9a39f9-istio-data\") pod \"e2e-ipp-b-f6979f-openshift-default-569f6f8dff-4k4gk\" (UID: \"c0f54831-726b-41bd-adfb-445f8b9a39f9\") " pod="openshift-ingress/e2e-ipp-b-f6979f-openshift-default-569f6f8dff-4k4gk"
Oct 31 00:56:55.269976 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:55.269957    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c0f54831-726b-41bd-adfb-445f8b9a39f9-istio-token\") pod \"e2e-ipp-b-f6979f-openshift-default-569f6f8dff-4k4gk\" (UID: \"c0f54831-726b-41bd-adfb-445f8b9a39f9\") " pod="openshift-ingress/e2e-ipp-b-f6979f-openshift-default-569f6f8dff-4k4gk"
Oct 31 00:56:55.270027 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:55.269986    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q57zj\" (UniqueName: \"kubernetes.io/projected/c0f54831-726b-41bd-adfb-445f8b9a39f9-kube-api-access-q57zj\") pod \"e2e-ipp-b-f6979f-openshift-default-569f6f8dff-4k4gk\" (UID: \"c0f54831-726b-41bd-adfb-445f8b9a39f9\") " pod="openshift-ingress/e2e-ipp-b-f6979f-openshift-default-569f6f8dff-4k4gk"
Oct 31 00:56:55.270125 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:55.270113    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c0f54831-726b-41bd-adfb-445f8b9a39f9-istiod-ca-cert\") pod \"e2e-ipp-b-f6979f-openshift-default-569f6f8dff-4k4gk\" (UID: \"c0f54831-726b-41bd-adfb-445f8b9a39f9\") " pod="openshift-ingress/e2e-ipp-b-f6979f-openshift-default-569f6f8dff-4k4gk"
Oct 31 00:56:55.310235 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:55.310192    4284 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-b-f6979f-openshift-default-dockercfg-2rcbp\""
Oct 31 00:56:55.438234 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:55.422647    4284 patch_prober.go:29] interesting pod/e2e-mt-7e6fe88b-openshift-default-5b64b75c89-l8cw5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.107:15021/healthz/ready\": dial tcp 10.134.0.107:15021: connect: connection refused" start-of-body=
Oct 31 00:56:55.438234 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:55.422724    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-7e6fe88b-openshift-default-5b64b75c89-l8cw5" podUID="965d1fab-96f5-400b-bcf8-1be86ab61076" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.107:15021/healthz/ready\": dial tcp 10.134.0.107:15021: connect: connection refused"
Oct 31 00:56:55.438234 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:55.427353    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c0f54831-726b-41bd-adfb-445f8b9a39f9-istiod-ca-cert\") pod \"e2e-ipp-b-f6979f-openshift-default-569f6f8dff-4k4gk\" (UID: \"c0f54831-726b-41bd-adfb-445f8b9a39f9\") " pod="openshift-ingress/e2e-ipp-b-f6979f-openshift-default-569f6f8dff-4k4gk"
Oct 31 00:56:55.438234 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:55.427510    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c0f54831-726b-41bd-adfb-445f8b9a39f9-workload-socket\") pod \"e2e-ipp-b-f6979f-openshift-default-569f6f8dff-4k4gk\" (UID: \"c0f54831-726b-41bd-adfb-445f8b9a39f9\") " pod="openshift-ingress/e2e-ipp-b-f6979f-openshift-default-569f6f8dff-4k4gk"
Oct 31 00:56:55.438234 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:55.427571    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c0f54831-726b-41bd-adfb-445f8b9a39f9-workload-certs\") pod \"e2e-ipp-b-f6979f-openshift-default-569f6f8dff-4k4gk\" (UID: \"c0f54831-726b-41bd-adfb-445f8b9a39f9\") " pod="openshift-ingress/e2e-ipp-b-f6979f-openshift-default-569f6f8dff-4k4gk"
Oct 31 00:56:55.438234 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:55.427605    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c0f54831-726b-41bd-adfb-445f8b9a39f9-istio-envoy\") pod \"e2e-ipp-b-f6979f-openshift-default-569f6f8dff-4k4gk\" (UID: \"c0f54831-726b-41bd-adfb-445f8b9a39f9\") " pod="openshift-ingress/e2e-ipp-b-f6979f-openshift-default-569f6f8dff-4k4gk"
Oct 31 00:56:55.438234 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:55.427664    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c0f54831-726b-41bd-adfb-445f8b9a39f9-istio-podinfo\") pod \"e2e-ipp-b-f6979f-openshift-default-569f6f8dff-4k4gk\" (UID: \"c0f54831-726b-41bd-adfb-445f8b9a39f9\") " pod="openshift-ingress/e2e-ipp-b-f6979f-openshift-default-569f6f8dff-4k4gk"
Oct 31 00:56:55.438234 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:55.427712    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c0f54831-726b-41bd-adfb-445f8b9a39f9-credential-socket\") pod \"e2e-ipp-b-f6979f-openshift-default-569f6f8dff-4k4gk\" (UID: \"c0f54831-726b-41bd-adfb-445f8b9a39f9\") " pod="openshift-ingress/e2e-ipp-b-f6979f-openshift-default-569f6f8dff-4k4gk"
Oct 31 00:56:55.438234 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:55.427740    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c0f54831-726b-41bd-adfb-445f8b9a39f9-istio-data\") pod \"e2e-ipp-b-f6979f-openshift-default-569f6f8dff-4k4gk\" (UID: \"c0f54831-726b-41bd-adfb-445f8b9a39f9\") " pod="openshift-ingress/e2e-ipp-b-f6979f-openshift-default-569f6f8dff-4k4gk"
Oct 31 00:56:55.438234 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:55.427777    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c0f54831-726b-41bd-adfb-445f8b9a39f9-istio-token\") pod \"e2e-ipp-b-f6979f-openshift-default-569f6f8dff-4k4gk\" (UID: \"c0f54831-726b-41bd-adfb-445f8b9a39f9\") " pod="openshift-ingress/e2e-ipp-b-f6979f-openshift-default-569f6f8dff-4k4gk"
Oct 31 00:56:55.438234 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:55.427800    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q57zj\" (UniqueName: \"kubernetes.io/projected/c0f54831-726b-41bd-adfb-445f8b9a39f9-kube-api-access-q57zj\") pod \"e2e-ipp-b-f6979f-openshift-default-569f6f8dff-4k4gk\" (UID: \"c0f54831-726b-41bd-adfb-445f8b9a39f9\") " pod="openshift-ingress/e2e-ipp-b-f6979f-openshift-default-569f6f8dff-4k4gk"
Oct 31 00:56:55.449447 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:55.447075    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c0f54831-726b-41bd-adfb-445f8b9a39f9-credential-socket\") pod \"e2e-ipp-b-f6979f-openshift-default-569f6f8dff-4k4gk\" (UID: \"c0f54831-726b-41bd-adfb-445f8b9a39f9\") " pod="openshift-ingress/e2e-ipp-b-f6979f-openshift-default-569f6f8dff-4k4gk"
Oct 31 00:56:55.449447 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:55.447405    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c0f54831-726b-41bd-adfb-445f8b9a39f9-workload-certs\") pod \"e2e-ipp-b-f6979f-openshift-default-569f6f8dff-4k4gk\" (UID: \"c0f54831-726b-41bd-adfb-445f8b9a39f9\") " pod="openshift-ingress/e2e-ipp-b-f6979f-openshift-default-569f6f8dff-4k4gk"
Oct 31 00:56:55.470887 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:55.470823    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c0f54831-726b-41bd-adfb-445f8b9a39f9-istiod-ca-cert\") pod \"e2e-ipp-b-f6979f-openshift-default-569f6f8dff-4k4gk\" (UID: \"c0f54831-726b-41bd-adfb-445f8b9a39f9\") " pod="openshift-ingress/e2e-ipp-b-f6979f-openshift-default-569f6f8dff-4k4gk"
Oct 31 00:56:55.510322 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:55.510122    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c0f54831-726b-41bd-adfb-445f8b9a39f9-istio-podinfo\") pod \"e2e-ipp-b-f6979f-openshift-default-569f6f8dff-4k4gk\" (UID: \"c0f54831-726b-41bd-adfb-445f8b9a39f9\") " pod="openshift-ingress/e2e-ipp-b-f6979f-openshift-default-569f6f8dff-4k4gk"
Oct 31 00:56:55.510509 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:55.510464    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c0f54831-726b-41bd-adfb-445f8b9a39f9-istio-data\") pod \"e2e-ipp-b-f6979f-openshift-default-569f6f8dff-4k4gk\" (UID: \"c0f54831-726b-41bd-adfb-445f8b9a39f9\") " pod="openshift-ingress/e2e-ipp-b-f6979f-openshift-default-569f6f8dff-4k4gk"
Oct 31 00:56:55.518247 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:55.511270    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c0f54831-726b-41bd-adfb-445f8b9a39f9-workload-socket\") pod \"e2e-ipp-b-f6979f-openshift-default-569f6f8dff-4k4gk\" (UID: \"c0f54831-726b-41bd-adfb-445f8b9a39f9\") " pod="openshift-ingress/e2e-ipp-b-f6979f-openshift-default-569f6f8dff-4k4gk"
Oct 31 00:56:55.624140 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:55.624108    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c0f54831-726b-41bd-adfb-445f8b9a39f9-istio-token\") pod \"e2e-ipp-b-f6979f-openshift-default-569f6f8dff-4k4gk\" (UID: \"c0f54831-726b-41bd-adfb-445f8b9a39f9\") " pod="openshift-ingress/e2e-ipp-b-f6979f-openshift-default-569f6f8dff-4k4gk"
Oct 31 00:56:55.624573 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:55.624543    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c0f54831-726b-41bd-adfb-445f8b9a39f9-istio-envoy\") pod \"e2e-ipp-b-f6979f-openshift-default-569f6f8dff-4k4gk\" (UID: \"c0f54831-726b-41bd-adfb-445f8b9a39f9\") " pod="openshift-ingress/e2e-ipp-b-f6979f-openshift-default-569f6f8dff-4k4gk"
Oct 31 00:56:55.658545 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:55.657099    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q57zj\" (UniqueName: \"kubernetes.io/projected/c0f54831-726b-41bd-adfb-445f8b9a39f9-kube-api-access-q57zj\") pod \"e2e-ipp-b-f6979f-openshift-default-569f6f8dff-4k4gk\" (UID: \"c0f54831-726b-41bd-adfb-445f8b9a39f9\") " pod="openshift-ingress/e2e-ipp-b-f6979f-openshift-default-569f6f8dff-4k4gk"
Oct 31 00:56:55.767382 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:55.746175    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:56:55.860997 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:55.860875    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-f6979f-openshift-default-569f6f8dff-4k4gk"
Oct 31 00:56:56.361337 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:56.361304    4284 patch_prober.go:29] interesting pod/e2e-mt-7e6fe88b-openshift-default-5b64b75c89-l8cw5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.107:15021/healthz/ready\": dial tcp 10.134.0.107:15021: connect: connection refused" start-of-body=
Oct 31 00:56:56.361549 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:56.361358    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-7e6fe88b-openshift-default-5b64b75c89-l8cw5" podUID="965d1fab-96f5-400b-bcf8-1be86ab61076" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.107:15021/healthz/ready\": dial tcp 10.134.0.107:15021: connect: connection refused"
Oct 31 00:56:56.546471 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:56.546443    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-f6979f-openshift-default-569f6f8dff-4k4gk"]
Oct 31 00:56:56.547043 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:56:56.547018    4284 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc0f54831_726b_41bd_adfb_445f8b9a39f9.slice/crio-57aed69861500ab272cabd7fafdd4d08e5acda41a95cf2aa07ddb1b2a0a82ca0 WatchSource:0}: Error finding container 57aed69861500ab272cabd7fafdd4d08e5acda41a95cf2aa07ddb1b2a0a82ca0: Status 404 returned error can't find the container with id 57aed69861500ab272cabd7fafdd4d08e5acda41a95cf2aa07ddb1b2a0a82ca0
Oct 31 00:56:56.549831 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:56.549799    4284 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 00:56:56.549923 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:56.549863    4284 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 00:56:56.549923 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:56.549892    4284 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 00:56:56.583445 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:56.583417    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:56:57.132329 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:57.132298    4284 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-a-b647c3-openshift-default-674fcc8445-tvzlw"]
Oct 31 00:56:57.135814 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:57.135794    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-b647c3-openshift-default-674fcc8445-tvzlw"
Oct 31 00:56:57.190242 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:57.186091    4284 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-a-b647c3-openshift-default-dockercfg-pqh6d\""
Oct 31 00:56:57.254235 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:57.252141    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-b647c3-openshift-default-674fcc8445-tvzlw"]
Oct 31 00:56:57.288706 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:57.277544    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/caf73a48-2172-46df-9cbc-fd85b49552c1-istiod-ca-cert\") pod \"e2e-ipp-a-b647c3-openshift-default-674fcc8445-tvzlw\" (UID: \"caf73a48-2172-46df-9cbc-fd85b49552c1\") " pod="openshift-ingress/e2e-ipp-a-b647c3-openshift-default-674fcc8445-tvzlw"
Oct 31 00:56:57.288706 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:57.277649    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/caf73a48-2172-46df-9cbc-fd85b49552c1-credential-socket\") pod \"e2e-ipp-a-b647c3-openshift-default-674fcc8445-tvzlw\" (UID: \"caf73a48-2172-46df-9cbc-fd85b49552c1\") " pod="openshift-ingress/e2e-ipp-a-b647c3-openshift-default-674fcc8445-tvzlw"
Oct 31 00:56:57.288706 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:57.277754    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/caf73a48-2172-46df-9cbc-fd85b49552c1-istio-podinfo\") pod \"e2e-ipp-a-b647c3-openshift-default-674fcc8445-tvzlw\" (UID: \"caf73a48-2172-46df-9cbc-fd85b49552c1\") " pod="openshift-ingress/e2e-ipp-a-b647c3-openshift-default-674fcc8445-tvzlw"
Oct 31 00:56:57.288706 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:57.277792    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/caf73a48-2172-46df-9cbc-fd85b49552c1-istio-data\") pod \"e2e-ipp-a-b647c3-openshift-default-674fcc8445-tvzlw\" (UID: \"caf73a48-2172-46df-9cbc-fd85b49552c1\") " pod="openshift-ingress/e2e-ipp-a-b647c3-openshift-default-674fcc8445-tvzlw"
Oct 31 00:56:57.288706 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:57.277847    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/caf73a48-2172-46df-9cbc-fd85b49552c1-workload-certs\") pod \"e2e-ipp-a-b647c3-openshift-default-674fcc8445-tvzlw\" (UID: \"caf73a48-2172-46df-9cbc-fd85b49552c1\") " pod="openshift-ingress/e2e-ipp-a-b647c3-openshift-default-674fcc8445-tvzlw"
Oct 31 00:56:57.288706 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:57.277876    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/caf73a48-2172-46df-9cbc-fd85b49552c1-istio-envoy\") pod \"e2e-ipp-a-b647c3-openshift-default-674fcc8445-tvzlw\" (UID: \"caf73a48-2172-46df-9cbc-fd85b49552c1\") " pod="openshift-ingress/e2e-ipp-a-b647c3-openshift-default-674fcc8445-tvzlw"
Oct 31 00:56:57.288706 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:57.277973    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xkn66\" (UniqueName: \"kubernetes.io/projected/caf73a48-2172-46df-9cbc-fd85b49552c1-kube-api-access-xkn66\") pod \"e2e-ipp-a-b647c3-openshift-default-674fcc8445-tvzlw\" (UID: \"caf73a48-2172-46df-9cbc-fd85b49552c1\") " pod="openshift-ingress/e2e-ipp-a-b647c3-openshift-default-674fcc8445-tvzlw"
Oct 31 00:56:57.288706 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:57.278011    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/caf73a48-2172-46df-9cbc-fd85b49552c1-workload-socket\") pod \"e2e-ipp-a-b647c3-openshift-default-674fcc8445-tvzlw\" (UID: \"caf73a48-2172-46df-9cbc-fd85b49552c1\") " pod="openshift-ingress/e2e-ipp-a-b647c3-openshift-default-674fcc8445-tvzlw"
Oct 31 00:56:57.288706 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:57.278065    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/caf73a48-2172-46df-9cbc-fd85b49552c1-istio-token\") pod \"e2e-ipp-a-b647c3-openshift-default-674fcc8445-tvzlw\" (UID: \"caf73a48-2172-46df-9cbc-fd85b49552c1\") " pod="openshift-ingress/e2e-ipp-a-b647c3-openshift-default-674fcc8445-tvzlw"
Oct 31 00:56:57.411753 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:57.409148    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:56:57.429912 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:57.415073    4284 patch_prober.go:29] interesting pod/e2e-mt-7e6fe88b-openshift-default-5b64b75c89-l8cw5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.107:15021/healthz/ready\": dial tcp 10.134.0.107:15021: connect: connection refused" start-of-body=
Oct 31 00:56:57.429912 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:57.415125    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-7e6fe88b-openshift-default-5b64b75c89-l8cw5" podUID="965d1fab-96f5-400b-bcf8-1be86ab61076" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.107:15021/healthz/ready\": dial tcp 10.134.0.107:15021: connect: connection refused"
Oct 31 00:56:57.461603 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:57.455358    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/caf73a48-2172-46df-9cbc-fd85b49552c1-istio-podinfo\") pod \"e2e-ipp-a-b647c3-openshift-default-674fcc8445-tvzlw\" (UID: \"caf73a48-2172-46df-9cbc-fd85b49552c1\") " pod="openshift-ingress/e2e-ipp-a-b647c3-openshift-default-674fcc8445-tvzlw"
Oct 31 00:56:57.461603 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:57.455416    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/caf73a48-2172-46df-9cbc-fd85b49552c1-istio-data\") pod \"e2e-ipp-a-b647c3-openshift-default-674fcc8445-tvzlw\" (UID: \"caf73a48-2172-46df-9cbc-fd85b49552c1\") " pod="openshift-ingress/e2e-ipp-a-b647c3-openshift-default-674fcc8445-tvzlw"
Oct 31 00:56:57.461603 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:57.455491    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/caf73a48-2172-46df-9cbc-fd85b49552c1-workload-certs\") pod \"e2e-ipp-a-b647c3-openshift-default-674fcc8445-tvzlw\" (UID: \"caf73a48-2172-46df-9cbc-fd85b49552c1\") " pod="openshift-ingress/e2e-ipp-a-b647c3-openshift-default-674fcc8445-tvzlw"
Oct 31 00:56:57.461603 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:57.455524    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/caf73a48-2172-46df-9cbc-fd85b49552c1-istio-envoy\") pod \"e2e-ipp-a-b647c3-openshift-default-674fcc8445-tvzlw\" (UID: \"caf73a48-2172-46df-9cbc-fd85b49552c1\") " pod="openshift-ingress/e2e-ipp-a-b647c3-openshift-default-674fcc8445-tvzlw"
Oct 31 00:56:57.461603 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:57.455644    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xkn66\" (UniqueName: \"kubernetes.io/projected/caf73a48-2172-46df-9cbc-fd85b49552c1-kube-api-access-xkn66\") pod \"e2e-ipp-a-b647c3-openshift-default-674fcc8445-tvzlw\" (UID: \"caf73a48-2172-46df-9cbc-fd85b49552c1\") " pod="openshift-ingress/e2e-ipp-a-b647c3-openshift-default-674fcc8445-tvzlw"
Oct 31 00:56:57.461603 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:57.455686    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/caf73a48-2172-46df-9cbc-fd85b49552c1-workload-socket\") pod \"e2e-ipp-a-b647c3-openshift-default-674fcc8445-tvzlw\" (UID: \"caf73a48-2172-46df-9cbc-fd85b49552c1\") " pod="openshift-ingress/e2e-ipp-a-b647c3-openshift-default-674fcc8445-tvzlw"
Oct 31 00:56:57.461603 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:57.455749    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/caf73a48-2172-46df-9cbc-fd85b49552c1-istio-token\") pod \"e2e-ipp-a-b647c3-openshift-default-674fcc8445-tvzlw\" (UID: \"caf73a48-2172-46df-9cbc-fd85b49552c1\") " pod="openshift-ingress/e2e-ipp-a-b647c3-openshift-default-674fcc8445-tvzlw"
Oct 31 00:56:57.461603 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:57.455822    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/caf73a48-2172-46df-9cbc-fd85b49552c1-istiod-ca-cert\") pod \"e2e-ipp-a-b647c3-openshift-default-674fcc8445-tvzlw\" (UID: \"caf73a48-2172-46df-9cbc-fd85b49552c1\") " pod="openshift-ingress/e2e-ipp-a-b647c3-openshift-default-674fcc8445-tvzlw"
Oct 31 00:56:57.461603 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:57.455889    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/caf73a48-2172-46df-9cbc-fd85b49552c1-credential-socket\") pod \"e2e-ipp-a-b647c3-openshift-default-674fcc8445-tvzlw\" (UID: \"caf73a48-2172-46df-9cbc-fd85b49552c1\") " pod="openshift-ingress/e2e-ipp-a-b647c3-openshift-default-674fcc8445-tvzlw"
Oct 31 00:56:57.461603 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:57.456420    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/caf73a48-2172-46df-9cbc-fd85b49552c1-credential-socket\") pod \"e2e-ipp-a-b647c3-openshift-default-674fcc8445-tvzlw\" (UID: \"caf73a48-2172-46df-9cbc-fd85b49552c1\") " pod="openshift-ingress/e2e-ipp-a-b647c3-openshift-default-674fcc8445-tvzlw"
Oct 31 00:56:57.461603 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:57.458929    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/caf73a48-2172-46df-9cbc-fd85b49552c1-istio-data\") pod \"e2e-ipp-a-b647c3-openshift-default-674fcc8445-tvzlw\" (UID: \"caf73a48-2172-46df-9cbc-fd85b49552c1\") " pod="openshift-ingress/e2e-ipp-a-b647c3-openshift-default-674fcc8445-tvzlw"
Oct 31 00:56:57.461603 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:57.459286    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/caf73a48-2172-46df-9cbc-fd85b49552c1-workload-certs\") pod \"e2e-ipp-a-b647c3-openshift-default-674fcc8445-tvzlw\" (UID: \"caf73a48-2172-46df-9cbc-fd85b49552c1\") " pod="openshift-ingress/e2e-ipp-a-b647c3-openshift-default-674fcc8445-tvzlw"
Oct 31 00:56:57.469103 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:57.467793    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/caf73a48-2172-46df-9cbc-fd85b49552c1-istio-envoy\") pod \"e2e-ipp-a-b647c3-openshift-default-674fcc8445-tvzlw\" (UID: \"caf73a48-2172-46df-9cbc-fd85b49552c1\") " pod="openshift-ingress/e2e-ipp-a-b647c3-openshift-default-674fcc8445-tvzlw"
Oct 31 00:56:57.469103 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:57.468424    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/caf73a48-2172-46df-9cbc-fd85b49552c1-workload-socket\") pod \"e2e-ipp-a-b647c3-openshift-default-674fcc8445-tvzlw\" (UID: \"caf73a48-2172-46df-9cbc-fd85b49552c1\") " pod="openshift-ingress/e2e-ipp-a-b647c3-openshift-default-674fcc8445-tvzlw"
Oct 31 00:56:57.469103 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:57.469044    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/caf73a48-2172-46df-9cbc-fd85b49552c1-istiod-ca-cert\") pod \"e2e-ipp-a-b647c3-openshift-default-674fcc8445-tvzlw\" (UID: \"caf73a48-2172-46df-9cbc-fd85b49552c1\") " pod="openshift-ingress/e2e-ipp-a-b647c3-openshift-default-674fcc8445-tvzlw"
Oct 31 00:56:57.480033 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:57.479993    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-f6979f-openshift-default-569f6f8dff-4k4gk" event={"ID":"c0f54831-726b-41bd-adfb-445f8b9a39f9","Type":"ContainerStarted","Data":"fbcb1a47b41849c5e466e4e7fbbe5368ac5730a6bad7c113a530a849b67c6762"}
Oct 31 00:56:57.480256 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:57.480057    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-f6979f-openshift-default-569f6f8dff-4k4gk" event={"ID":"c0f54831-726b-41bd-adfb-445f8b9a39f9","Type":"ContainerStarted","Data":"57aed69861500ab272cabd7fafdd4d08e5acda41a95cf2aa07ddb1b2a0a82ca0"}
Oct 31 00:56:57.481761 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:57.481733    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/caf73a48-2172-46df-9cbc-fd85b49552c1-istio-podinfo\") pod \"e2e-ipp-a-b647c3-openshift-default-674fcc8445-tvzlw\" (UID: \"caf73a48-2172-46df-9cbc-fd85b49552c1\") " pod="openshift-ingress/e2e-ipp-a-b647c3-openshift-default-674fcc8445-tvzlw"
Oct 31 00:56:57.543452 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:57.543421    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xkn66\" (UniqueName: \"kubernetes.io/projected/caf73a48-2172-46df-9cbc-fd85b49552c1-kube-api-access-xkn66\") pod \"e2e-ipp-a-b647c3-openshift-default-674fcc8445-tvzlw\" (UID: \"caf73a48-2172-46df-9cbc-fd85b49552c1\") " pod="openshift-ingress/e2e-ipp-a-b647c3-openshift-default-674fcc8445-tvzlw"
Oct 31 00:56:57.564022 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:57.563986    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/caf73a48-2172-46df-9cbc-fd85b49552c1-istio-token\") pod \"e2e-ipp-a-b647c3-openshift-default-674fcc8445-tvzlw\" (UID: \"caf73a48-2172-46df-9cbc-fd85b49552c1\") " pod="openshift-ingress/e2e-ipp-a-b647c3-openshift-default-674fcc8445-tvzlw"
Oct 31 00:56:57.629726 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:57.629670    4284 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-b-f6979f-openshift-default-569f6f8dff-4k4gk" podStartSLOduration=2.629653836 podStartE2EDuration="2.629653836s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 00:56:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:56:57.626382861 +0000 UTC m=+1731.771389773" watchObservedRunningTime="2026-10-31 00:56:57.629653836 +0000 UTC m=+1731.774660822"
Oct 31 00:56:57.676344 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:57.676259    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:56:57.753305 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:57.751804    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-b647c3-openshift-default-674fcc8445-tvzlw"
Oct 31 00:56:57.836260 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:57.834939    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:56:57.863000 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:57.862965    4284 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-f6979f-openshift-default-569f6f8dff-4k4gk"
Oct 31 00:56:57.871963 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:57.871064    4284 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-f6979f-openshift-default-569f6f8dff-4k4gk"
Oct 31 00:56:58.280136 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:58.280109    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-b647c3-openshift-default-674fcc8445-tvzlw"]
Oct 31 00:56:58.280729 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:56:58.280701    4284 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcaf73a48_2172_46df_9cbc_fd85b49552c1.slice/crio-41efd4be8621908e58ede650447224f6712b3dcb0c90423bee2a347e6c8b9ea7 WatchSource:0}: Error finding container 41efd4be8621908e58ede650447224f6712b3dcb0c90423bee2a347e6c8b9ea7: Status 404 returned error can't find the container with id 41efd4be8621908e58ede650447224f6712b3dcb0c90423bee2a347e6c8b9ea7
Oct 31 00:56:58.283385 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:58.283353    4284 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 00:56:58.283498 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:58.283416    4284 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 00:56:58.283498 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:58.283448    4284 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 00:56:58.298884 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:58.298859    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-f66180-794b84955-ndwz8"]
Oct 31 00:56:58.420053 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:58.416370    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-f66180-97cbb6f6c-t5b4n"]
Oct 31 00:56:58.474691 ip-10-0-132-173 kubenswrapper[4284]: E1031 00:56:58.474659    4284 cadvisor_stats_provider.go:569] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod66210261_9baa_4961_855e_b92e2c4e154e.slice/crio-conmon-b19053f555a381b9431742ed6595a28a718339359ee7db011f0ede450776068a.scope\": RecentStats: unable to find data in memory cache]"
Oct 31 00:56:58.500415 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:58.498286    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-b647c3-openshift-default-674fcc8445-tvzlw" event={"ID":"caf73a48-2172-46df-9cbc-fd85b49552c1","Type":"ContainerStarted","Data":"200c2f3410b7b909a860d413e1b5e3459658ebce88cb879986f4357bffa51f2a"}
Oct 31 00:56:58.500415 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:58.498354    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-f6979f-openshift-default-569f6f8dff-4k4gk"
Oct 31 00:56:58.500415 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:58.498381    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-b647c3-openshift-default-674fcc8445-tvzlw" event={"ID":"caf73a48-2172-46df-9cbc-fd85b49552c1","Type":"ContainerStarted","Data":"41efd4be8621908e58ede650447224f6712b3dcb0c90423bee2a347e6c8b9ea7"}
Oct 31 00:56:58.500415 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:58.499460    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-f6979f-openshift-default-569f6f8dff-4k4gk"
Oct 31 00:56:58.586663 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:58.586624    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:56:58.618980 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:58.618945    4284 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-7e6fe88b-openshift-default-5b64b75c89-l8cw5"
Oct 31 00:56:58.619697 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:58.619666    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-7e6fe88b-openshift-default-5b64b75c89-l8cw5"
Oct 31 00:56:58.620803 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:58.620781    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-7e6fe88b-openshift-default-5b64b75c89-l8cw5"
Oct 31 00:56:58.725861 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:58.725838    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-69a4bc-openshift-default-797b9d8659-rgs5q"
Oct 31 00:56:58.742170 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:58.742137    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:56:58.752187 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:58.752155    4284 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-b647c3-openshift-default-674fcc8445-tvzlw"
Oct 31 00:56:58.773342 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:58.773286    4284 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-a-b647c3-openshift-default-674fcc8445-tvzlw" podStartSLOduration=1.7732692860000001 podStartE2EDuration="1.773269286s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 00:56:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:56:58.770277151 +0000 UTC m=+1732.915284068" watchObservedRunningTime="2026-10-31 00:56:58.773269286 +0000 UTC m=+1732.918276202"
Oct 31 00:56:58.776826 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:58.776801    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/66210261-9baa-4961-855e-b92e2c4e154e-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/66210261-9baa-4961-855e-b92e2c4e154e-workload-certs\") pod \"66210261-9baa-4961-855e-b92e2c4e154e\" (UID: \"66210261-9baa-4961-855e-b92e2c4e154e\") "
Oct 31 00:56:58.777010 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:58.776898    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/66210261-9baa-4961-855e-b92e2c4e154e-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/66210261-9baa-4961-855e-b92e2c4e154e-istio-data\") pod \"66210261-9baa-4961-855e-b92e2c4e154e\" (UID: \"66210261-9baa-4961-855e-b92e2c4e154e\") "
Oct 31 00:56:58.777010 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:58.776934    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/66210261-9baa-4961-855e-b92e2c4e154e-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/66210261-9baa-4961-855e-b92e2c4e154e-istiod-ca-cert\") pod \"66210261-9baa-4961-855e-b92e2c4e154e\" (UID: \"66210261-9baa-4961-855e-b92e2c4e154e\") "
Oct 31 00:56:58.777010 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:58.776958    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/66210261-9baa-4961-855e-b92e2c4e154e-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/66210261-9baa-4961-855e-b92e2c4e154e-workload-socket\") pod \"66210261-9baa-4961-855e-b92e2c4e154e\" (UID: \"66210261-9baa-4961-855e-b92e2c4e154e\") "
Oct 31 00:56:58.777010 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:58.776997    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/66210261-9baa-4961-855e-b92e2c4e154e-kube-api-access-k8sv9\" (UniqueName: \"kubernetes.io/projected/66210261-9baa-4961-855e-b92e2c4e154e-kube-api-access-k8sv9\") pod \"66210261-9baa-4961-855e-b92e2c4e154e\" (UID: \"66210261-9baa-4961-855e-b92e2c4e154e\") "
Oct 31 00:56:58.777251 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:58.777039    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/66210261-9baa-4961-855e-b92e2c4e154e-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/66210261-9baa-4961-855e-b92e2c4e154e-istio-envoy\") pod \"66210261-9baa-4961-855e-b92e2c4e154e\" (UID: \"66210261-9baa-4961-855e-b92e2c4e154e\") "
Oct 31 00:56:58.777251 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:58.777082    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/66210261-9baa-4961-855e-b92e2c4e154e-workload-certs" pod "66210261-9baa-4961-855e-b92e2c4e154e" (UID: "66210261-9baa-4961-855e-b92e2c4e154e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:56:58.777251 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:58.777103    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/66210261-9baa-4961-855e-b92e2c4e154e-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/66210261-9baa-4961-855e-b92e2c4e154e-credential-socket\") pod \"66210261-9baa-4961-855e-b92e2c4e154e\" (UID: \"66210261-9baa-4961-855e-b92e2c4e154e\") "
Oct 31 00:56:58.777251 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:58.777132    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/66210261-9baa-4961-855e-b92e2c4e154e-istio-token\" (UniqueName: \"kubernetes.io/projected/66210261-9baa-4961-855e-b92e2c4e154e-istio-token\") pod \"66210261-9baa-4961-855e-b92e2c4e154e\" (UID: \"66210261-9baa-4961-855e-b92e2c4e154e\") "
Oct 31 00:56:58.777251 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:58.777164    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/66210261-9baa-4961-855e-b92e2c4e154e-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/66210261-9baa-4961-855e-b92e2c4e154e-istio-podinfo\") pod \"66210261-9baa-4961-855e-b92e2c4e154e\" (UID: \"66210261-9baa-4961-855e-b92e2c4e154e\") "
Oct 31 00:56:58.777251 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:58.777202    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/66210261-9baa-4961-855e-b92e2c4e154e-istio-data" pod "66210261-9baa-4961-855e-b92e2c4e154e" (UID: "66210261-9baa-4961-855e-b92e2c4e154e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:56:58.777584 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:58.777293    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/66210261-9baa-4961-855e-b92e2c4e154e-istiod-ca-cert" pod "66210261-9baa-4961-855e-b92e2c4e154e" (UID: "66210261-9baa-4961-855e-b92e2c4e154e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 00:56:58.777682 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:58.777658    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/66210261-9baa-4961-855e-b92e2c4e154e-credential-socket" pod "66210261-9baa-4961-855e-b92e2c4e154e" (UID: "66210261-9baa-4961-855e-b92e2c4e154e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:56:58.777847 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:58.777826    4284 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/66210261-9baa-4961-855e-b92e2c4e154e-workload-certs\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:56:58.778335 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:58.777852    4284 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/66210261-9baa-4961-855e-b92e2c4e154e-istio-data\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:56:58.778335 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:58.777867    4284 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/66210261-9baa-4961-855e-b92e2c4e154e-istiod-ca-cert\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:56:58.778335 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:58.777883    4284 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/66210261-9baa-4961-855e-b92e2c4e154e-credential-socket\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:56:58.778545 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:58.778330    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/66210261-9baa-4961-855e-b92e2c4e154e-workload-socket" pod "66210261-9baa-4961-855e-b92e2c4e154e" (UID: "66210261-9baa-4961-855e-b92e2c4e154e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:56:58.779821 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:58.779797    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/66210261-9baa-4961-855e-b92e2c4e154e-kube-api-access-k8sv9" pod "66210261-9baa-4961-855e-b92e2c4e154e" (UID: "66210261-9baa-4961-855e-b92e2c4e154e"). InnerVolumeSpecName "kube-api-access-k8sv9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 00:56:58.779984 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:58.779964    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/66210261-9baa-4961-855e-b92e2c4e154e-istio-envoy" pod "66210261-9baa-4961-855e-b92e2c4e154e" (UID: "66210261-9baa-4961-855e-b92e2c4e154e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:56:58.780124 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:58.780101    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/66210261-9baa-4961-855e-b92e2c4e154e-istio-podinfo" pod "66210261-9baa-4961-855e-b92e2c4e154e" (UID: "66210261-9baa-4961-855e-b92e2c4e154e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 00:56:58.781116 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:58.781093    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/66210261-9baa-4961-855e-b92e2c4e154e-istio-token" pod "66210261-9baa-4961-855e-b92e2c4e154e" (UID: "66210261-9baa-4961-855e-b92e2c4e154e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 00:56:58.879693 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:58.879663    4284 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/66210261-9baa-4961-855e-b92e2c4e154e-workload-socket\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:56:58.879693 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:58.879690    4284 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k8sv9\" (UniqueName: \"kubernetes.io/projected/66210261-9baa-4961-855e-b92e2c4e154e-kube-api-access-k8sv9\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:56:58.879915 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:58.879701    4284 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/66210261-9baa-4961-855e-b92e2c4e154e-istio-envoy\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:56:58.879915 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:58.879711    4284 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/66210261-9baa-4961-855e-b92e2c4e154e-istio-token\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:56:58.879915 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:58.879719    4284 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/66210261-9baa-4961-855e-b92e2c4e154e-istio-podinfo\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:56:58.895092 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:58.895052    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:56:59.503781 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:59.503751    4284 generic.go:356] "Generic (PLEG): container finished" podID="66210261-9baa-4961-855e-b92e2c4e154e" containerID="b19053f555a381b9431742ed6595a28a718339359ee7db011f0ede450776068a" exitCode=0
Oct 31 00:56:59.503947 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:59.503803    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-69a4bc-openshift-default-797b9d8659-rgs5q"
Oct 31 00:56:59.503947 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:59.503850    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-69a4bc-openshift-default-797b9d8659-rgs5q" event={"ID":"66210261-9baa-4961-855e-b92e2c4e154e","Type":"ContainerDied","Data":"b19053f555a381b9431742ed6595a28a718339359ee7db011f0ede450776068a"}
Oct 31 00:56:59.503947 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:59.503903    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-69a4bc-openshift-default-797b9d8659-rgs5q" event={"ID":"66210261-9baa-4961-855e-b92e2c4e154e","Type":"ContainerDied","Data":"c415a3d6cc811e0f480d827e796cd4f1070ed5ea1f425f32c0a7b2cc52d6defd"}
Oct 31 00:56:59.503947 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:59.503932    4284 scope.go:122] "RemoveContainer" containerID="b19053f555a381b9431742ed6595a28a718339359ee7db011f0ede450776068a"
Oct 31 00:56:59.519524 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:59.519501    4284 scope.go:122] "RemoveContainer" containerID="ec2024aa775dd9368e0a4c1c2655250955a14d7564e3d508b509215a11518476"
Oct 31 00:56:59.530173 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:59.530151    4284 scope.go:122] "RemoveContainer" containerID="b19053f555a381b9431742ed6595a28a718339359ee7db011f0ede450776068a"
Oct 31 00:56:59.530461 ip-10-0-132-173 kubenswrapper[4284]: E1031 00:56:59.530442    4284 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b19053f555a381b9431742ed6595a28a718339359ee7db011f0ede450776068a\": container with ID starting with b19053f555a381b9431742ed6595a28a718339359ee7db011f0ede450776068a not found: ID does not exist" containerID="b19053f555a381b9431742ed6595a28a718339359ee7db011f0ede450776068a"
Oct 31 00:56:59.530564 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:59.530470    4284 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b19053f555a381b9431742ed6595a28a718339359ee7db011f0ede450776068a"} err="failed to get container status \"b19053f555a381b9431742ed6595a28a718339359ee7db011f0ede450776068a\": rpc error: code = NotFound desc = could not find container \"b19053f555a381b9431742ed6595a28a718339359ee7db011f0ede450776068a\": container with ID starting with b19053f555a381b9431742ed6595a28a718339359ee7db011f0ede450776068a not found: ID does not exist"
Oct 31 00:56:59.530564 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:59.530490    4284 scope.go:122] "RemoveContainer" containerID="ec2024aa775dd9368e0a4c1c2655250955a14d7564e3d508b509215a11518476"
Oct 31 00:56:59.530781 ip-10-0-132-173 kubenswrapper[4284]: E1031 00:56:59.530749    4284 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ec2024aa775dd9368e0a4c1c2655250955a14d7564e3d508b509215a11518476\": container with ID starting with ec2024aa775dd9368e0a4c1c2655250955a14d7564e3d508b509215a11518476 not found: ID does not exist" containerID="ec2024aa775dd9368e0a4c1c2655250955a14d7564e3d508b509215a11518476"
Oct 31 00:56:59.530854 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:59.530785    4284 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ec2024aa775dd9368e0a4c1c2655250955a14d7564e3d508b509215a11518476"} err="failed to get container status \"ec2024aa775dd9368e0a4c1c2655250955a14d7564e3d508b509215a11518476\": rpc error: code = NotFound desc = could not find container \"ec2024aa775dd9368e0a4c1c2655250955a14d7564e3d508b509215a11518476\": container with ID starting with ec2024aa775dd9368e0a4c1c2655250955a14d7564e3d508b509215a11518476 not found: ID does not exist"
Oct 31 00:56:59.664895 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:59.664864    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-69a4bc-openshift-default-797b9d8659-rgs5q"]
Oct 31 00:56:59.670823 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:59.670799    4284 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-69a4bc-openshift-default-797b9d8659-rgs5q"]
Oct 31 00:56:59.757554 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:56:59.757479    4284 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-b647c3-openshift-default-674fcc8445-tvzlw"
Oct 31 00:57:00.010321 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:00.007419    4284 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="66210261-9baa-4961-855e-b92e2c4e154e" path="/var/lib/kubelet/pods/66210261-9baa-4961-855e-b92e2c4e154e/volumes"
Oct 31 00:57:00.511522 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:00.511489    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-b647c3-openshift-default-674fcc8445-tvzlw"
Oct 31 00:57:00.512505 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:00.512478    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-b647c3-openshift-default-674fcc8445-tvzlw"
Oct 31 00:57:00.749003 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:00.748970    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:57:00.917695 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:00.917640    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:57:01.074012 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:01.073977    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:57:02.048999 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:02.048954    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:57:02.248335 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:02.248302    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:57:03.236687 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:03.236652    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:57:03.314889 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:03.314842    4284 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-a-f66180-794b84955-ndwz8" podUID="d7d574bb-fe1e-4566-8f50-bcf9dabba619" containerName="payload-processing" containerID="cri-o://6d92b26bd9ba4dffd110f46c137aa9da449d2e1020f100e29002c345a58dcbac" gracePeriod=25
Oct 31 00:57:03.417241 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:03.417189    4284 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-f66180-97cbb6f6c-t5b4n" podUID="804be1f8-9d63-4f09-9199-b90409a88db5" containerName="payload-pre-processing" containerID="cri-o://35df1344a93032601558b8aa73ce8c7b79f20ee0abccc2e959f7e5214564e846" gracePeriod=25
Oct 31 00:57:03.535257 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:03.535195    4284 generic.go:356] "Generic (PLEG): container finished" podID="d7d574bb-fe1e-4566-8f50-bcf9dabba619" containerID="6d92b26bd9ba4dffd110f46c137aa9da449d2e1020f100e29002c345a58dcbac" exitCode=0
Oct 31 00:57:03.535257 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:03.535243    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-f66180-794b84955-ndwz8" event={"ID":"d7d574bb-fe1e-4566-8f50-bcf9dabba619","Type":"ContainerDied","Data":"6d92b26bd9ba4dffd110f46c137aa9da449d2e1020f100e29002c345a58dcbac"}
Oct 31 00:57:03.607289 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:03.607178    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-f66180-794b84955-ndwz8"
Oct 31 00:57:03.701665 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:03.701641    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-f66180-97cbb6f6c-t5b4n"
Oct 31 00:57:03.719986 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:03.719949    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d7d574bb-fe1e-4566-8f50-bcf9dabba619-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d7d574bb-fe1e-4566-8f50-bcf9dabba619-plugins-config-volume\") pod \"d7d574bb-fe1e-4566-8f50-bcf9dabba619\" (UID: \"d7d574bb-fe1e-4566-8f50-bcf9dabba619\") "
Oct 31 00:57:03.720166 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:03.720051    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d7d574bb-fe1e-4566-8f50-bcf9dabba619-kube-api-access-ptj2c\" (UniqueName: \"kubernetes.io/projected/d7d574bb-fe1e-4566-8f50-bcf9dabba619-kube-api-access-ptj2c\") pod \"d7d574bb-fe1e-4566-8f50-bcf9dabba619\" (UID: \"d7d574bb-fe1e-4566-8f50-bcf9dabba619\") "
Oct 31 00:57:03.720384 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:03.720357    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d7d574bb-fe1e-4566-8f50-bcf9dabba619-plugins-config-volume" pod "d7d574bb-fe1e-4566-8f50-bcf9dabba619" (UID: "d7d574bb-fe1e-4566-8f50-bcf9dabba619"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 00:57:03.720705 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:03.720679    4284 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d7d574bb-fe1e-4566-8f50-bcf9dabba619-plugins-config-volume\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:57:03.722678 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:03.722652    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d7d574bb-fe1e-4566-8f50-bcf9dabba619-kube-api-access-ptj2c" pod "d7d574bb-fe1e-4566-8f50-bcf9dabba619" (UID: "d7d574bb-fe1e-4566-8f50-bcf9dabba619"). InnerVolumeSpecName "kube-api-access-ptj2c". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 00:57:03.822082 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:03.821985    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/804be1f8-9d63-4f09-9199-b90409a88db5-kube-api-access-8hbpk\" (UniqueName: \"kubernetes.io/projected/804be1f8-9d63-4f09-9199-b90409a88db5-kube-api-access-8hbpk\") pod \"804be1f8-9d63-4f09-9199-b90409a88db5\" (UID: \"804be1f8-9d63-4f09-9199-b90409a88db5\") "
Oct 31 00:57:03.822279 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:03.822158    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/804be1f8-9d63-4f09-9199-b90409a88db5-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/804be1f8-9d63-4f09-9199-b90409a88db5-plugins-config-volume\") pod \"804be1f8-9d63-4f09-9199-b90409a88db5\" (UID: \"804be1f8-9d63-4f09-9199-b90409a88db5\") "
Oct 31 00:57:03.822539 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:03.822515    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/804be1f8-9d63-4f09-9199-b90409a88db5-plugins-config-volume" pod "804be1f8-9d63-4f09-9199-b90409a88db5" (UID: "804be1f8-9d63-4f09-9199-b90409a88db5"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 00:57:03.822648 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:03.822634    4284 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/804be1f8-9d63-4f09-9199-b90409a88db5-plugins-config-volume\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:57:03.822879 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:03.822650    4284 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ptj2c\" (UniqueName: \"kubernetes.io/projected/d7d574bb-fe1e-4566-8f50-bcf9dabba619-kube-api-access-ptj2c\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:57:03.824251 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:03.824227    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/804be1f8-9d63-4f09-9199-b90409a88db5-kube-api-access-8hbpk" pod "804be1f8-9d63-4f09-9199-b90409a88db5" (UID: "804be1f8-9d63-4f09-9199-b90409a88db5"). InnerVolumeSpecName "kube-api-access-8hbpk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 00:57:03.924064 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:03.924026    4284 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8hbpk\" (UniqueName: \"kubernetes.io/projected/804be1f8-9d63-4f09-9199-b90409a88db5-kube-api-access-8hbpk\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:57:04.425998 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:04.425957    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:57:04.589241 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:04.588244    4284 generic.go:356] "Generic (PLEG): container finished" podID="804be1f8-9d63-4f09-9199-b90409a88db5" containerID="35df1344a93032601558b8aa73ce8c7b79f20ee0abccc2e959f7e5214564e846" exitCode=0
Oct 31 00:57:04.589241 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:04.588385    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-f66180-97cbb6f6c-t5b4n" event={"ID":"804be1f8-9d63-4f09-9199-b90409a88db5","Type":"ContainerDied","Data":"35df1344a93032601558b8aa73ce8c7b79f20ee0abccc2e959f7e5214564e846"}
Oct 31 00:57:04.589241 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:04.588438    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-f66180-97cbb6f6c-t5b4n" event={"ID":"804be1f8-9d63-4f09-9199-b90409a88db5","Type":"ContainerDied","Data":"2bf12c4c5a62f2607d9647fda8a08839a46bfc685d7fe6f74b033c3df1fc30f4"}
Oct 31 00:57:04.589241 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:04.588466    4284 scope.go:122] "RemoveContainer" containerID="35df1344a93032601558b8aa73ce8c7b79f20ee0abccc2e959f7e5214564e846"
Oct 31 00:57:04.589241 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:04.588598    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-f66180-97cbb6f6c-t5b4n"
Oct 31 00:57:04.597905 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:04.597870    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-f66180-794b84955-ndwz8" event={"ID":"d7d574bb-fe1e-4566-8f50-bcf9dabba619","Type":"ContainerDied","Data":"61f497e07eb3338718d44465870542c729987dd96ca716809aaddeb571958a28"}
Oct 31 00:57:04.598302 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:04.598274    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-f66180-794b84955-ndwz8"
Oct 31 00:57:04.646288 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:04.626594    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:57:04.814750 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:04.813484    4284 scope.go:122] "RemoveContainer" containerID="35df1344a93032601558b8aa73ce8c7b79f20ee0abccc2e959f7e5214564e846"
Oct 31 00:57:04.823052 ip-10-0-132-173 kubenswrapper[4284]: E1031 00:57:04.822990    4284 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"35df1344a93032601558b8aa73ce8c7b79f20ee0abccc2e959f7e5214564e846\": container with ID starting with 35df1344a93032601558b8aa73ce8c7b79f20ee0abccc2e959f7e5214564e846 not found: ID does not exist" containerID="35df1344a93032601558b8aa73ce8c7b79f20ee0abccc2e959f7e5214564e846"
Oct 31 00:57:04.823726 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:04.823027    4284 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"35df1344a93032601558b8aa73ce8c7b79f20ee0abccc2e959f7e5214564e846"} err="failed to get container status \"35df1344a93032601558b8aa73ce8c7b79f20ee0abccc2e959f7e5214564e846\": rpc error: code = NotFound desc = could not find container \"35df1344a93032601558b8aa73ce8c7b79f20ee0abccc2e959f7e5214564e846\": container with ID starting with 35df1344a93032601558b8aa73ce8c7b79f20ee0abccc2e959f7e5214564e846 not found: ID does not exist"
Oct 31 00:57:04.823726 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:04.823271    4284 scope.go:122] "RemoveContainer" containerID="6d92b26bd9ba4dffd110f46c137aa9da449d2e1020f100e29002c345a58dcbac"
Oct 31 00:57:04.833251 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:04.829256    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-f66180-97cbb6f6c-t5b4n"]
Oct 31 00:57:04.882255 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:04.834675    4284 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-f66180-97cbb6f6c-t5b4n"]
Oct 31 00:57:04.937674 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:04.937638    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-f66180-794b84955-ndwz8"]
Oct 31 00:57:04.940990 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:04.940960    4284 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-f66180-794b84955-ndwz8"]
Oct 31 00:57:05.187360 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:05.187311    4284 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-7e6fe88b-openshift-default-5b64b75c89-8l2zb"]
Oct 31 00:57:05.188049 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:05.187982    4284 cpu_manager.go:414] "containerMap: removing container" podUID="d7d574bb-fe1e-4566-8f50-bcf9dabba619" containerName="payload-processing"
Oct 31 00:57:05.188049 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:05.188004    4284 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d7d574bb-fe1e-4566-8f50-bcf9dabba619" containerName="payload-processing"
Oct 31 00:57:05.188201 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:05.188058    4284 cpu_manager.go:414] "containerMap: removing container" podUID="66210261-9baa-4961-855e-b92e2c4e154e" containerName="istio-proxy"
Oct 31 00:57:05.188201 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:05.188066    4284 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="66210261-9baa-4961-855e-b92e2c4e154e" containerName="istio-proxy"
Oct 31 00:57:05.188201 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:05.188079    4284 cpu_manager.go:414] "containerMap: removing container" podUID="804be1f8-9d63-4f09-9199-b90409a88db5" containerName="payload-pre-processing"
Oct 31 00:57:05.188201 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:05.188087    4284 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="804be1f8-9d63-4f09-9199-b90409a88db5" containerName="payload-pre-processing"
Oct 31 00:57:05.188201 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:05.188108    4284 cpu_manager.go:414] "containerMap: removing container" podUID="66210261-9baa-4961-855e-b92e2c4e154e" containerName="istio-proxy"
Oct 31 00:57:05.188201 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:05.188115    4284 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="66210261-9baa-4961-855e-b92e2c4e154e" containerName="istio-proxy"
Oct 31 00:57:05.188597 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:05.188303    4284 memory_manager.go:365] "RemoveStaleState removing state" podUID="d7d574bb-fe1e-4566-8f50-bcf9dabba619" containerName="payload-processing"
Oct 31 00:57:05.188597 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:05.188322    4284 memory_manager.go:365] "RemoveStaleState removing state" podUID="66210261-9baa-4961-855e-b92e2c4e154e" containerName="istio-proxy"
Oct 31 00:57:05.188597 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:05.188333    4284 memory_manager.go:365] "RemoveStaleState removing state" podUID="804be1f8-9d63-4f09-9199-b90409a88db5" containerName="payload-pre-processing"
Oct 31 00:57:05.188597 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:05.188349    4284 memory_manager.go:365] "RemoveStaleState removing state" podUID="66210261-9baa-4961-855e-b92e2c4e154e" containerName="istio-proxy"
Oct 31 00:57:05.191905 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:05.191884    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-7e6fe88b-openshift-default-5b64b75c89-8l2zb"
Oct 31 00:57:05.263249 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:05.261805    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-7e6fe88b-openshift-default-5b64b75c89-8l2zb"]
Oct 31 00:57:05.301417 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:05.301384    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/95491ac4-0f03-4ee0-a8cd-687621cbfa85-istio-envoy\") pod \"e2e-mt-7e6fe88b-openshift-default-5b64b75c89-8l2zb\" (UID: \"95491ac4-0f03-4ee0-a8cd-687621cbfa85\") " pod="openshift-ingress/e2e-mt-7e6fe88b-openshift-default-5b64b75c89-8l2zb"
Oct 31 00:57:05.301611 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:05.301454    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/95491ac4-0f03-4ee0-a8cd-687621cbfa85-credential-socket\") pod \"e2e-mt-7e6fe88b-openshift-default-5b64b75c89-8l2zb\" (UID: \"95491ac4-0f03-4ee0-a8cd-687621cbfa85\") " pod="openshift-ingress/e2e-mt-7e6fe88b-openshift-default-5b64b75c89-8l2zb"
Oct 31 00:57:05.301611 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:05.301593    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hbwft\" (UniqueName: \"kubernetes.io/projected/95491ac4-0f03-4ee0-a8cd-687621cbfa85-kube-api-access-hbwft\") pod \"e2e-mt-7e6fe88b-openshift-default-5b64b75c89-8l2zb\" (UID: \"95491ac4-0f03-4ee0-a8cd-687621cbfa85\") " pod="openshift-ingress/e2e-mt-7e6fe88b-openshift-default-5b64b75c89-8l2zb"
Oct 31 00:57:05.301742 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:05.301712    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/95491ac4-0f03-4ee0-a8cd-687621cbfa85-istio-token\") pod \"e2e-mt-7e6fe88b-openshift-default-5b64b75c89-8l2zb\" (UID: \"95491ac4-0f03-4ee0-a8cd-687621cbfa85\") " pod="openshift-ingress/e2e-mt-7e6fe88b-openshift-default-5b64b75c89-8l2zb"
Oct 31 00:57:05.301788 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:05.301758    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/95491ac4-0f03-4ee0-a8cd-687621cbfa85-istiod-ca-cert\") pod \"e2e-mt-7e6fe88b-openshift-default-5b64b75c89-8l2zb\" (UID: \"95491ac4-0f03-4ee0-a8cd-687621cbfa85\") " pod="openshift-ingress/e2e-mt-7e6fe88b-openshift-default-5b64b75c89-8l2zb"
Oct 31 00:57:05.301837 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:05.301800    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/95491ac4-0f03-4ee0-a8cd-687621cbfa85-workload-socket\") pod \"e2e-mt-7e6fe88b-openshift-default-5b64b75c89-8l2zb\" (UID: \"95491ac4-0f03-4ee0-a8cd-687621cbfa85\") " pod="openshift-ingress/e2e-mt-7e6fe88b-openshift-default-5b64b75c89-8l2zb"
Oct 31 00:57:05.301946 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:05.301928    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/95491ac4-0f03-4ee0-a8cd-687621cbfa85-workload-certs\") pod \"e2e-mt-7e6fe88b-openshift-default-5b64b75c89-8l2zb\" (UID: \"95491ac4-0f03-4ee0-a8cd-687621cbfa85\") " pod="openshift-ingress/e2e-mt-7e6fe88b-openshift-default-5b64b75c89-8l2zb"
Oct 31 00:57:05.302018 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:05.302005    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/95491ac4-0f03-4ee0-a8cd-687621cbfa85-istio-data\") pod \"e2e-mt-7e6fe88b-openshift-default-5b64b75c89-8l2zb\" (UID: \"95491ac4-0f03-4ee0-a8cd-687621cbfa85\") " pod="openshift-ingress/e2e-mt-7e6fe88b-openshift-default-5b64b75c89-8l2zb"
Oct 31 00:57:05.302124 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:05.302105    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/95491ac4-0f03-4ee0-a8cd-687621cbfa85-istio-podinfo\") pod \"e2e-mt-7e6fe88b-openshift-default-5b64b75c89-8l2zb\" (UID: \"95491ac4-0f03-4ee0-a8cd-687621cbfa85\") " pod="openshift-ingress/e2e-mt-7e6fe88b-openshift-default-5b64b75c89-8l2zb"
Oct 31 00:57:05.403350 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:05.403310    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/95491ac4-0f03-4ee0-a8cd-687621cbfa85-workload-socket\") pod \"e2e-mt-7e6fe88b-openshift-default-5b64b75c89-8l2zb\" (UID: \"95491ac4-0f03-4ee0-a8cd-687621cbfa85\") " pod="openshift-ingress/e2e-mt-7e6fe88b-openshift-default-5b64b75c89-8l2zb"
Oct 31 00:57:05.403562 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:05.403369    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/95491ac4-0f03-4ee0-a8cd-687621cbfa85-workload-certs\") pod \"e2e-mt-7e6fe88b-openshift-default-5b64b75c89-8l2zb\" (UID: \"95491ac4-0f03-4ee0-a8cd-687621cbfa85\") " pod="openshift-ingress/e2e-mt-7e6fe88b-openshift-default-5b64b75c89-8l2zb"
Oct 31 00:57:05.403562 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:05.403479    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/95491ac4-0f03-4ee0-a8cd-687621cbfa85-istio-data\") pod \"e2e-mt-7e6fe88b-openshift-default-5b64b75c89-8l2zb\" (UID: \"95491ac4-0f03-4ee0-a8cd-687621cbfa85\") " pod="openshift-ingress/e2e-mt-7e6fe88b-openshift-default-5b64b75c89-8l2zb"
Oct 31 00:57:05.403562 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:05.403527    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/95491ac4-0f03-4ee0-a8cd-687621cbfa85-istio-podinfo\") pod \"e2e-mt-7e6fe88b-openshift-default-5b64b75c89-8l2zb\" (UID: \"95491ac4-0f03-4ee0-a8cd-687621cbfa85\") " pod="openshift-ingress/e2e-mt-7e6fe88b-openshift-default-5b64b75c89-8l2zb"
Oct 31 00:57:05.403724 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:05.403566    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/95491ac4-0f03-4ee0-a8cd-687621cbfa85-istio-envoy\") pod \"e2e-mt-7e6fe88b-openshift-default-5b64b75c89-8l2zb\" (UID: \"95491ac4-0f03-4ee0-a8cd-687621cbfa85\") " pod="openshift-ingress/e2e-mt-7e6fe88b-openshift-default-5b64b75c89-8l2zb"
Oct 31 00:57:05.403724 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:05.403647    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/95491ac4-0f03-4ee0-a8cd-687621cbfa85-credential-socket\") pod \"e2e-mt-7e6fe88b-openshift-default-5b64b75c89-8l2zb\" (UID: \"95491ac4-0f03-4ee0-a8cd-687621cbfa85\") " pod="openshift-ingress/e2e-mt-7e6fe88b-openshift-default-5b64b75c89-8l2zb"
Oct 31 00:57:05.403830 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:05.403719    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hbwft\" (UniqueName: \"kubernetes.io/projected/95491ac4-0f03-4ee0-a8cd-687621cbfa85-kube-api-access-hbwft\") pod \"e2e-mt-7e6fe88b-openshift-default-5b64b75c89-8l2zb\" (UID: \"95491ac4-0f03-4ee0-a8cd-687621cbfa85\") " pod="openshift-ingress/e2e-mt-7e6fe88b-openshift-default-5b64b75c89-8l2zb"
Oct 31 00:57:05.403830 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:05.403774    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/95491ac4-0f03-4ee0-a8cd-687621cbfa85-workload-socket\") pod \"e2e-mt-7e6fe88b-openshift-default-5b64b75c89-8l2zb\" (UID: \"95491ac4-0f03-4ee0-a8cd-687621cbfa85\") " pod="openshift-ingress/e2e-mt-7e6fe88b-openshift-default-5b64b75c89-8l2zb"
Oct 31 00:57:05.403830 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:05.403792    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/95491ac4-0f03-4ee0-a8cd-687621cbfa85-workload-certs\") pod \"e2e-mt-7e6fe88b-openshift-default-5b64b75c89-8l2zb\" (UID: \"95491ac4-0f03-4ee0-a8cd-687621cbfa85\") " pod="openshift-ingress/e2e-mt-7e6fe88b-openshift-default-5b64b75c89-8l2zb"
Oct 31 00:57:05.403981 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:05.403862    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/95491ac4-0f03-4ee0-a8cd-687621cbfa85-istio-data\") pod \"e2e-mt-7e6fe88b-openshift-default-5b64b75c89-8l2zb\" (UID: \"95491ac4-0f03-4ee0-a8cd-687621cbfa85\") " pod="openshift-ingress/e2e-mt-7e6fe88b-openshift-default-5b64b75c89-8l2zb"
Oct 31 00:57:05.403981 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:05.403801    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/95491ac4-0f03-4ee0-a8cd-687621cbfa85-istio-token\") pod \"e2e-mt-7e6fe88b-openshift-default-5b64b75c89-8l2zb\" (UID: \"95491ac4-0f03-4ee0-a8cd-687621cbfa85\") " pod="openshift-ingress/e2e-mt-7e6fe88b-openshift-default-5b64b75c89-8l2zb"
Oct 31 00:57:05.403981 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:05.403947    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/95491ac4-0f03-4ee0-a8cd-687621cbfa85-istiod-ca-cert\") pod \"e2e-mt-7e6fe88b-openshift-default-5b64b75c89-8l2zb\" (UID: \"95491ac4-0f03-4ee0-a8cd-687621cbfa85\") " pod="openshift-ingress/e2e-mt-7e6fe88b-openshift-default-5b64b75c89-8l2zb"
Oct 31 00:57:05.404120 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:05.403990    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/95491ac4-0f03-4ee0-a8cd-687621cbfa85-credential-socket\") pod \"e2e-mt-7e6fe88b-openshift-default-5b64b75c89-8l2zb\" (UID: \"95491ac4-0f03-4ee0-a8cd-687621cbfa85\") " pod="openshift-ingress/e2e-mt-7e6fe88b-openshift-default-5b64b75c89-8l2zb"
Oct 31 00:57:05.404646 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:05.404627    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/95491ac4-0f03-4ee0-a8cd-687621cbfa85-istiod-ca-cert\") pod \"e2e-mt-7e6fe88b-openshift-default-5b64b75c89-8l2zb\" (UID: \"95491ac4-0f03-4ee0-a8cd-687621cbfa85\") " pod="openshift-ingress/e2e-mt-7e6fe88b-openshift-default-5b64b75c89-8l2zb"
Oct 31 00:57:05.406046 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:05.406016    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/95491ac4-0f03-4ee0-a8cd-687621cbfa85-istio-envoy\") pod \"e2e-mt-7e6fe88b-openshift-default-5b64b75c89-8l2zb\" (UID: \"95491ac4-0f03-4ee0-a8cd-687621cbfa85\") " pod="openshift-ingress/e2e-mt-7e6fe88b-openshift-default-5b64b75c89-8l2zb"
Oct 31 00:57:05.406545 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:05.406516    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/95491ac4-0f03-4ee0-a8cd-687621cbfa85-istio-podinfo\") pod \"e2e-mt-7e6fe88b-openshift-default-5b64b75c89-8l2zb\" (UID: \"95491ac4-0f03-4ee0-a8cd-687621cbfa85\") " pod="openshift-ingress/e2e-mt-7e6fe88b-openshift-default-5b64b75c89-8l2zb"
Oct 31 00:57:05.463492 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:05.463407    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/95491ac4-0f03-4ee0-a8cd-687621cbfa85-istio-token\") pod \"e2e-mt-7e6fe88b-openshift-default-5b64b75c89-8l2zb\" (UID: \"95491ac4-0f03-4ee0-a8cd-687621cbfa85\") " pod="openshift-ingress/e2e-mt-7e6fe88b-openshift-default-5b64b75c89-8l2zb"
Oct 31 00:57:05.463825 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:05.463509    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hbwft\" (UniqueName: \"kubernetes.io/projected/95491ac4-0f03-4ee0-a8cd-687621cbfa85-kube-api-access-hbwft\") pod \"e2e-mt-7e6fe88b-openshift-default-5b64b75c89-8l2zb\" (UID: \"95491ac4-0f03-4ee0-a8cd-687621cbfa85\") " pod="openshift-ingress/e2e-mt-7e6fe88b-openshift-default-5b64b75c89-8l2zb"
Oct 31 00:57:05.508300 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:05.508266    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-7e6fe88b-openshift-default-5b64b75c89-8l2zb"
Oct 31 00:57:05.928097 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:05.928071    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-7e6fe88b-openshift-default-5b64b75c89-8l2zb"]
Oct 31 00:57:05.928788 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:57:05.928763    4284 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod95491ac4_0f03_4ee0_a8cd_687621cbfa85.slice/crio-54d29ff0addadd8cc978e650c3161752c46c63bd2f556a8cdf850def4095d306 WatchSource:0}: Error finding container 54d29ff0addadd8cc978e650c3161752c46c63bd2f556a8cdf850def4095d306: Status 404 returned error can't find the container with id 54d29ff0addadd8cc978e650c3161752c46c63bd2f556a8cdf850def4095d306
Oct 31 00:57:05.931461 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:05.931429    4284 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 00:57:05.931570 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:05.931510    4284 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 00:57:05.931570 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:05.931547    4284 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 00:57:05.965128 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:05.965100    4284 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="804be1f8-9d63-4f09-9199-b90409a88db5" path="/var/lib/kubelet/pods/804be1f8-9d63-4f09-9199-b90409a88db5/volumes"
Oct 31 00:57:05.965638 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:05.965621    4284 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d7d574bb-fe1e-4566-8f50-bcf9dabba619" path="/var/lib/kubelet/pods/d7d574bb-fe1e-4566-8f50-bcf9dabba619/volumes"
Oct 31 00:57:06.615631 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:06.615598    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-7e6fe88b-openshift-default-5b64b75c89-8l2zb" event={"ID":"95491ac4-0f03-4ee0-a8cd-687621cbfa85","Type":"ContainerStarted","Data":"3b1157049d2cded92bfb5e01cbb09e205cba94ac6aa06477ca1c8a3ecc67f0b2"}
Oct 31 00:57:06.616043 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:06.615662    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-7e6fe88b-openshift-default-5b64b75c89-8l2zb" event={"ID":"95491ac4-0f03-4ee0-a8cd-687621cbfa85","Type":"ContainerStarted","Data":"54d29ff0addadd8cc978e650c3161752c46c63bd2f556a8cdf850def4095d306"}
Oct 31 00:57:06.813236 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:06.801387    4284 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-7e6fe88b-openshift-default-5b64b75c89-8l2zb" podStartSLOduration=1.801360997 podStartE2EDuration="1.801360997s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 00:57:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:57:06.784842471 +0000 UTC m=+1740.929849387" watchObservedRunningTime="2026-10-31 00:57:06.801360997 +0000 UTC m=+1740.946367915"
Oct 31 00:57:07.509425 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:07.509373    4284 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-7e6fe88b-openshift-default-5b64b75c89-8l2zb"
Oct 31 00:57:07.511411 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:07.511380    4284 patch_prober.go:29] interesting pod/e2e-mt-7e6fe88b-openshift-default-5b64b75c89-8l2zb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.110:15021/healthz/ready\": dial tcp 10.134.0.110:15021: connect: connection refused" start-of-body=
Oct 31 00:57:07.511578 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:07.511433    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-7e6fe88b-openshift-default-5b64b75c89-8l2zb" podUID="95491ac4-0f03-4ee0-a8cd-687621cbfa85" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.110:15021/healthz/ready\": dial tcp 10.134.0.110:15021: connect: connection refused"
Oct 31 00:57:08.224264 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:08.224196    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-e942ab-7979b88bd7-s4r4v"]
Oct 31 00:57:08.346762 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:08.346732    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-e942ab-7db488f9ff-qqfk7"]
Oct 31 00:57:08.509715 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:08.509637    4284 patch_prober.go:29] interesting pod/e2e-mt-7e6fe88b-openshift-default-5b64b75c89-8l2zb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.110:15021/healthz/ready\": dial tcp 10.134.0.110:15021: connect: connection refused" start-of-body=
Oct 31 00:57:08.509874 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:08.509704    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-7e6fe88b-openshift-default-5b64b75c89-8l2zb" podUID="95491ac4-0f03-4ee0-a8cd-687621cbfa85" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.110:15021/healthz/ready\": dial tcp 10.134.0.110:15021: connect: connection refused"
Oct 31 00:57:08.609747 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:08.609705    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:57:08.762513 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:08.762425    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:57:09.041681 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:09.041569    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:57:09.510391 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:09.510343    4284 patch_prober.go:29] interesting pod/e2e-mt-7e6fe88b-openshift-default-5b64b75c89-8l2zb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.110:15021/healthz/ready\": dial tcp 10.134.0.110:15021: connect: connection refused" start-of-body=
Oct 31 00:57:09.510836 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:09.510400    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-7e6fe88b-openshift-default-5b64b75c89-8l2zb" podUID="95491ac4-0f03-4ee0-a8cd-687621cbfa85" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.110:15021/healthz/ready\": dial tcp 10.134.0.110:15021: connect: connection refused"
Oct 31 00:57:10.509699 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:10.509661    4284 patch_prober.go:29] interesting pod/e2e-mt-7e6fe88b-openshift-default-5b64b75c89-8l2zb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.110:15021/healthz/ready\": dial tcp 10.134.0.110:15021: connect: connection refused" start-of-body=
Oct 31 00:57:10.509868 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:10.509778    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-7e6fe88b-openshift-default-5b64b75c89-8l2zb" podUID="95491ac4-0f03-4ee0-a8cd-687621cbfa85" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.110:15021/healthz/ready\": dial tcp 10.134.0.110:15021: connect: connection refused"
Oct 31 00:57:11.509735 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:11.509696    4284 patch_prober.go:29] interesting pod/e2e-mt-7e6fe88b-openshift-default-5b64b75c89-8l2zb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.110:15021/healthz/ready\": dial tcp 10.134.0.110:15021: connect: connection refused" start-of-body=
Oct 31 00:57:11.510164 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:11.509793    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-7e6fe88b-openshift-default-5b64b75c89-8l2zb" podUID="95491ac4-0f03-4ee0-a8cd-687621cbfa85" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.110:15021/healthz/ready\": dial tcp 10.134.0.110:15021: connect: connection refused"
Oct 31 00:57:12.508889 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:12.508851    4284 patch_prober.go:29] interesting pod/e2e-mt-7e6fe88b-openshift-default-5b64b75c89-8l2zb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.110:15021/healthz/ready\": dial tcp 10.134.0.110:15021: connect: connection refused" start-of-body=
Oct 31 00:57:12.509070 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:12.508918    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-7e6fe88b-openshift-default-5b64b75c89-8l2zb" podUID="95491ac4-0f03-4ee0-a8cd-687621cbfa85" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.110:15021/healthz/ready\": dial tcp 10.134.0.110:15021: connect: connection refused"
Oct 31 00:57:12.532020 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:12.531986    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:57:12.808289 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:12.800652    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:57:12.975512 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:12.975467    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:57:13.241334 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:13.241273    4284 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-b-e942ab-7979b88bd7-s4r4v" podUID="c940cc14-8405-40e8-8698-db68813e6e2d" containerName="payload-processing" containerID="cri-o://63ab012d55a3f78b825b32ae20b3c20285a6259463ed910a3367d69c4569f449" gracePeriod=25
Oct 31 00:57:13.291177 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:13.290833    4284 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-b647c3-5d89c86dd7-s999b"]
Oct 31 00:57:13.360536 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:13.319954    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:57:13.422816 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:13.394473    4284 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-b-e942ab-7db488f9ff-qqfk7" podUID="35f802b7-40d6-4f42-a069-941c04f99562" containerName="payload-pre-processing" containerID="cri-o://95747a97a4f778a59e50dddc240abcc1c7962ee95828a801341672fc7c01ee10" gracePeriod=25
Oct 31 00:57:13.478761 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:13.477083    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-b647c3-5d89c86dd7-s999b"]
Oct 31 00:57:13.478761 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:13.477246    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-b647c3-5d89c86dd7-s999b"
Oct 31 00:57:13.568297 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:13.542728    4284 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-7e6fe88b-openshift-default-5b64b75c89-8l2zb"
Oct 31 00:57:13.568297 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:13.543637    4284 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-a-b647c3\""
Oct 31 00:57:13.571797 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:13.570206    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-7e6fe88b-openshift-default-5b64b75c89-8l2zb"
Oct 31 00:57:13.571797 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:13.571156    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z24ns\" (UniqueName: \"kubernetes.io/projected/8e9971e4-4766-4644-bf14-341b643426d9-kube-api-access-z24ns\") pod \"payload-pre-processing-e2e-ipp-a-b647c3-5d89c86dd7-s999b\" (UID: \"8e9971e4-4766-4644-bf14-341b643426d9\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-b647c3-5d89c86dd7-s999b"
Oct 31 00:57:13.573282 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:13.572756    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-7e6fe88b-openshift-default-5b64b75c89-8l2zb"
Oct 31 00:57:13.573282 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:57:13.572905    4284 logging.go:55] [core] [Channel #2702 SubChannel #2703]grpc: addrConn.createTransport failed to connect to {Addr: "10.134.0.93:50052", ServerName: "10.134.0.93:50052", BalancerAttributes: {"<%!p(pickfirstleaf.managedByPickfirstKeyType={})>": "<%!p(bool=true)>" }}. Err: connection error: desc = "transport: Error while dialing: dial tcp 10.134.0.93:50052: connect: connection refused"
Oct 31 00:57:13.578469 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:13.578438    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8e9971e4-4766-4644-bf14-341b643426d9-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-b647c3-5d89c86dd7-s999b\" (UID: \"8e9971e4-4766-4644-bf14-341b643426d9\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-b647c3-5d89c86dd7-s999b"
Oct 31 00:57:13.595523 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:13.592469    4284 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-a-b647c3-dockercfg-8zjxk\""
Oct 31 00:57:13.774246 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:13.687163    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z24ns\" (UniqueName: \"kubernetes.io/projected/8e9971e4-4766-4644-bf14-341b643426d9-kube-api-access-z24ns\") pod \"payload-pre-processing-e2e-ipp-a-b647c3-5d89c86dd7-s999b\" (UID: \"8e9971e4-4766-4644-bf14-341b643426d9\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-b647c3-5d89c86dd7-s999b"
Oct 31 00:57:13.774246 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:13.687311    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8e9971e4-4766-4644-bf14-341b643426d9-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-b647c3-5d89c86dd7-s999b\" (UID: \"8e9971e4-4766-4644-bf14-341b643426d9\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-b647c3-5d89c86dd7-s999b"
Oct 31 00:57:13.774246 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:13.688355    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8e9971e4-4766-4644-bf14-341b643426d9-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-b647c3-5d89c86dd7-s999b\" (UID: \"8e9971e4-4766-4644-bf14-341b643426d9\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-b647c3-5d89c86dd7-s999b"
Oct 31 00:57:13.833240 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:13.833144    4284 generic.go:356] "Generic (PLEG): container finished" podID="35f802b7-40d6-4f42-a069-941c04f99562" containerID="95747a97a4f778a59e50dddc240abcc1c7962ee95828a801341672fc7c01ee10" exitCode=0
Oct 31 00:57:13.833644 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:13.833418    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-e942ab-7db488f9ff-qqfk7" event={"ID":"35f802b7-40d6-4f42-a069-941c04f99562","Type":"ContainerDied","Data":"95747a97a4f778a59e50dddc240abcc1c7962ee95828a801341672fc7c01ee10"}
Oct 31 00:57:13.844380 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:13.844023    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z24ns\" (UniqueName: \"kubernetes.io/projected/8e9971e4-4766-4644-bf14-341b643426d9-kube-api-access-z24ns\") pod \"payload-pre-processing-e2e-ipp-a-b647c3-5d89c86dd7-s999b\" (UID: \"8e9971e4-4766-4644-bf14-341b643426d9\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-b647c3-5d89c86dd7-s999b"
Oct 31 00:57:13.844380 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:13.844232    4284 generic.go:356] "Generic (PLEG): container finished" podID="c940cc14-8405-40e8-8698-db68813e6e2d" containerID="63ab012d55a3f78b825b32ae20b3c20285a6259463ed910a3367d69c4569f449" exitCode=0
Oct 31 00:57:13.844380 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:13.844356    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-e942ab-7979b88bd7-s4r4v" event={"ID":"c940cc14-8405-40e8-8698-db68813e6e2d","Type":"ContainerDied","Data":"63ab012d55a3f78b825b32ae20b3c20285a6259463ed910a3367d69c4569f449"}
Oct 31 00:57:13.897192 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:13.897154    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-e942ab-7979b88bd7-s4r4v"
Oct 31 00:57:13.898882 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:13.898862    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-b647c3-5d89c86dd7-s999b"
Oct 31 00:57:13.926943 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:13.926906    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c940cc14-8405-40e8-8698-db68813e6e2d-kube-api-access-75czz\" (UniqueName: \"kubernetes.io/projected/c940cc14-8405-40e8-8698-db68813e6e2d-kube-api-access-75czz\") pod \"c940cc14-8405-40e8-8698-db68813e6e2d\" (UID: \"c940cc14-8405-40e8-8698-db68813e6e2d\") "
Oct 31 00:57:13.927438 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:13.927411    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c940cc14-8405-40e8-8698-db68813e6e2d-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c940cc14-8405-40e8-8698-db68813e6e2d-plugins-config-volume\") pod \"c940cc14-8405-40e8-8698-db68813e6e2d\" (UID: \"c940cc14-8405-40e8-8698-db68813e6e2d\") "
Oct 31 00:57:13.929487 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:13.929459    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c940cc14-8405-40e8-8698-db68813e6e2d-plugins-config-volume" pod "c940cc14-8405-40e8-8698-db68813e6e2d" (UID: "c940cc14-8405-40e8-8698-db68813e6e2d"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 00:57:13.931522 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:13.931498    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c940cc14-8405-40e8-8698-db68813e6e2d-kube-api-access-75czz" pod "c940cc14-8405-40e8-8698-db68813e6e2d" (UID: "c940cc14-8405-40e8-8698-db68813e6e2d"). InnerVolumeSpecName "kube-api-access-75czz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 00:57:14.002795 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:14.002759    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-e942ab-7db488f9ff-qqfk7"
Oct 31 00:57:14.029580 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:14.029552    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/35f802b7-40d6-4f42-a069-941c04f99562-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/35f802b7-40d6-4f42-a069-941c04f99562-plugins-config-volume\") pod \"35f802b7-40d6-4f42-a069-941c04f99562\" (UID: \"35f802b7-40d6-4f42-a069-941c04f99562\") "
Oct 31 00:57:14.029756 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:14.029679    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/35f802b7-40d6-4f42-a069-941c04f99562-kube-api-access-5d8dt\" (UniqueName: \"kubernetes.io/projected/35f802b7-40d6-4f42-a069-941c04f99562-kube-api-access-5d8dt\") pod \"35f802b7-40d6-4f42-a069-941c04f99562\" (UID: \"35f802b7-40d6-4f42-a069-941c04f99562\") "
Oct 31 00:57:14.029999 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:14.029972    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/35f802b7-40d6-4f42-a069-941c04f99562-plugins-config-volume" pod "35f802b7-40d6-4f42-a069-941c04f99562" (UID: "35f802b7-40d6-4f42-a069-941c04f99562"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 00:57:14.030207 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:14.030186    4284 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/35f802b7-40d6-4f42-a069-941c04f99562-plugins-config-volume\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:57:14.030315 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:14.030238    4284 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-75czz\" (UniqueName: \"kubernetes.io/projected/c940cc14-8405-40e8-8698-db68813e6e2d-kube-api-access-75czz\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:57:14.030315 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:14.030255    4284 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c940cc14-8405-40e8-8698-db68813e6e2d-plugins-config-volume\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:57:14.031978 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:14.031958    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/35f802b7-40d6-4f42-a069-941c04f99562-kube-api-access-5d8dt" pod "35f802b7-40d6-4f42-a069-941c04f99562" (UID: "35f802b7-40d6-4f42-a069-941c04f99562"). InnerVolumeSpecName "kube-api-access-5d8dt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 00:57:14.131501 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:14.131410    4284 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5d8dt\" (UniqueName: \"kubernetes.io/projected/35f802b7-40d6-4f42-a069-941c04f99562-kube-api-access-5d8dt\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:57:14.298661 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:14.298633    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-b647c3-5d89c86dd7-s999b"]
Oct 31 00:57:14.299515 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:57:14.299465    4284 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8e9971e4_4766_4644_bf14_341b643426d9.slice/crio-028ac02c29584f1b391f236e96a5760362d32fed72db9fb94959abada1ec70be WatchSource:0}: Error finding container 028ac02c29584f1b391f236e96a5760362d32fed72db9fb94959abada1ec70be: Status 404 returned error can't find the container with id 028ac02c29584f1b391f236e96a5760362d32fed72db9fb94959abada1ec70be
Oct 31 00:57:14.349534 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:14.349506    4284 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-ac952a-74bb985447-kbvfk"]
Oct 31 00:57:14.350129 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:14.350114    4284 cpu_manager.go:414] "containerMap: removing container" podUID="c940cc14-8405-40e8-8698-db68813e6e2d" containerName="payload-processing"
Oct 31 00:57:14.350177 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:14.350131    4284 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c940cc14-8405-40e8-8698-db68813e6e2d" containerName="payload-processing"
Oct 31 00:57:14.350177 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:14.350161    4284 cpu_manager.go:414] "containerMap: removing container" podUID="35f802b7-40d6-4f42-a069-941c04f99562" containerName="payload-pre-processing"
Oct 31 00:57:14.350177 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:14.350167    4284 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="35f802b7-40d6-4f42-a069-941c04f99562" containerName="payload-pre-processing"
Oct 31 00:57:14.350371 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:14.350359    4284 memory_manager.go:365] "RemoveStaleState removing state" podUID="35f802b7-40d6-4f42-a069-941c04f99562" containerName="payload-pre-processing"
Oct 31 00:57:14.350409 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:14.350373    4284 memory_manager.go:365] "RemoveStaleState removing state" podUID="c940cc14-8405-40e8-8698-db68813e6e2d" containerName="payload-processing"
Oct 31 00:57:14.354269 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:14.354248    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-ac952a-74bb985447-kbvfk"
Oct 31 00:57:14.405293 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:14.405098    4284 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-a-ac952a\""
Oct 31 00:57:14.405293 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:14.405258    4284 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-a-ac952a-dockercfg-g75xn\""
Oct 31 00:57:14.408416 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:14.408390    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-ac952a-74bb985447-kbvfk"]
Oct 31 00:57:14.435617 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:14.435586    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zq4kw\" (UniqueName: \"kubernetes.io/projected/ab8bfde5-7ca3-40a0-b8ec-3a50139d0dd5-kube-api-access-zq4kw\") pod \"payload-pre-processing-e2e-ipp-a-ac952a-74bb985447-kbvfk\" (UID: \"ab8bfde5-7ca3-40a0-b8ec-3a50139d0dd5\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-ac952a-74bb985447-kbvfk"
Oct 31 00:57:14.435783 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:14.435627    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ab8bfde5-7ca3-40a0-b8ec-3a50139d0dd5-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-ac952a-74bb985447-kbvfk\" (UID: \"ab8bfde5-7ca3-40a0-b8ec-3a50139d0dd5\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-ac952a-74bb985447-kbvfk"
Oct 31 00:57:14.536896 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:14.536860    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zq4kw\" (UniqueName: \"kubernetes.io/projected/ab8bfde5-7ca3-40a0-b8ec-3a50139d0dd5-kube-api-access-zq4kw\") pod \"payload-pre-processing-e2e-ipp-a-ac952a-74bb985447-kbvfk\" (UID: \"ab8bfde5-7ca3-40a0-b8ec-3a50139d0dd5\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-ac952a-74bb985447-kbvfk"
Oct 31 00:57:14.537051 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:14.536904    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ab8bfde5-7ca3-40a0-b8ec-3a50139d0dd5-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-ac952a-74bb985447-kbvfk\" (UID: \"ab8bfde5-7ca3-40a0-b8ec-3a50139d0dd5\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-ac952a-74bb985447-kbvfk"
Oct 31 00:57:14.537624 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:14.537604    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ab8bfde5-7ca3-40a0-b8ec-3a50139d0dd5-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-ac952a-74bb985447-kbvfk\" (UID: \"ab8bfde5-7ca3-40a0-b8ec-3a50139d0dd5\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-ac952a-74bb985447-kbvfk"
Oct 31 00:57:14.568792 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:14.568755    4284 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-api-b-e942ab-7979b88bd7-s4r4v" podUID="c940cc14-8405-40e8-8698-db68813e6e2d" containerName="payload-processing" probeResult="failure" output="timeout: failed to connect service \"10.134.0.93:50052\" within 1s: context deadline exceeded"
Oct 31 00:57:14.595700 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:14.595677    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zq4kw\" (UniqueName: \"kubernetes.io/projected/ab8bfde5-7ca3-40a0-b8ec-3a50139d0dd5-kube-api-access-zq4kw\") pod \"payload-pre-processing-e2e-ipp-a-ac952a-74bb985447-kbvfk\" (UID: \"ab8bfde5-7ca3-40a0-b8ec-3a50139d0dd5\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-ac952a-74bb985447-kbvfk"
Oct 31 00:57:14.682388 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:14.682359    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-ac952a-74bb985447-kbvfk"
Oct 31 00:57:14.872968 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:14.872856    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-e942ab-7979b88bd7-s4r4v"
Oct 31 00:57:14.873368 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:14.873338    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-e942ab-7979b88bd7-s4r4v" event={"ID":"c940cc14-8405-40e8-8698-db68813e6e2d","Type":"ContainerDied","Data":"429774e68afad07dfddf800cf55ca81c324d33e18377c2d9ec245ccf99b5eaa7"}
Oct 31 00:57:14.873604 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:14.873574    4284 scope.go:122] "RemoveContainer" containerID="63ab012d55a3f78b825b32ae20b3c20285a6259463ed910a3367d69c4569f449"
Oct 31 00:57:14.877015 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:14.876983    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-e942ab-7db488f9ff-qqfk7" event={"ID":"35f802b7-40d6-4f42-a069-941c04f99562","Type":"ContainerDied","Data":"33ff7fd58c03e3407f8890ecb1756a45a74a6a06d072253c367de8d15c92f51e"}
Oct 31 00:57:14.877152 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:14.877094    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-e942ab-7db488f9ff-qqfk7"
Oct 31 00:57:14.882253 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:14.882187    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-b647c3-5d89c86dd7-s999b" event={"ID":"8e9971e4-4766-4644-bf14-341b643426d9","Type":"ContainerStarted","Data":"c8fbf267f6a7e73cd18abd0d74d6f8cda70734e2c386cb72e04e2f67d3fba0ff"}
Oct 31 00:57:14.882484 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:14.882465    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-b647c3-5d89c86dd7-s999b"
Oct 31 00:57:14.882565 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:14.882507    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-b647c3-5d89c86dd7-s999b" event={"ID":"8e9971e4-4766-4644-bf14-341b643426d9","Type":"ContainerStarted","Data":"028ac02c29584f1b391f236e96a5760362d32fed72db9fb94959abada1ec70be"}
Oct 31 00:57:14.887902 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:14.887885    4284 scope.go:122] "RemoveContainer" containerID="95747a97a4f778a59e50dddc240abcc1c7962ee95828a801341672fc7c01ee10"
Oct 31 00:57:15.032795 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:15.032720    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-e942ab-7979b88bd7-s4r4v"]
Oct 31 00:57:15.038139 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:15.038113    4284 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-e942ab-7979b88bd7-s4r4v"]
Oct 31 00:57:15.053791 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:15.053766    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-ac952a-74bb985447-kbvfk"]
Oct 31 00:57:15.088622 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:57:15.088593    4284 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podab8bfde5_7ca3_40a0_b8ec_3a50139d0dd5.slice/crio-717b12272849aa5599700115ebb35c8ab127d70574a38cbc44689e5f09c2a44e WatchSource:0}: Error finding container 717b12272849aa5599700115ebb35c8ab127d70574a38cbc44689e5f09c2a44e: Status 404 returned error can't find the container with id 717b12272849aa5599700115ebb35c8ab127d70574a38cbc44689e5f09c2a44e
Oct 31 00:57:15.143062 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:15.143028    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-e942ab-7db488f9ff-qqfk7"]
Oct 31 00:57:15.151297 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:15.151273    4284 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-e942ab-7db488f9ff-qqfk7"]
Oct 31 00:57:15.260056 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:15.260005    4284 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-b647c3-5d89c86dd7-s999b" podStartSLOduration=2.25998993 podStartE2EDuration="2.25998993s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 00:57:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:57:15.258407036 +0000 UTC m=+1749.403413955" watchObservedRunningTime="2026-10-31 00:57:15.25998993 +0000 UTC m=+1749.404996846"
Oct 31 00:57:15.891122 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:15.891085    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-ac952a-74bb985447-kbvfk" event={"ID":"ab8bfde5-7ca3-40a0-b8ec-3a50139d0dd5","Type":"ContainerStarted","Data":"717b12272849aa5599700115ebb35c8ab127d70574a38cbc44689e5f09c2a44e"}
Oct 31 00:57:15.964233 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:15.964181    4284 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="35f802b7-40d6-4f42-a069-941c04f99562" path="/var/lib/kubelet/pods/35f802b7-40d6-4f42-a069-941c04f99562/volumes"
Oct 31 00:57:15.964693 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:15.964672    4284 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c940cc14-8405-40e8-8698-db68813e6e2d" path="/var/lib/kubelet/pods/c940cc14-8405-40e8-8698-db68813e6e2d/volumes"
Oct 31 00:57:16.036978 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:16.036933    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:57:16.315600 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:16.315567    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:57:16.468358 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:16.468273    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:57:17.017039 ip-10-0-132-173 kubenswrapper[4284]: E1031 00:57:17.016999    4284 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fd8sp" containerName="extractor"
Oct 31 00:57:18.931240 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:18.931168    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-ac952a-74bb985447-kbvfk" event={"ID":"ab8bfde5-7ca3-40a0-b8ec-3a50139d0dd5","Type":"ContainerStarted","Data":"237cadc2421c9dce2ea5bc6b3928a67f404ef1d0ea2a3e72824968da3f009b59"}
Oct 31 00:57:18.931692 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:18.931284    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-ac952a-74bb985447-kbvfk"
Oct 31 00:57:19.046083 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:19.046004    4284 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-ac952a-74bb985447-kbvfk" podStartSLOduration=1.919884187 podStartE2EDuration="5.045986819s" totalImagesPullingTime="3.126102632s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 00:57:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:57:19.042667787 +0000 UTC m=+1753.187674704" watchObservedRunningTime="2026-10-31 00:57:19.045986819 +0000 UTC m=+1753.190993735"
Oct 31 00:57:20.138962 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:20.138924    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-7e6fe88b-55755c67c7-rct6j"]
Oct 31 00:57:20.139441 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:20.139144    4284 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-7e6fe88b-55755c67c7-rct6j" podUID="376421f4-5e4b-457f-bf37-a8f794171def" containerName="maas-api" containerID="cri-o://ae5b5fa52c6375596d189375d0b77597b8bc17f992ade7ee1940fa52aed83226" gracePeriod=30
Oct 31 00:57:20.226884 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:20.226840    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:57:20.405541 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:20.405522    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-7e6fe88b-55755c67c7-rct6j"
Oct 31 00:57:20.525544 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:20.525510    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/376421f4-5e4b-457f-bf37-a8f794171def-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/376421f4-5e4b-457f-bf37-a8f794171def-openshift-service-ca\") pod \"376421f4-5e4b-457f-bf37-a8f794171def\" (UID: \"376421f4-5e4b-457f-bf37-a8f794171def\") "
Oct 31 00:57:20.525732 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:20.525574    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/376421f4-5e4b-457f-bf37-a8f794171def-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/376421f4-5e4b-457f-bf37-a8f794171def-maas-api-tls\") pod \"376421f4-5e4b-457f-bf37-a8f794171def\" (UID: \"376421f4-5e4b-457f-bf37-a8f794171def\") "
Oct 31 00:57:20.525880 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:20.525862    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/376421f4-5e4b-457f-bf37-a8f794171def-metrics-certs\" (UniqueName: \"kubernetes.io/secret/376421f4-5e4b-457f-bf37-a8f794171def-metrics-certs\") pod \"376421f4-5e4b-457f-bf37-a8f794171def\" (UID: \"376421f4-5e4b-457f-bf37-a8f794171def\") "
Oct 31 00:57:20.525965 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:20.525907    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/376421f4-5e4b-457f-bf37-a8f794171def-kube-api-access-q6hbp\" (UniqueName: \"kubernetes.io/projected/376421f4-5e4b-457f-bf37-a8f794171def-kube-api-access-q6hbp\") pod \"376421f4-5e4b-457f-bf37-a8f794171def\" (UID: \"376421f4-5e4b-457f-bf37-a8f794171def\") "
Oct 31 00:57:20.525965 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:20.525924    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/376421f4-5e4b-457f-bf37-a8f794171def-openshift-service-ca" pod "376421f4-5e4b-457f-bf37-a8f794171def" (UID: "376421f4-5e4b-457f-bf37-a8f794171def"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 00:57:20.526530 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:20.526507    4284 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/376421f4-5e4b-457f-bf37-a8f794171def-openshift-service-ca\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:57:20.528705 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:20.528669    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/376421f4-5e4b-457f-bf37-a8f794171def-maas-api-tls" pod "376421f4-5e4b-457f-bf37-a8f794171def" (UID: "376421f4-5e4b-457f-bf37-a8f794171def"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 00:57:20.529334 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:20.529304    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/376421f4-5e4b-457f-bf37-a8f794171def-metrics-certs" pod "376421f4-5e4b-457f-bf37-a8f794171def" (UID: "376421f4-5e4b-457f-bf37-a8f794171def"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 00:57:20.529468 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:20.529446    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/376421f4-5e4b-457f-bf37-a8f794171def-kube-api-access-q6hbp" pod "376421f4-5e4b-457f-bf37-a8f794171def" (UID: "376421f4-5e4b-457f-bf37-a8f794171def"). InnerVolumeSpecName "kube-api-access-q6hbp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 00:57:20.628439 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:20.628404    4284 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/376421f4-5e4b-457f-bf37-a8f794171def-metrics-certs\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:57:20.628439 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:20.628433    4284 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q6hbp\" (UniqueName: \"kubernetes.io/projected/376421f4-5e4b-457f-bf37-a8f794171def-kube-api-access-q6hbp\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:57:20.628439 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:20.628443    4284 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/376421f4-5e4b-457f-bf37-a8f794171def-maas-api-tls\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:57:20.944648 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:20.944617    4284 generic.go:356] "Generic (PLEG): container finished" podID="376421f4-5e4b-457f-bf37-a8f794171def" containerID="ae5b5fa52c6375596d189375d0b77597b8bc17f992ade7ee1940fa52aed83226" exitCode=0
Oct 31 00:57:20.944824 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:20.944676    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-7e6fe88b-55755c67c7-rct6j"
Oct 31 00:57:20.944824 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:20.944702    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-7e6fe88b-55755c67c7-rct6j" event={"ID":"376421f4-5e4b-457f-bf37-a8f794171def","Type":"ContainerDied","Data":"ae5b5fa52c6375596d189375d0b77597b8bc17f992ade7ee1940fa52aed83226"}
Oct 31 00:57:20.944824 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:20.944759    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-7e6fe88b-55755c67c7-rct6j" event={"ID":"376421f4-5e4b-457f-bf37-a8f794171def","Type":"ContainerDied","Data":"c3668db7c120a1c31eb0df7fcd1abe0f80f68d807f712072a1c67497d12966fe"}
Oct 31 00:57:20.944824 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:20.944788    4284 scope.go:122] "RemoveContainer" containerID="ae5b5fa52c6375596d189375d0b77597b8bc17f992ade7ee1940fa52aed83226"
Oct 31 00:57:20.955181 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:20.955156    4284 scope.go:122] "RemoveContainer" containerID="ae5b5fa52c6375596d189375d0b77597b8bc17f992ade7ee1940fa52aed83226"
Oct 31 00:57:20.955573 ip-10-0-132-173 kubenswrapper[4284]: E1031 00:57:20.955546    4284 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ae5b5fa52c6375596d189375d0b77597b8bc17f992ade7ee1940fa52aed83226\": container with ID starting with ae5b5fa52c6375596d189375d0b77597b8bc17f992ade7ee1940fa52aed83226 not found: ID does not exist" containerID="ae5b5fa52c6375596d189375d0b77597b8bc17f992ade7ee1940fa52aed83226"
Oct 31 00:57:20.955672 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:20.955579    4284 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ae5b5fa52c6375596d189375d0b77597b8bc17f992ade7ee1940fa52aed83226"} err="failed to get container status \"ae5b5fa52c6375596d189375d0b77597b8bc17f992ade7ee1940fa52aed83226\": rpc error: code = NotFound desc = could not find container \"ae5b5fa52c6375596d189375d0b77597b8bc17f992ade7ee1940fa52aed83226\": container with ID starting with ae5b5fa52c6375596d189375d0b77597b8bc17f992ade7ee1940fa52aed83226 not found: ID does not exist"
Oct 31 00:57:21.107237 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:21.106858    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-7e6fe88b-55755c67c7-rct6j"]
Oct 31 00:57:21.112171 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:21.112132    4284 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-7e6fe88b-55755c67c7-rct6j"]
Oct 31 00:57:21.761018 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:21.760978    4284 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-cb2b8a-77c6c7fdbf-mzb85"]
Oct 31 00:57:21.761865 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:21.761774    4284 cpu_manager.go:414] "containerMap: removing container" podUID="376421f4-5e4b-457f-bf37-a8f794171def" containerName="maas-api"
Oct 31 00:57:21.761865 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:21.761799    4284 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="376421f4-5e4b-457f-bf37-a8f794171def" containerName="maas-api"
Oct 31 00:57:21.762034 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:21.762000    4284 memory_manager.go:365] "RemoveStaleState removing state" podUID="376421f4-5e4b-457f-bf37-a8f794171def" containerName="maas-api"
Oct 31 00:57:21.765618 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:21.765596    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-cb2b8a-77c6c7fdbf-mzb85"
Oct 31 00:57:21.812779 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:21.812745    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:57:21.815485 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:21.815460    4284 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-b-cb2b8a-serving-cert\""
Oct 31 00:57:21.868759 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:21.868728    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-cb2b8a-77c6c7fdbf-mzb85"]
Oct 31 00:57:21.876018 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:21.875983    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/aa9a9164-762f-4869-a04d-5cd82e3d8bfa-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-cb2b8a-77c6c7fdbf-mzb85\" (UID: \"aa9a9164-762f-4869-a04d-5cd82e3d8bfa\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-cb2b8a-77c6c7fdbf-mzb85"
Oct 31 00:57:21.877602 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:21.877573    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/aa9a9164-762f-4869-a04d-5cd82e3d8bfa-metrics-certs\") pod \"maas-api-e2e-ipp-b-cb2b8a-77c6c7fdbf-mzb85\" (UID: \"aa9a9164-762f-4869-a04d-5cd82e3d8bfa\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-cb2b8a-77c6c7fdbf-mzb85"
Oct 31 00:57:21.877729 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:21.877670    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/aa9a9164-762f-4869-a04d-5cd82e3d8bfa-maas-api-tls\") pod \"maas-api-e2e-ipp-b-cb2b8a-77c6c7fdbf-mzb85\" (UID: \"aa9a9164-762f-4869-a04d-5cd82e3d8bfa\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-cb2b8a-77c6c7fdbf-mzb85"
Oct 31 00:57:21.877890 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:21.877835    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gmgnv\" (UniqueName: \"kubernetes.io/projected/aa9a9164-762f-4869-a04d-5cd82e3d8bfa-kube-api-access-gmgnv\") pod \"maas-api-e2e-ipp-b-cb2b8a-77c6c7fdbf-mzb85\" (UID: \"aa9a9164-762f-4869-a04d-5cd82e3d8bfa\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-cb2b8a-77c6c7fdbf-mzb85"
Oct 31 00:57:21.963580 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:21.963545    4284 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="376421f4-5e4b-457f-bf37-a8f794171def" path="/var/lib/kubelet/pods/376421f4-5e4b-457f-bf37-a8f794171def/volumes"
Oct 31 00:57:21.965733 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:21.965706    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:57:21.980175 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:21.980143    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/aa9a9164-762f-4869-a04d-5cd82e3d8bfa-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-cb2b8a-77c6c7fdbf-mzb85\" (UID: \"aa9a9164-762f-4869-a04d-5cd82e3d8bfa\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-cb2b8a-77c6c7fdbf-mzb85"
Oct 31 00:57:21.980307 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:21.980182    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/aa9a9164-762f-4869-a04d-5cd82e3d8bfa-metrics-certs\") pod \"maas-api-e2e-ipp-b-cb2b8a-77c6c7fdbf-mzb85\" (UID: \"aa9a9164-762f-4869-a04d-5cd82e3d8bfa\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-cb2b8a-77c6c7fdbf-mzb85"
Oct 31 00:57:21.980307 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:21.980266    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/aa9a9164-762f-4869-a04d-5cd82e3d8bfa-maas-api-tls\") pod \"maas-api-e2e-ipp-b-cb2b8a-77c6c7fdbf-mzb85\" (UID: \"aa9a9164-762f-4869-a04d-5cd82e3d8bfa\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-cb2b8a-77c6c7fdbf-mzb85"
Oct 31 00:57:21.980392 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:21.980328    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gmgnv\" (UniqueName: \"kubernetes.io/projected/aa9a9164-762f-4869-a04d-5cd82e3d8bfa-kube-api-access-gmgnv\") pod \"maas-api-e2e-ipp-b-cb2b8a-77c6c7fdbf-mzb85\" (UID: \"aa9a9164-762f-4869-a04d-5cd82e3d8bfa\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-cb2b8a-77c6c7fdbf-mzb85"
Oct 31 00:57:21.980442 ip-10-0-132-173 kubenswrapper[4284]: E1031 00:57:21.980407    4284 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-b-cb2b8a-serving-cert: secret "maas-api-e2e-ipp-b-cb2b8a-serving-cert" not found
Oct 31 00:57:21.980498 ip-10-0-132-173 kubenswrapper[4284]: E1031 00:57:21.980489    4284 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aa9a9164-762f-4869-a04d-5cd82e3d8bfa-maas-api-tls podName:aa9a9164-762f-4869-a04d-5cd82e3d8bfa nodeName:}" failed. No retries permitted until 2026-10-31 00:57:22.480466759 +0000 UTC m=+1756.625473682 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/aa9a9164-762f-4869-a04d-5cd82e3d8bfa-maas-api-tls") pod "maas-api-e2e-ipp-b-cb2b8a-77c6c7fdbf-mzb85" (UID: "aa9a9164-762f-4869-a04d-5cd82e3d8bfa") : secret "maas-api-e2e-ipp-b-cb2b8a-serving-cert" not found
Oct 31 00:57:21.980972 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:21.980950    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/aa9a9164-762f-4869-a04d-5cd82e3d8bfa-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-cb2b8a-77c6c7fdbf-mzb85\" (UID: \"aa9a9164-762f-4869-a04d-5cd82e3d8bfa\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-cb2b8a-77c6c7fdbf-mzb85"
Oct 31 00:57:22.010279 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:22.004955    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/aa9a9164-762f-4869-a04d-5cd82e3d8bfa-metrics-certs\") pod \"maas-api-e2e-ipp-b-cb2b8a-77c6c7fdbf-mzb85\" (UID: \"aa9a9164-762f-4869-a04d-5cd82e3d8bfa\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-cb2b8a-77c6c7fdbf-mzb85"
Oct 31 00:57:22.071260 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:22.071115    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gmgnv\" (UniqueName: \"kubernetes.io/projected/aa9a9164-762f-4869-a04d-5cd82e3d8bfa-kube-api-access-gmgnv\") pod \"maas-api-e2e-ipp-b-cb2b8a-77c6c7fdbf-mzb85\" (UID: \"aa9a9164-762f-4869-a04d-5cd82e3d8bfa\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-cb2b8a-77c6c7fdbf-mzb85"
Oct 31 00:57:22.122671 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:22.122632    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:57:22.287321 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:22.283886    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:57:22.488639 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:22.488600    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/aa9a9164-762f-4869-a04d-5cd82e3d8bfa-maas-api-tls\") pod \"maas-api-e2e-ipp-b-cb2b8a-77c6c7fdbf-mzb85\" (UID: \"aa9a9164-762f-4869-a04d-5cd82e3d8bfa\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-cb2b8a-77c6c7fdbf-mzb85"
Oct 31 00:57:22.491230 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:22.491192    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/aa9a9164-762f-4869-a04d-5cd82e3d8bfa-maas-api-tls\") pod \"maas-api-e2e-ipp-b-cb2b8a-77c6c7fdbf-mzb85\" (UID: \"aa9a9164-762f-4869-a04d-5cd82e3d8bfa\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-cb2b8a-77c6c7fdbf-mzb85"
Oct 31 00:57:22.678390 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:22.678359    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-cb2b8a-77c6c7fdbf-mzb85"
Oct 31 00:57:22.778043 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:22.778006    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:57:23.053587 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:23.053562    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-cb2b8a-77c6c7fdbf-mzb85"]
Oct 31 00:57:23.054281 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:57:23.054253    4284 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaa9a9164_762f_4869_a04d_5cd82e3d8bfa.slice/crio-742e0e54dbd77bb4b241b3f173d1b89b80fbeb5f1feea715a5bcb7b69ca293c3 WatchSource:0}: Error finding container 742e0e54dbd77bb4b241b3f173d1b89b80fbeb5f1feea715a5bcb7b69ca293c3: Status 404 returned error can't find the container with id 742e0e54dbd77bb4b241b3f173d1b89b80fbeb5f1feea715a5bcb7b69ca293c3
Oct 31 00:57:23.965197 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:23.965162    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-cb2b8a-77c6c7fdbf-mzb85" event={"ID":"aa9a9164-762f-4869-a04d-5cd82e3d8bfa","Type":"ContainerStarted","Data":"742e0e54dbd77bb4b241b3f173d1b89b80fbeb5f1feea715a5bcb7b69ca293c3"}
Oct 31 00:57:24.998237 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:24.989570    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-cb2b8a-77c6c7fdbf-mzb85" event={"ID":"aa9a9164-762f-4869-a04d-5cd82e3d8bfa","Type":"ContainerStarted","Data":"6d7206673a82d00907791d9c5b8a234bdf1a8e6f2f6349177aaa8a0dfe1b905b"}
Oct 31 00:57:24.998237 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:24.990764    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-cb2b8a-77c6c7fdbf-mzb85"
Oct 31 00:57:25.111621 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:25.111570    4284 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-cb2b8a-77c6c7fdbf-mzb85" podStartSLOduration=3.206730764 podStartE2EDuration="4.111552984s" totalImagesPullingTime="904.82222ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 00:57:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:57:25.110064885 +0000 UTC m=+1759.255071805" watchObservedRunningTime="2026-10-31 00:57:25.111552984 +0000 UTC m=+1759.256559902"
Oct 31 00:57:25.895048 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:25.895014    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-b647c3-5d89c86dd7-s999b"
Oct 31 00:57:27.351197 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:27.351146    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:57:27.814151 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:27.814105    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:57:29.939392 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:29.939360    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-ac952a-74bb985447-kbvfk"
Oct 31 00:57:32.006488 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:32.006457    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-cb2b8a-77c6c7fdbf-mzb85"
Oct 31 00:57:32.236278 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:32.236241    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:57:37.029877 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:37.029844    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:57:42.027139 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:42.027106    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:57:45.435301 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:45.435253    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:57:45.961562 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:45.961502    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:57:47.016724 ip-10-0-132-173 kubenswrapper[4284]: E1031 00:57:47.016681    4284 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fd8sp" containerName="extractor"
Oct 31 00:57:47.061782 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:47.061750    4284 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-f66180-84487bc787-gjmlg"]
Oct 31 00:57:47.067653 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:47.067627    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-f66180-84487bc787-gjmlg"
Oct 31 00:57:47.122667 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:47.122636    4284 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-a-f66180\""
Oct 31 00:57:47.122862 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:47.122744    4284 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-a-f66180-dockercfg-jhlz6\""
Oct 31 00:57:47.125946 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:47.125914    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-f66180-84487bc787-gjmlg"]
Oct 31 00:57:47.244518 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:47.244478    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f2cb10e0-cf06-4c51-b383-498ea90f2052-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-f66180-84487bc787-gjmlg\" (UID: \"f2cb10e0-cf06-4c51-b383-498ea90f2052\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-f66180-84487bc787-gjmlg"
Oct 31 00:57:47.244704 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:47.244570    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lkcjf\" (UniqueName: \"kubernetes.io/projected/f2cb10e0-cf06-4c51-b383-498ea90f2052-kube-api-access-lkcjf\") pod \"payload-pre-processing-e2e-ipp-a-f66180-84487bc787-gjmlg\" (UID: \"f2cb10e0-cf06-4c51-b383-498ea90f2052\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-f66180-84487bc787-gjmlg"
Oct 31 00:57:47.346449 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:47.346361    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f2cb10e0-cf06-4c51-b383-498ea90f2052-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-f66180-84487bc787-gjmlg\" (UID: \"f2cb10e0-cf06-4c51-b383-498ea90f2052\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-f66180-84487bc787-gjmlg"
Oct 31 00:57:47.346616 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:47.346494    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lkcjf\" (UniqueName: \"kubernetes.io/projected/f2cb10e0-cf06-4c51-b383-498ea90f2052-kube-api-access-lkcjf\") pod \"payload-pre-processing-e2e-ipp-a-f66180-84487bc787-gjmlg\" (UID: \"f2cb10e0-cf06-4c51-b383-498ea90f2052\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-f66180-84487bc787-gjmlg"
Oct 31 00:57:47.347232 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:47.347192    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f2cb10e0-cf06-4c51-b383-498ea90f2052-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-f66180-84487bc787-gjmlg\" (UID: \"f2cb10e0-cf06-4c51-b383-498ea90f2052\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-f66180-84487bc787-gjmlg"
Oct 31 00:57:47.406378 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:47.406342    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lkcjf\" (UniqueName: \"kubernetes.io/projected/f2cb10e0-cf06-4c51-b383-498ea90f2052-kube-api-access-lkcjf\") pod \"payload-pre-processing-e2e-ipp-a-f66180-84487bc787-gjmlg\" (UID: \"f2cb10e0-cf06-4c51-b383-498ea90f2052\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-f66180-84487bc787-gjmlg"
Oct 31 00:57:47.682481 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:47.682448    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-f66180-84487bc787-gjmlg"
Oct 31 00:57:48.064230 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:48.064179    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-f66180-84487bc787-gjmlg"]
Oct 31 00:57:48.065676 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:57:48.065647    4284 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf2cb10e0_cf06_4c51_b383_498ea90f2052.slice/crio-2592808cd02827fa7b0636d41e50dc255b1311b2f2e287e24962a3f685264b75 WatchSource:0}: Error finding container 2592808cd02827fa7b0636d41e50dc255b1311b2f2e287e24962a3f685264b75: Status 404 returned error can't find the container with id 2592808cd02827fa7b0636d41e50dc255b1311b2f2e287e24962a3f685264b75
Oct 31 00:57:48.228179 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:48.228143    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-f66180-84487bc787-gjmlg" event={"ID":"f2cb10e0-cf06-4c51-b383-498ea90f2052","Type":"ContainerStarted","Data":"f4c54fe6db42294aecf7c2cd73993e69c0b3e9bf8e197057b67ee25f8cfd2073"}
Oct 31 00:57:48.228389 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:48.228206    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-f66180-84487bc787-gjmlg" event={"ID":"f2cb10e0-cf06-4c51-b383-498ea90f2052","Type":"ContainerStarted","Data":"2592808cd02827fa7b0636d41e50dc255b1311b2f2e287e24962a3f685264b75"}
Oct 31 00:57:48.228389 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:48.228266    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-f66180-84487bc787-gjmlg"
Oct 31 00:57:48.344346 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:48.344228    4284 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-f66180-84487bc787-gjmlg" podStartSLOduration=1.344198567 podStartE2EDuration="1.344198567s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 00:57:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:57:48.340720772 +0000 UTC m=+1782.485727685" watchObservedRunningTime="2026-10-31 00:57:48.344198567 +0000 UTC m=+1782.489205504"
Oct 31 00:57:48.700207 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:48.700160    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:57:50.244302 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:50.244264    4284 generic.go:356] "Generic (PLEG): container finished" podID="d999d6fd-4e88-49eb-b8ac-f9c1e03baa4c" containerID="37925e0229115d194189405ddd0fef06de3719e8b611f08a2d0a091046aaa467" exitCode=0
Oct 31 00:57:50.244745 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:50.244342    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-cb2b8a-openshift-default-75dd976c57-4jgr7" event={"ID":"d999d6fd-4e88-49eb-b8ac-f9c1e03baa4c","Type":"ContainerDied","Data":"37925e0229115d194189405ddd0fef06de3719e8b611f08a2d0a091046aaa467"}
Oct 31 00:57:50.244745 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:50.244740    4284 scope.go:122] "RemoveContainer" containerID="37925e0229115d194189405ddd0fef06de3719e8b611f08a2d0a091046aaa467"
Oct 31 00:57:50.246195 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:50.246171    4284 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 00:57:50.246274 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:50.246263    4284 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 00:57:50.246314 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:50.246297    4284 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 00:57:51.253239 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:51.253173    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-cb2b8a-openshift-default-75dd976c57-4jgr7" event={"ID":"d999d6fd-4e88-49eb-b8ac-f9c1e03baa4c","Type":"ContainerStarted","Data":"7285956f5dd8d2eb76fe948a272aea4828a3b3ba3268cdeaba9e4d3c418eee90"}
Oct 31 00:57:52.018297 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:52.018260    4284 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-cb2b8a-openshift-default-75dd976c57-4jgr7"
Oct 31 00:57:52.019377 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:52.019355    4284 patch_prober.go:29] interesting pod/e2e-ipp-b-cb2b8a-openshift-default-75dd976c57-4jgr7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.103:15021/healthz/ready\": dial tcp 10.134.0.103:15021: connect: connection refused" start-of-body=
Oct 31 00:57:52.019450 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:52.019413    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-cb2b8a-openshift-default-75dd976c57-4jgr7" podUID="d999d6fd-4e88-49eb-b8ac-f9c1e03baa4c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.103:15021/healthz/ready\": dial tcp 10.134.0.103:15021: connect: connection refused"
Oct 31 00:57:53.018273 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:53.018239    4284 patch_prober.go:29] interesting pod/e2e-ipp-b-cb2b8a-openshift-default-75dd976c57-4jgr7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.103:15021/healthz/ready\": dial tcp 10.134.0.103:15021: connect: connection refused" start-of-body=
Oct 31 00:57:53.018770 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:53.018332    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-cb2b8a-openshift-default-75dd976c57-4jgr7" podUID="d999d6fd-4e88-49eb-b8ac-f9c1e03baa4c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.103:15021/healthz/ready\": dial tcp 10.134.0.103:15021: connect: connection refused"
Oct 31 00:57:53.542423 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:53.542388    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:57:53.750897 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:53.750864    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:57:54.018176 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:54.018142    4284 patch_prober.go:29] interesting pod/e2e-ipp-b-cb2b8a-openshift-default-75dd976c57-4jgr7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.103:15021/healthz/ready\": dial tcp 10.134.0.103:15021: connect: connection refused" start-of-body=
Oct 31 00:57:54.018414 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:54.018193    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-cb2b8a-openshift-default-75dd976c57-4jgr7" podUID="d999d6fd-4e88-49eb-b8ac-f9c1e03baa4c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.103:15021/healthz/ready\": dial tcp 10.134.0.103:15021: connect: connection refused"
Oct 31 00:57:55.018914 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:55.018876    4284 patch_prober.go:29] interesting pod/e2e-ipp-b-cb2b8a-openshift-default-75dd976c57-4jgr7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.103:15021/healthz/ready\": dial tcp 10.134.0.103:15021: connect: connection refused" start-of-body=
Oct 31 00:57:55.019413 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:55.018929    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-cb2b8a-openshift-default-75dd976c57-4jgr7" podUID="d999d6fd-4e88-49eb-b8ac-f9c1e03baa4c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.103:15021/healthz/ready\": dial tcp 10.134.0.103:15021: connect: connection refused"
Oct 31 00:57:56.018339 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:56.018295    4284 patch_prober.go:29] interesting pod/e2e-ipp-b-cb2b8a-openshift-default-75dd976c57-4jgr7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.103:15021/healthz/ready\": dial tcp 10.134.0.103:15021: connect: connection refused" start-of-body=
Oct 31 00:57:56.018541 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:56.018367    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-cb2b8a-openshift-default-75dd976c57-4jgr7" podUID="d999d6fd-4e88-49eb-b8ac-f9c1e03baa4c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.103:15021/healthz/ready\": dial tcp 10.134.0.103:15021: connect: connection refused"
Oct 31 00:57:56.604710 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:56.604674    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-cb2b8a-openshift-default-75dd976c57-4jgr7"
Oct 31 00:57:57.024456 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:57.024428    4284 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-cb2b8a-openshift-default-75dd976c57-4jgr7"
Oct 31 00:57:57.343674 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:57.343585    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-cb2b8a-openshift-default-75dd976c57-4jgr7"
Oct 31 00:57:58.261243 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:58.261192    4284 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-b647c3-6bc8996947-bd4gb"]
Oct 31 00:57:58.265462 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:58.265435    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-b647c3-6bc8996947-bd4gb"
Oct 31 00:57:58.362825 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:58.362785    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/990f55f0-10cc-4930-8f0a-62989ef4aded-maas-api-tls\") pod \"maas-api-e2e-ipp-a-b647c3-6bc8996947-bd4gb\" (UID: \"990f55f0-10cc-4930-8f0a-62989ef4aded\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-b647c3-6bc8996947-bd4gb"
Oct 31 00:57:58.363306 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:58.363282    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kkblb\" (UniqueName: \"kubernetes.io/projected/990f55f0-10cc-4930-8f0a-62989ef4aded-kube-api-access-kkblb\") pod \"maas-api-e2e-ipp-a-b647c3-6bc8996947-bd4gb\" (UID: \"990f55f0-10cc-4930-8f0a-62989ef4aded\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-b647c3-6bc8996947-bd4gb"
Oct 31 00:57:58.363394 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:58.363350    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/990f55f0-10cc-4930-8f0a-62989ef4aded-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-b647c3-6bc8996947-bd4gb\" (UID: \"990f55f0-10cc-4930-8f0a-62989ef4aded\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-b647c3-6bc8996947-bd4gb"
Oct 31 00:57:58.363542 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:58.363518    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/990f55f0-10cc-4930-8f0a-62989ef4aded-metrics-certs\") pod \"maas-api-e2e-ipp-a-b647c3-6bc8996947-bd4gb\" (UID: \"990f55f0-10cc-4930-8f0a-62989ef4aded\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-b647c3-6bc8996947-bd4gb"
Oct 31 00:57:58.365299 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:58.365237    4284 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-a-b647c3-serving-cert\""
Oct 31 00:57:58.369511 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:58.369486    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-b647c3-6bc8996947-bd4gb"]
Oct 31 00:57:58.461744 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:58.461705    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:57:58.465099 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:58.465069    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kkblb\" (UniqueName: \"kubernetes.io/projected/990f55f0-10cc-4930-8f0a-62989ef4aded-kube-api-access-kkblb\") pod \"maas-api-e2e-ipp-a-b647c3-6bc8996947-bd4gb\" (UID: \"990f55f0-10cc-4930-8f0a-62989ef4aded\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-b647c3-6bc8996947-bd4gb"
Oct 31 00:57:58.465290 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:58.465266    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/990f55f0-10cc-4930-8f0a-62989ef4aded-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-b647c3-6bc8996947-bd4gb\" (UID: \"990f55f0-10cc-4930-8f0a-62989ef4aded\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-b647c3-6bc8996947-bd4gb"
Oct 31 00:57:58.465472 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:58.465445    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/990f55f0-10cc-4930-8f0a-62989ef4aded-metrics-certs\") pod \"maas-api-e2e-ipp-a-b647c3-6bc8996947-bd4gb\" (UID: \"990f55f0-10cc-4930-8f0a-62989ef4aded\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-b647c3-6bc8996947-bd4gb"
Oct 31 00:57:58.465558 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:58.465542    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/990f55f0-10cc-4930-8f0a-62989ef4aded-maas-api-tls\") pod \"maas-api-e2e-ipp-a-b647c3-6bc8996947-bd4gb\" (UID: \"990f55f0-10cc-4930-8f0a-62989ef4aded\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-b647c3-6bc8996947-bd4gb"
Oct 31 00:57:58.465693 ip-10-0-132-173 kubenswrapper[4284]: E1031 00:57:58.465675    4284 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-a-b647c3-serving-cert: secret "maas-api-e2e-ipp-a-b647c3-serving-cert" not found
Oct 31 00:57:58.465765 ip-10-0-132-173 kubenswrapper[4284]: E1031 00:57:58.465751    4284 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/990f55f0-10cc-4930-8f0a-62989ef4aded-maas-api-tls podName:990f55f0-10cc-4930-8f0a-62989ef4aded nodeName:}" failed. No retries permitted until 2026-10-31 00:57:58.965730387 +0000 UTC m=+1793.110737290 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/990f55f0-10cc-4930-8f0a-62989ef4aded-maas-api-tls") pod "maas-api-e2e-ipp-a-b647c3-6bc8996947-bd4gb" (UID: "990f55f0-10cc-4930-8f0a-62989ef4aded") : secret "maas-api-e2e-ipp-a-b647c3-serving-cert" not found
Oct 31 00:57:58.465952 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:58.465929    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/990f55f0-10cc-4930-8f0a-62989ef4aded-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-b647c3-6bc8996947-bd4gb\" (UID: \"990f55f0-10cc-4930-8f0a-62989ef4aded\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-b647c3-6bc8996947-bd4gb"
Oct 31 00:57:58.467987 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:58.467967    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/990f55f0-10cc-4930-8f0a-62989ef4aded-metrics-certs\") pod \"maas-api-e2e-ipp-a-b647c3-6bc8996947-bd4gb\" (UID: \"990f55f0-10cc-4930-8f0a-62989ef4aded\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-b647c3-6bc8996947-bd4gb"
Oct 31 00:57:58.541864 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:58.533934    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kkblb\" (UniqueName: \"kubernetes.io/projected/990f55f0-10cc-4930-8f0a-62989ef4aded-kube-api-access-kkblb\") pod \"maas-api-e2e-ipp-a-b647c3-6bc8996947-bd4gb\" (UID: \"990f55f0-10cc-4930-8f0a-62989ef4aded\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-b647c3-6bc8996947-bd4gb"
Oct 31 00:57:58.628490 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:58.628447    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:57:58.775556 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:58.775524    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:57:58.929641 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:58.929599    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:57:58.976143 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:58.976102    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/990f55f0-10cc-4930-8f0a-62989ef4aded-maas-api-tls\") pod \"maas-api-e2e-ipp-a-b647c3-6bc8996947-bd4gb\" (UID: \"990f55f0-10cc-4930-8f0a-62989ef4aded\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-b647c3-6bc8996947-bd4gb"
Oct 31 00:57:58.978807 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:58.978777    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/990f55f0-10cc-4930-8f0a-62989ef4aded-maas-api-tls\") pod \"maas-api-e2e-ipp-a-b647c3-6bc8996947-bd4gb\" (UID: \"990f55f0-10cc-4930-8f0a-62989ef4aded\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-b647c3-6bc8996947-bd4gb"
Oct 31 00:57:59.178731 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:59.178694    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-b647c3-6bc8996947-bd4gb"
Oct 31 00:57:59.239451 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:59.239427    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-f66180-84487bc787-gjmlg"
Oct 31 00:57:59.334124 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:59.334089    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:57:59.493137 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:59.493054    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:57:59.596749 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:57:59.596720    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-b647c3-6bc8996947-bd4gb"]
Oct 31 00:57:59.597863 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:57:59.597835    4284 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod990f55f0_10cc_4930_8f0a_62989ef4aded.slice/crio-07815d151c6405e6dc4960fdc727673549698ab8a966ca66ee7492379a4ef103 WatchSource:0}: Error finding container 07815d151c6405e6dc4960fdc727673549698ab8a966ca66ee7492379a4ef103: Status 404 returned error can't find the container with id 07815d151c6405e6dc4960fdc727673549698ab8a966ca66ee7492379a4ef103
Oct 31 00:58:00.370680 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:00.370648    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-b647c3-6bc8996947-bd4gb" event={"ID":"990f55f0-10cc-4930-8f0a-62989ef4aded","Type":"ContainerStarted","Data":"07815d151c6405e6dc4960fdc727673549698ab8a966ca66ee7492379a4ef103"}
Oct 31 00:58:01.398526 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:01.398430    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-b647c3-6bc8996947-bd4gb" event={"ID":"990f55f0-10cc-4930-8f0a-62989ef4aded","Type":"ContainerStarted","Data":"b61f38ac5d15cee202f2a732243fc0393386888c1ace9a6d1f392650dab66fbb"}
Oct 31 00:58:01.399023 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:01.398645    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-b647c3-6bc8996947-bd4gb"
Oct 31 00:58:01.515905 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:01.515850    4284 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-b647c3-6bc8996947-bd4gb" podStartSLOduration=2.635488145 podStartE2EDuration="3.51583617s" totalImagesPullingTime="880.348025ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 00:57:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:58:01.511759046 +0000 UTC m=+1795.656765963" watchObservedRunningTime="2026-10-31 00:58:01.51583617 +0000 UTC m=+1795.660843127"
Oct 31 00:58:03.501494 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:03.501453    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:58:04.717933 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:04.717899    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:58:06.026725 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:06.026690    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:58:07.409673 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:07.409643    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-b647c3-6bc8996947-bd4gb"
Oct 31 00:58:08.487458 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:08.487427    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:58:08.700639 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:08.700596    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:58:11.883534 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:11.883498    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:58:12.074948 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:12.074911    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:58:12.237418 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:12.237386    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-7e6fe88b-openshift-default-5b64b75c89-qzgdm"]
Oct 31 00:58:12.237938 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:12.237904    4284 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-7e6fe88b-openshift-default-5b64b75c89-qzgdm" podUID="b58c53d0-4acf-4de9-b322-8ead4dc9b013" containerName="istio-proxy" containerID="cri-o://051b5ac4888bf6adde575026483832446a241a62588127de7ac51a390e228ee2" gracePeriod=30
Oct 31 00:58:12.243867 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:12.243830    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:58:12.257662 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:12.257639    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-7e6fe88b-openshift-default-5b64b75c89-vtxl5"]
Oct 31 00:58:12.257832 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:12.257814    4284 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-7e6fe88b-openshift-default-5b64b75c89-vtxl5" podUID="c381f046-a134-41bd-8e86-ce8a9d580cb9" containerName="istio-proxy" containerID="cri-o://62331e7d0778d8c7d56790518744d30374d6bb135b70a871775015fb520a54aa" gracePeriod=30
Oct 31 00:58:12.266586 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:12.266562    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-7e6fe88b-openshift-default-5b64b75c89-l8cw5"]
Oct 31 00:58:12.266784 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:12.266763    4284 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-7e6fe88b-openshift-default-5b64b75c89-l8cw5" podUID="965d1fab-96f5-400b-bcf8-1be86ab61076" containerName="istio-proxy" containerID="cri-o://77b99edf1a3273033b494ff3534649c142c507990d3758bb7b6ababbef95d2e8" gracePeriod=30
Oct 31 00:58:12.281648 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:12.281627    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-7e6fe88b-openshift-default-5b64b75c89-8l2zb"]
Oct 31 00:58:12.281815 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:12.281786    4284 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-7e6fe88b-openshift-default-5b64b75c89-8l2zb" podUID="95491ac4-0f03-4ee0-a8cd-687621cbfa85" containerName="istio-proxy" containerID="cri-o://3b1157049d2cded92bfb5e01cbb09e205cba94ac6aa06477ca1c8a3ecc67f0b2" gracePeriod=30
Oct 31 00:58:13.071088 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:13.071055    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:58:13.573183 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:13.573146    4284 patch_prober.go:29] interesting pod/e2e-mt-7e6fe88b-openshift-default-5b64b75c89-8l2zb container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 31 00:58:13.573401 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:13.573232    4284 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-7e6fe88b-openshift-default-5b64b75c89-8l2zb" podUID="95491ac4-0f03-4ee0-a8cd-687621cbfa85" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 31 00:58:13.610593 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:13.610558    4284 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-eee7f1-openshift-default-78d7b57c9-cn4g6"]
Oct 31 00:58:13.616298 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:13.616274    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-eee7f1-openshift-default-78d7b57c9-cn4g6"
Oct 31 00:58:13.616637 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:13.616608    4284 patch_prober.go:29] interesting pod/e2e-mt-7e6fe88b-openshift-default-5b64b75c89-vtxl5 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 31 00:58:13.616779 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:13.616673    4284 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-7e6fe88b-openshift-default-5b64b75c89-vtxl5" podUID="c381f046-a134-41bd-8e86-ce8a9d580cb9" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 31 00:58:13.622709 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:13.622682    4284 patch_prober.go:29] interesting pod/e2e-mt-7e6fe88b-openshift-default-5b64b75c89-l8cw5 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 31 00:58:13.622916 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:13.622887    4284 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-7e6fe88b-openshift-default-5b64b75c89-l8cw5" podUID="965d1fab-96f5-400b-bcf8-1be86ab61076" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 31 00:58:13.669236 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:13.668540    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4fe7b9bb-dcce-4bb5-b37a-82fe1639b77e-istio-token\") pod \"e2e-api-a-eee7f1-openshift-default-78d7b57c9-cn4g6\" (UID: \"4fe7b9bb-dcce-4bb5-b37a-82fe1639b77e\") " pod="openshift-ingress/e2e-api-a-eee7f1-openshift-default-78d7b57c9-cn4g6"
Oct 31 00:58:13.669236 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:13.668803    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4fe7b9bb-dcce-4bb5-b37a-82fe1639b77e-istio-envoy\") pod \"e2e-api-a-eee7f1-openshift-default-78d7b57c9-cn4g6\" (UID: \"4fe7b9bb-dcce-4bb5-b37a-82fe1639b77e\") " pod="openshift-ingress/e2e-api-a-eee7f1-openshift-default-78d7b57c9-cn4g6"
Oct 31 00:58:13.669236 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:13.668864    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4fe7b9bb-dcce-4bb5-b37a-82fe1639b77e-istio-podinfo\") pod \"e2e-api-a-eee7f1-openshift-default-78d7b57c9-cn4g6\" (UID: \"4fe7b9bb-dcce-4bb5-b37a-82fe1639b77e\") " pod="openshift-ingress/e2e-api-a-eee7f1-openshift-default-78d7b57c9-cn4g6"
Oct 31 00:58:13.669236 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:13.668901    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8scvs\" (UniqueName: \"kubernetes.io/projected/4fe7b9bb-dcce-4bb5-b37a-82fe1639b77e-kube-api-access-8scvs\") pod \"e2e-api-a-eee7f1-openshift-default-78d7b57c9-cn4g6\" (UID: \"4fe7b9bb-dcce-4bb5-b37a-82fe1639b77e\") " pod="openshift-ingress/e2e-api-a-eee7f1-openshift-default-78d7b57c9-cn4g6"
Oct 31 00:58:13.669236 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:13.668972    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4fe7b9bb-dcce-4bb5-b37a-82fe1639b77e-workload-socket\") pod \"e2e-api-a-eee7f1-openshift-default-78d7b57c9-cn4g6\" (UID: \"4fe7b9bb-dcce-4bb5-b37a-82fe1639b77e\") " pod="openshift-ingress/e2e-api-a-eee7f1-openshift-default-78d7b57c9-cn4g6"
Oct 31 00:58:13.669236 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:13.669015    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4fe7b9bb-dcce-4bb5-b37a-82fe1639b77e-credential-socket\") pod \"e2e-api-a-eee7f1-openshift-default-78d7b57c9-cn4g6\" (UID: \"4fe7b9bb-dcce-4bb5-b37a-82fe1639b77e\") " pod="openshift-ingress/e2e-api-a-eee7f1-openshift-default-78d7b57c9-cn4g6"
Oct 31 00:58:13.669236 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:13.669149    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4fe7b9bb-dcce-4bb5-b37a-82fe1639b77e-istiod-ca-cert\") pod \"e2e-api-a-eee7f1-openshift-default-78d7b57c9-cn4g6\" (UID: \"4fe7b9bb-dcce-4bb5-b37a-82fe1639b77e\") " pod="openshift-ingress/e2e-api-a-eee7f1-openshift-default-78d7b57c9-cn4g6"
Oct 31 00:58:13.669236 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:13.669249    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4fe7b9bb-dcce-4bb5-b37a-82fe1639b77e-istio-data\") pod \"e2e-api-a-eee7f1-openshift-default-78d7b57c9-cn4g6\" (UID: \"4fe7b9bb-dcce-4bb5-b37a-82fe1639b77e\") " pod="openshift-ingress/e2e-api-a-eee7f1-openshift-default-78d7b57c9-cn4g6"
Oct 31 00:58:13.669830 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:13.669280    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4fe7b9bb-dcce-4bb5-b37a-82fe1639b77e-workload-certs\") pod \"e2e-api-a-eee7f1-openshift-default-78d7b57c9-cn4g6\" (UID: \"4fe7b9bb-dcce-4bb5-b37a-82fe1639b77e\") " pod="openshift-ingress/e2e-api-a-eee7f1-openshift-default-78d7b57c9-cn4g6"
Oct 31 00:58:13.733239 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:13.725249    4284 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-eee7f1-openshift-default-dockercfg-pd9n5\""
Oct 31 00:58:13.733239 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:13.728054    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-eee7f1-openshift-default-78d7b57c9-cn4g6"]
Oct 31 00:58:13.877748 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:13.867081    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4fe7b9bb-dcce-4bb5-b37a-82fe1639b77e-istio-data\") pod \"e2e-api-a-eee7f1-openshift-default-78d7b57c9-cn4g6\" (UID: \"4fe7b9bb-dcce-4bb5-b37a-82fe1639b77e\") " pod="openshift-ingress/e2e-api-a-eee7f1-openshift-default-78d7b57c9-cn4g6"
Oct 31 00:58:13.877748 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:13.867139    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4fe7b9bb-dcce-4bb5-b37a-82fe1639b77e-workload-certs\") pod \"e2e-api-a-eee7f1-openshift-default-78d7b57c9-cn4g6\" (UID: \"4fe7b9bb-dcce-4bb5-b37a-82fe1639b77e\") " pod="openshift-ingress/e2e-api-a-eee7f1-openshift-default-78d7b57c9-cn4g6"
Oct 31 00:58:13.877748 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:13.867241    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4fe7b9bb-dcce-4bb5-b37a-82fe1639b77e-istio-token\") pod \"e2e-api-a-eee7f1-openshift-default-78d7b57c9-cn4g6\" (UID: \"4fe7b9bb-dcce-4bb5-b37a-82fe1639b77e\") " pod="openshift-ingress/e2e-api-a-eee7f1-openshift-default-78d7b57c9-cn4g6"
Oct 31 00:58:13.877748 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:13.876847    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:58:13.877748 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:13.877252    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4fe7b9bb-dcce-4bb5-b37a-82fe1639b77e-istio-envoy\") pod \"e2e-api-a-eee7f1-openshift-default-78d7b57c9-cn4g6\" (UID: \"4fe7b9bb-dcce-4bb5-b37a-82fe1639b77e\") " pod="openshift-ingress/e2e-api-a-eee7f1-openshift-default-78d7b57c9-cn4g6"
Oct 31 00:58:13.877748 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:13.877285    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4fe7b9bb-dcce-4bb5-b37a-82fe1639b77e-istio-data\") pod \"e2e-api-a-eee7f1-openshift-default-78d7b57c9-cn4g6\" (UID: \"4fe7b9bb-dcce-4bb5-b37a-82fe1639b77e\") " pod="openshift-ingress/e2e-api-a-eee7f1-openshift-default-78d7b57c9-cn4g6"
Oct 31 00:58:13.877748 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:13.877355    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4fe7b9bb-dcce-4bb5-b37a-82fe1639b77e-istio-podinfo\") pod \"e2e-api-a-eee7f1-openshift-default-78d7b57c9-cn4g6\" (UID: \"4fe7b9bb-dcce-4bb5-b37a-82fe1639b77e\") " pod="openshift-ingress/e2e-api-a-eee7f1-openshift-default-78d7b57c9-cn4g6"
Oct 31 00:58:13.877748 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:13.877393    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8scvs\" (UniqueName: \"kubernetes.io/projected/4fe7b9bb-dcce-4bb5-b37a-82fe1639b77e-kube-api-access-8scvs\") pod \"e2e-api-a-eee7f1-openshift-default-78d7b57c9-cn4g6\" (UID: \"4fe7b9bb-dcce-4bb5-b37a-82fe1639b77e\") " pod="openshift-ingress/e2e-api-a-eee7f1-openshift-default-78d7b57c9-cn4g6"
Oct 31 00:58:13.877748 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:13.877431    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4fe7b9bb-dcce-4bb5-b37a-82fe1639b77e-workload-socket\") pod \"e2e-api-a-eee7f1-openshift-default-78d7b57c9-cn4g6\" (UID: \"4fe7b9bb-dcce-4bb5-b37a-82fe1639b77e\") " pod="openshift-ingress/e2e-api-a-eee7f1-openshift-default-78d7b57c9-cn4g6"
Oct 31 00:58:13.877748 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:13.877481    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4fe7b9bb-dcce-4bb5-b37a-82fe1639b77e-credential-socket\") pod \"e2e-api-a-eee7f1-openshift-default-78d7b57c9-cn4g6\" (UID: \"4fe7b9bb-dcce-4bb5-b37a-82fe1639b77e\") " pod="openshift-ingress/e2e-api-a-eee7f1-openshift-default-78d7b57c9-cn4g6"
Oct 31 00:58:13.877748 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:13.877658    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4fe7b9bb-dcce-4bb5-b37a-82fe1639b77e-istiod-ca-cert\") pod \"e2e-api-a-eee7f1-openshift-default-78d7b57c9-cn4g6\" (UID: \"4fe7b9bb-dcce-4bb5-b37a-82fe1639b77e\") " pod="openshift-ingress/e2e-api-a-eee7f1-openshift-default-78d7b57c9-cn4g6"
Oct 31 00:58:13.888913 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:13.878483    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4fe7b9bb-dcce-4bb5-b37a-82fe1639b77e-istiod-ca-cert\") pod \"e2e-api-a-eee7f1-openshift-default-78d7b57c9-cn4g6\" (UID: \"4fe7b9bb-dcce-4bb5-b37a-82fe1639b77e\") " pod="openshift-ingress/e2e-api-a-eee7f1-openshift-default-78d7b57c9-cn4g6"
Oct 31 00:58:13.888913 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:13.882446    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4fe7b9bb-dcce-4bb5-b37a-82fe1639b77e-istio-podinfo\") pod \"e2e-api-a-eee7f1-openshift-default-78d7b57c9-cn4g6\" (UID: \"4fe7b9bb-dcce-4bb5-b37a-82fe1639b77e\") " pod="openshift-ingress/e2e-api-a-eee7f1-openshift-default-78d7b57c9-cn4g6"
Oct 31 00:58:13.888913 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:13.883076    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4fe7b9bb-dcce-4bb5-b37a-82fe1639b77e-workload-socket\") pod \"e2e-api-a-eee7f1-openshift-default-78d7b57c9-cn4g6\" (UID: \"4fe7b9bb-dcce-4bb5-b37a-82fe1639b77e\") " pod="openshift-ingress/e2e-api-a-eee7f1-openshift-default-78d7b57c9-cn4g6"
Oct 31 00:58:13.888913 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:13.883349    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4fe7b9bb-dcce-4bb5-b37a-82fe1639b77e-credential-socket\") pod \"e2e-api-a-eee7f1-openshift-default-78d7b57c9-cn4g6\" (UID: \"4fe7b9bb-dcce-4bb5-b37a-82fe1639b77e\") " pod="openshift-ingress/e2e-api-a-eee7f1-openshift-default-78d7b57c9-cn4g6"
Oct 31 00:58:13.888913 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:13.883602    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4fe7b9bb-dcce-4bb5-b37a-82fe1639b77e-workload-certs\") pod \"e2e-api-a-eee7f1-openshift-default-78d7b57c9-cn4g6\" (UID: \"4fe7b9bb-dcce-4bb5-b37a-82fe1639b77e\") " pod="openshift-ingress/e2e-api-a-eee7f1-openshift-default-78d7b57c9-cn4g6"
Oct 31 00:58:13.983726 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:13.983575    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4fe7b9bb-dcce-4bb5-b37a-82fe1639b77e-istio-envoy\") pod \"e2e-api-a-eee7f1-openshift-default-78d7b57c9-cn4g6\" (UID: \"4fe7b9bb-dcce-4bb5-b37a-82fe1639b77e\") " pod="openshift-ingress/e2e-api-a-eee7f1-openshift-default-78d7b57c9-cn4g6"
Oct 31 00:58:14.028031 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:14.028000    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4fe7b9bb-dcce-4bb5-b37a-82fe1639b77e-istio-token\") pod \"e2e-api-a-eee7f1-openshift-default-78d7b57c9-cn4g6\" (UID: \"4fe7b9bb-dcce-4bb5-b37a-82fe1639b77e\") " pod="openshift-ingress/e2e-api-a-eee7f1-openshift-default-78d7b57c9-cn4g6"
Oct 31 00:58:14.028321 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:14.028037    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8scvs\" (UniqueName: \"kubernetes.io/projected/4fe7b9bb-dcce-4bb5-b37a-82fe1639b77e-kube-api-access-8scvs\") pod \"e2e-api-a-eee7f1-openshift-default-78d7b57c9-cn4g6\" (UID: \"4fe7b9bb-dcce-4bb5-b37a-82fe1639b77e\") " pod="openshift-ingress/e2e-api-a-eee7f1-openshift-default-78d7b57c9-cn4g6"
Oct 31 00:58:14.038591 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:14.038558    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:58:14.246788 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:14.246753    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-eee7f1-openshift-default-78d7b57c9-cn4g6"
Oct 31 00:58:14.724579 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:14.723870    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-eee7f1-openshift-default-78d7b57c9-cn4g6"]
Oct 31 00:58:14.730762 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:14.730729    4284 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 00:58:14.730860 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:14.730796    4284 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 00:58:14.730860 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:14.730822    4284 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 00:58:15.367000 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:15.366957    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:58:15.536957 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:15.536921    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-eee7f1-openshift-default-78d7b57c9-cn4g6" event={"ID":"4fe7b9bb-dcce-4bb5-b37a-82fe1639b77e","Type":"ContainerStarted","Data":"dbae75592cdb87487f1720b192cbb998410a7c09334712b8ca46c5e262be5ebe"}
Oct 31 00:58:15.537162 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:15.536971    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-eee7f1-openshift-default-78d7b57c9-cn4g6" event={"ID":"4fe7b9bb-dcce-4bb5-b37a-82fe1639b77e","Type":"ContainerStarted","Data":"f25a3cada30c511faae376abea6bc2307f0d6fba63f615e9e53e544f7c736bff"}
Oct 31 00:58:15.675829 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:15.675781    4284 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-eee7f1-openshift-default-78d7b57c9-cn4g6" podStartSLOduration=2.675767218 podStartE2EDuration="2.675767218s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 00:58:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:58:15.673938207 +0000 UTC m=+1809.818945122" watchObservedRunningTime="2026-10-31 00:58:15.675767218 +0000 UTC m=+1809.820774134"
Oct 31 00:58:15.929561 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:15.929478    4284 patch_prober.go:29] interesting pod/e2e-mt-7e6fe88b-openshift-default-5b64b75c89-qzgdm container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 31 00:58:15.929561 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:15.929526    4284 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-7e6fe88b-openshift-default-5b64b75c89-qzgdm" podUID="b58c53d0-4acf-4de9-b322-8ead4dc9b013" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 31 00:58:16.247785 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:16.247701    4284 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-eee7f1-openshift-default-78d7b57c9-cn4g6"
Oct 31 00:58:16.253275 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:16.253241    4284 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-eee7f1-openshift-default-78d7b57c9-cn4g6"
Oct 31 00:58:16.544059 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:16.543938    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-eee7f1-openshift-default-78d7b57c9-cn4g6"
Oct 31 00:58:16.544931 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:16.544911    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-eee7f1-openshift-default-78d7b57c9-cn4g6"
Oct 31 00:58:17.018416 ip-10-0-132-173 kubenswrapper[4284]: E1031 00:58:17.018380    4284 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fd8sp" containerName="extractor"
Oct 31 00:58:17.558192 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:17.558140    4284 generic.go:356] "Generic (PLEG): container finished" podID="965d1fab-96f5-400b-bcf8-1be86ab61076" containerID="77b99edf1a3273033b494ff3534649c142c507990d3758bb7b6ababbef95d2e8" exitCode=0
Oct 31 00:58:17.558637 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:17.558346    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-7e6fe88b-openshift-default-5b64b75c89-l8cw5" event={"ID":"965d1fab-96f5-400b-bcf8-1be86ab61076","Type":"ContainerDied","Data":"77b99edf1a3273033b494ff3534649c142c507990d3758bb7b6ababbef95d2e8"}
Oct 31 00:58:17.561195 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:17.561169    4284 generic.go:356] "Generic (PLEG): container finished" podID="b58c53d0-4acf-4de9-b322-8ead4dc9b013" containerID="051b5ac4888bf6adde575026483832446a241a62588127de7ac51a390e228ee2" exitCode=0
Oct 31 00:58:17.561442 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:17.561418    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-7e6fe88b-openshift-default-5b64b75c89-qzgdm" event={"ID":"b58c53d0-4acf-4de9-b322-8ead4dc9b013","Type":"ContainerDied","Data":"051b5ac4888bf6adde575026483832446a241a62588127de7ac51a390e228ee2"}
Oct 31 00:58:17.561513 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:17.561482    4284 scope.go:122] "RemoveContainer" containerID="c7da3b2ece1e204bf15a08c3bb655c87bf30646ca899dde9f0835ee35199ee7a"
Oct 31 00:58:17.564081 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:17.564058    4284 generic.go:356] "Generic (PLEG): container finished" podID="c381f046-a134-41bd-8e86-ce8a9d580cb9" containerID="62331e7d0778d8c7d56790518744d30374d6bb135b70a871775015fb520a54aa" exitCode=0
Oct 31 00:58:17.564195 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:17.564145    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-7e6fe88b-openshift-default-5b64b75c89-vtxl5" event={"ID":"c381f046-a134-41bd-8e86-ce8a9d580cb9","Type":"ContainerDied","Data":"62331e7d0778d8c7d56790518744d30374d6bb135b70a871775015fb520a54aa"}
Oct 31 00:58:17.565893 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:17.565876    4284 generic.go:356] "Generic (PLEG): container finished" podID="95491ac4-0f03-4ee0-a8cd-687621cbfa85" containerID="3b1157049d2cded92bfb5e01cbb09e205cba94ac6aa06477ca1c8a3ecc67f0b2" exitCode=0
Oct 31 00:58:17.566004 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:17.565954    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-7e6fe88b-openshift-default-5b64b75c89-8l2zb" event={"ID":"95491ac4-0f03-4ee0-a8cd-687621cbfa85","Type":"ContainerDied","Data":"3b1157049d2cded92bfb5e01cbb09e205cba94ac6aa06477ca1c8a3ecc67f0b2"}
Oct 31 00:58:17.574491 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:17.574463    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-7e6fe88b-openshift-default-5b64b75c89-l8cw5"
Oct 31 00:58:17.693874 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:17.693847    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/965d1fab-96f5-400b-bcf8-1be86ab61076-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/965d1fab-96f5-400b-bcf8-1be86ab61076-istiod-ca-cert\") pod \"965d1fab-96f5-400b-bcf8-1be86ab61076\" (UID: \"965d1fab-96f5-400b-bcf8-1be86ab61076\") "
Oct 31 00:58:17.694033 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:17.693941    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/965d1fab-96f5-400b-bcf8-1be86ab61076-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/965d1fab-96f5-400b-bcf8-1be86ab61076-istio-envoy\") pod \"965d1fab-96f5-400b-bcf8-1be86ab61076\" (UID: \"965d1fab-96f5-400b-bcf8-1be86ab61076\") "
Oct 31 00:58:17.694033 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:17.693993    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/965d1fab-96f5-400b-bcf8-1be86ab61076-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/965d1fab-96f5-400b-bcf8-1be86ab61076-credential-socket\") pod \"965d1fab-96f5-400b-bcf8-1be86ab61076\" (UID: \"965d1fab-96f5-400b-bcf8-1be86ab61076\") "
Oct 31 00:58:17.694166 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:17.694044    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/965d1fab-96f5-400b-bcf8-1be86ab61076-kube-api-access-vx88b\" (UniqueName: \"kubernetes.io/projected/965d1fab-96f5-400b-bcf8-1be86ab61076-kube-api-access-vx88b\") pod \"965d1fab-96f5-400b-bcf8-1be86ab61076\" (UID: \"965d1fab-96f5-400b-bcf8-1be86ab61076\") "
Oct 31 00:58:17.694166 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:17.694139    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/965d1fab-96f5-400b-bcf8-1be86ab61076-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/965d1fab-96f5-400b-bcf8-1be86ab61076-istio-data\") pod \"965d1fab-96f5-400b-bcf8-1be86ab61076\" (UID: \"965d1fab-96f5-400b-bcf8-1be86ab61076\") "
Oct 31 00:58:17.694417 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:17.694191    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/965d1fab-96f5-400b-bcf8-1be86ab61076-istio-token\" (UniqueName: \"kubernetes.io/projected/965d1fab-96f5-400b-bcf8-1be86ab61076-istio-token\") pod \"965d1fab-96f5-400b-bcf8-1be86ab61076\" (UID: \"965d1fab-96f5-400b-bcf8-1be86ab61076\") "
Oct 31 00:58:17.694417 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:17.694298    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/965d1fab-96f5-400b-bcf8-1be86ab61076-istiod-ca-cert" pod "965d1fab-96f5-400b-bcf8-1be86ab61076" (UID: "965d1fab-96f5-400b-bcf8-1be86ab61076"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 00:58:17.694417 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:17.694309    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/965d1fab-96f5-400b-bcf8-1be86ab61076-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/965d1fab-96f5-400b-bcf8-1be86ab61076-workload-socket\") pod \"965d1fab-96f5-400b-bcf8-1be86ab61076\" (UID: \"965d1fab-96f5-400b-bcf8-1be86ab61076\") "
Oct 31 00:58:17.694590 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:17.694501    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/965d1fab-96f5-400b-bcf8-1be86ab61076-workload-socket" pod "965d1fab-96f5-400b-bcf8-1be86ab61076" (UID: "965d1fab-96f5-400b-bcf8-1be86ab61076"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:58:17.694652 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:17.694610    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/965d1fab-96f5-400b-bcf8-1be86ab61076-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/965d1fab-96f5-400b-bcf8-1be86ab61076-istio-podinfo\") pod \"965d1fab-96f5-400b-bcf8-1be86ab61076\" (UID: \"965d1fab-96f5-400b-bcf8-1be86ab61076\") "
Oct 31 00:58:17.694708 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:17.694670    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/965d1fab-96f5-400b-bcf8-1be86ab61076-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/965d1fab-96f5-400b-bcf8-1be86ab61076-workload-certs\") pod \"965d1fab-96f5-400b-bcf8-1be86ab61076\" (UID: \"965d1fab-96f5-400b-bcf8-1be86ab61076\") "
Oct 31 00:58:17.694761 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:17.694742    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/965d1fab-96f5-400b-bcf8-1be86ab61076-istio-data" pod "965d1fab-96f5-400b-bcf8-1be86ab61076" (UID: "965d1fab-96f5-400b-bcf8-1be86ab61076"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:58:17.694914 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:17.694897    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/965d1fab-96f5-400b-bcf8-1be86ab61076-workload-certs" pod "965d1fab-96f5-400b-bcf8-1be86ab61076" (UID: "965d1fab-96f5-400b-bcf8-1be86ab61076"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:58:17.695117 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:17.695090    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/965d1fab-96f5-400b-bcf8-1be86ab61076-credential-socket" pod "965d1fab-96f5-400b-bcf8-1be86ab61076" (UID: "965d1fab-96f5-400b-bcf8-1be86ab61076"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:58:17.695819 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:17.695684    4284 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/965d1fab-96f5-400b-bcf8-1be86ab61076-workload-socket\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:58:17.695819 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:17.695709    4284 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/965d1fab-96f5-400b-bcf8-1be86ab61076-workload-certs\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:58:17.695819 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:17.695724    4284 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/965d1fab-96f5-400b-bcf8-1be86ab61076-istiod-ca-cert\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:58:17.695819 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:17.695773    4284 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/965d1fab-96f5-400b-bcf8-1be86ab61076-credential-socket\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:58:17.695819 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:17.695792    4284 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/965d1fab-96f5-400b-bcf8-1be86ab61076-istio-data\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:58:17.697714 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:17.697687    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/965d1fab-96f5-400b-bcf8-1be86ab61076-istio-envoy" pod "965d1fab-96f5-400b-bcf8-1be86ab61076" (UID: "965d1fab-96f5-400b-bcf8-1be86ab61076"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:58:17.697791 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:17.697734    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/965d1fab-96f5-400b-bcf8-1be86ab61076-istio-token" pod "965d1fab-96f5-400b-bcf8-1be86ab61076" (UID: "965d1fab-96f5-400b-bcf8-1be86ab61076"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 00:58:17.697847 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:17.697813    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/965d1fab-96f5-400b-bcf8-1be86ab61076-istio-podinfo" pod "965d1fab-96f5-400b-bcf8-1be86ab61076" (UID: "965d1fab-96f5-400b-bcf8-1be86ab61076"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 00:58:17.698343 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:17.698322    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/965d1fab-96f5-400b-bcf8-1be86ab61076-kube-api-access-vx88b" pod "965d1fab-96f5-400b-bcf8-1be86ab61076" (UID: "965d1fab-96f5-400b-bcf8-1be86ab61076"). InnerVolumeSpecName "kube-api-access-vx88b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 00:58:17.708022 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:17.708002    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-7e6fe88b-openshift-default-5b64b75c89-qzgdm"
Oct 31 00:58:17.710241 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:17.710202    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-7e6fe88b-openshift-default-5b64b75c89-vtxl5"
Oct 31 00:58:17.713822 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:17.713802    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-7e6fe88b-openshift-default-5b64b75c89-8l2zb"
Oct 31 00:58:17.796440 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:17.796407    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/95491ac4-0f03-4ee0-a8cd-687621cbfa85-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/95491ac4-0f03-4ee0-a8cd-687621cbfa85-workload-certs\") pod \"95491ac4-0f03-4ee0-a8cd-687621cbfa85\" (UID: \"95491ac4-0f03-4ee0-a8cd-687621cbfa85\") "
Oct 31 00:58:17.796440 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:17.796445    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b58c53d0-4acf-4de9-b322-8ead4dc9b013-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b58c53d0-4acf-4de9-b322-8ead4dc9b013-istio-data\") pod \"b58c53d0-4acf-4de9-b322-8ead4dc9b013\" (UID: \"b58c53d0-4acf-4de9-b322-8ead4dc9b013\") "
Oct 31 00:58:17.796694 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:17.796473    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/95491ac4-0f03-4ee0-a8cd-687621cbfa85-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/95491ac4-0f03-4ee0-a8cd-687621cbfa85-istio-envoy\") pod \"95491ac4-0f03-4ee0-a8cd-687621cbfa85\" (UID: \"95491ac4-0f03-4ee0-a8cd-687621cbfa85\") "
Oct 31 00:58:17.796694 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:17.796553    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c381f046-a134-41bd-8e86-ce8a9d580cb9-kube-api-access-zxk4p\" (UniqueName: \"kubernetes.io/projected/c381f046-a134-41bd-8e86-ce8a9d580cb9-kube-api-access-zxk4p\") pod \"c381f046-a134-41bd-8e86-ce8a9d580cb9\" (UID: \"c381f046-a134-41bd-8e86-ce8a9d580cb9\") "
Oct 31 00:58:17.796694 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:17.796587    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/b58c53d0-4acf-4de9-b322-8ead4dc9b013-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b58c53d0-4acf-4de9-b322-8ead4dc9b013-istio-podinfo\") pod \"b58c53d0-4acf-4de9-b322-8ead4dc9b013\" (UID: \"b58c53d0-4acf-4de9-b322-8ead4dc9b013\") "
Oct 31 00:58:17.796694 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:17.796628    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/95491ac4-0f03-4ee0-a8cd-687621cbfa85-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/95491ac4-0f03-4ee0-a8cd-687621cbfa85-credential-socket\") pod \"95491ac4-0f03-4ee0-a8cd-687621cbfa85\" (UID: \"95491ac4-0f03-4ee0-a8cd-687621cbfa85\") "
Oct 31 00:58:17.796694 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:17.796658    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b58c53d0-4acf-4de9-b322-8ead4dc9b013-kube-api-access-l89vc\" (UniqueName: \"kubernetes.io/projected/b58c53d0-4acf-4de9-b322-8ead4dc9b013-kube-api-access-l89vc\") pod \"b58c53d0-4acf-4de9-b322-8ead4dc9b013\" (UID: \"b58c53d0-4acf-4de9-b322-8ead4dc9b013\") "
Oct 31 00:58:17.796963 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:17.796730    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b58c53d0-4acf-4de9-b322-8ead4dc9b013-istio-data" pod "b58c53d0-4acf-4de9-b322-8ead4dc9b013" (UID: "b58c53d0-4acf-4de9-b322-8ead4dc9b013"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:58:17.796963 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:17.796741    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b58c53d0-4acf-4de9-b322-8ead4dc9b013-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b58c53d0-4acf-4de9-b322-8ead4dc9b013-credential-socket\") pod \"b58c53d0-4acf-4de9-b322-8ead4dc9b013\" (UID: \"b58c53d0-4acf-4de9-b322-8ead4dc9b013\") "
Oct 31 00:58:17.796963 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:17.796740    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/95491ac4-0f03-4ee0-a8cd-687621cbfa85-workload-certs" pod "95491ac4-0f03-4ee0-a8cd-687621cbfa85" (UID: "95491ac4-0f03-4ee0-a8cd-687621cbfa85"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:58:17.796963 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:17.796767    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c381f046-a134-41bd-8e86-ce8a9d580cb9-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c381f046-a134-41bd-8e86-ce8a9d580cb9-credential-socket\") pod \"c381f046-a134-41bd-8e86-ce8a9d580cb9\" (UID: \"c381f046-a134-41bd-8e86-ce8a9d580cb9\") "
Oct 31 00:58:17.796963 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:17.796798    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/95491ac4-0f03-4ee0-a8cd-687621cbfa85-istio-token\" (UniqueName: \"kubernetes.io/projected/95491ac4-0f03-4ee0-a8cd-687621cbfa85-istio-token\") pod \"95491ac4-0f03-4ee0-a8cd-687621cbfa85\" (UID: \"95491ac4-0f03-4ee0-a8cd-687621cbfa85\") "
Oct 31 00:58:17.796963 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:17.796833    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/95491ac4-0f03-4ee0-a8cd-687621cbfa85-kube-api-access-hbwft\" (UniqueName: \"kubernetes.io/projected/95491ac4-0f03-4ee0-a8cd-687621cbfa85-kube-api-access-hbwft\") pod \"95491ac4-0f03-4ee0-a8cd-687621cbfa85\" (UID: \"95491ac4-0f03-4ee0-a8cd-687621cbfa85\") "
Oct 31 00:58:17.796963 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:17.796863    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b58c53d0-4acf-4de9-b322-8ead4dc9b013-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b58c53d0-4acf-4de9-b322-8ead4dc9b013-workload-certs\") pod \"b58c53d0-4acf-4de9-b322-8ead4dc9b013\" (UID: \"b58c53d0-4acf-4de9-b322-8ead4dc9b013\") "
Oct 31 00:58:17.796963 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:17.796919    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b58c53d0-4acf-4de9-b322-8ead4dc9b013-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b58c53d0-4acf-4de9-b322-8ead4dc9b013-istio-envoy\") pod \"b58c53d0-4acf-4de9-b322-8ead4dc9b013\" (UID: \"b58c53d0-4acf-4de9-b322-8ead4dc9b013\") "
Oct 31 00:58:17.796963 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:17.796953    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/c381f046-a134-41bd-8e86-ce8a9d580cb9-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c381f046-a134-41bd-8e86-ce8a9d580cb9-istio-podinfo\") pod \"c381f046-a134-41bd-8e86-ce8a9d580cb9\" (UID: \"c381f046-a134-41bd-8e86-ce8a9d580cb9\") "
Oct 31 00:58:17.797562 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:17.796981    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/95491ac4-0f03-4ee0-a8cd-687621cbfa85-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/95491ac4-0f03-4ee0-a8cd-687621cbfa85-istiod-ca-cert\") pod \"95491ac4-0f03-4ee0-a8cd-687621cbfa85\" (UID: \"95491ac4-0f03-4ee0-a8cd-687621cbfa85\") "
Oct 31 00:58:17.797562 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:17.797041    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b58c53d0-4acf-4de9-b322-8ead4dc9b013-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b58c53d0-4acf-4de9-b322-8ead4dc9b013-workload-socket\") pod \"b58c53d0-4acf-4de9-b322-8ead4dc9b013\" (UID: \"b58c53d0-4acf-4de9-b322-8ead4dc9b013\") "
Oct 31 00:58:17.797562 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:17.797097    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/95491ac4-0f03-4ee0-a8cd-687621cbfa85-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/95491ac4-0f03-4ee0-a8cd-687621cbfa85-istio-data\") pod \"95491ac4-0f03-4ee0-a8cd-687621cbfa85\" (UID: \"95491ac4-0f03-4ee0-a8cd-687621cbfa85\") "
Oct 31 00:58:17.797562 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:17.797145    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b58c53d0-4acf-4de9-b322-8ead4dc9b013-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b58c53d0-4acf-4de9-b322-8ead4dc9b013-istiod-ca-cert\") pod \"b58c53d0-4acf-4de9-b322-8ead4dc9b013\" (UID: \"b58c53d0-4acf-4de9-b322-8ead4dc9b013\") "
Oct 31 00:58:17.797562 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:17.797190    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c381f046-a134-41bd-8e86-ce8a9d580cb9-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c381f046-a134-41bd-8e86-ce8a9d580cb9-istio-envoy\") pod \"c381f046-a134-41bd-8e86-ce8a9d580cb9\" (UID: \"c381f046-a134-41bd-8e86-ce8a9d580cb9\") "
Oct 31 00:58:17.797562 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:17.797250    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c381f046-a134-41bd-8e86-ce8a9d580cb9-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c381f046-a134-41bd-8e86-ce8a9d580cb9-workload-certs\") pod \"c381f046-a134-41bd-8e86-ce8a9d580cb9\" (UID: \"c381f046-a134-41bd-8e86-ce8a9d580cb9\") "
Oct 31 00:58:17.797562 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:17.797312    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c381f046-a134-41bd-8e86-ce8a9d580cb9-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c381f046-a134-41bd-8e86-ce8a9d580cb9-istiod-ca-cert\") pod \"c381f046-a134-41bd-8e86-ce8a9d580cb9\" (UID: \"c381f046-a134-41bd-8e86-ce8a9d580cb9\") "
Oct 31 00:58:17.797562 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:17.797355    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/95491ac4-0f03-4ee0-a8cd-687621cbfa85-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/95491ac4-0f03-4ee0-a8cd-687621cbfa85-istio-podinfo\") pod \"95491ac4-0f03-4ee0-a8cd-687621cbfa85\" (UID: \"95491ac4-0f03-4ee0-a8cd-687621cbfa85\") "
Oct 31 00:58:17.798053 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:17.797573    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c381f046-a134-41bd-8e86-ce8a9d580cb9-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c381f046-a134-41bd-8e86-ce8a9d580cb9-istio-data\") pod \"c381f046-a134-41bd-8e86-ce8a9d580cb9\" (UID: \"c381f046-a134-41bd-8e86-ce8a9d580cb9\") "
Oct 31 00:58:17.798053 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:17.797600    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b58c53d0-4acf-4de9-b322-8ead4dc9b013-istio-token\" (UniqueName: \"kubernetes.io/projected/b58c53d0-4acf-4de9-b322-8ead4dc9b013-istio-token\") pod \"b58c53d0-4acf-4de9-b322-8ead4dc9b013\" (UID: \"b58c53d0-4acf-4de9-b322-8ead4dc9b013\") "
Oct 31 00:58:17.798053 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:17.797639    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c381f046-a134-41bd-8e86-ce8a9d580cb9-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c381f046-a134-41bd-8e86-ce8a9d580cb9-workload-socket\") pod \"c381f046-a134-41bd-8e86-ce8a9d580cb9\" (UID: \"c381f046-a134-41bd-8e86-ce8a9d580cb9\") "
Oct 31 00:58:17.798053 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:17.797978    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c381f046-a134-41bd-8e86-ce8a9d580cb9-credential-socket" pod "c381f046-a134-41bd-8e86-ce8a9d580cb9" (UID: "c381f046-a134-41bd-8e86-ce8a9d580cb9"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:58:17.798285 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:17.798264    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/95491ac4-0f03-4ee0-a8cd-687621cbfa85-credential-socket" pod "95491ac4-0f03-4ee0-a8cd-687621cbfa85" (UID: "95491ac4-0f03-4ee0-a8cd-687621cbfa85"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:58:17.798409 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:17.798358    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b58c53d0-4acf-4de9-b322-8ead4dc9b013-istiod-ca-cert" pod "b58c53d0-4acf-4de9-b322-8ead4dc9b013" (UID: "b58c53d0-4acf-4de9-b322-8ead4dc9b013"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 00:58:17.799286 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:17.798573    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b58c53d0-4acf-4de9-b322-8ead4dc9b013-credential-socket" pod "b58c53d0-4acf-4de9-b322-8ead4dc9b013" (UID: "b58c53d0-4acf-4de9-b322-8ead4dc9b013"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:58:17.799286 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:17.798790    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/95491ac4-0f03-4ee0-a8cd-687621cbfa85-istiod-ca-cert" pod "95491ac4-0f03-4ee0-a8cd-687621cbfa85" (UID: "95491ac4-0f03-4ee0-a8cd-687621cbfa85"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 00:58:17.799286 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:17.798904    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b58c53d0-4acf-4de9-b322-8ead4dc9b013-workload-socket" pod "b58c53d0-4acf-4de9-b322-8ead4dc9b013" (UID: "b58c53d0-4acf-4de9-b322-8ead4dc9b013"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:58:17.799286 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:17.799020    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/95491ac4-0f03-4ee0-a8cd-687621cbfa85-istio-data" pod "95491ac4-0f03-4ee0-a8cd-687621cbfa85" (UID: "95491ac4-0f03-4ee0-a8cd-687621cbfa85"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:58:17.799286 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:17.799193    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/95491ac4-0f03-4ee0-a8cd-687621cbfa85-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/95491ac4-0f03-4ee0-a8cd-687621cbfa85-workload-socket\") pod \"95491ac4-0f03-4ee0-a8cd-687621cbfa85\" (UID: \"95491ac4-0f03-4ee0-a8cd-687621cbfa85\") "
Oct 31 00:58:17.799286 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:17.799263    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c381f046-a134-41bd-8e86-ce8a9d580cb9-istio-token\" (UniqueName: \"kubernetes.io/projected/c381f046-a134-41bd-8e86-ce8a9d580cb9-istio-token\") pod \"c381f046-a134-41bd-8e86-ce8a9d580cb9\" (UID: \"c381f046-a134-41bd-8e86-ce8a9d580cb9\") "
Oct 31 00:58:17.799679 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:17.799596    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b58c53d0-4acf-4de9-b322-8ead4dc9b013-istio-podinfo" pod "b58c53d0-4acf-4de9-b322-8ead4dc9b013" (UID: "b58c53d0-4acf-4de9-b322-8ead4dc9b013"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 00:58:17.799883 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:17.799859    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b58c53d0-4acf-4de9-b322-8ead4dc9b013-workload-certs" pod "b58c53d0-4acf-4de9-b322-8ead4dc9b013" (UID: "b58c53d0-4acf-4de9-b322-8ead4dc9b013"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:58:17.800070 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:17.800050    4284 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b58c53d0-4acf-4de9-b322-8ead4dc9b013-credential-socket\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:58:17.800148 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:17.800080    4284 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c381f046-a134-41bd-8e86-ce8a9d580cb9-credential-socket\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:58:17.800148 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:17.800103    4284 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b58c53d0-4acf-4de9-b322-8ead4dc9b013-workload-certs\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:58:17.800148 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:17.800119    4284 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/95491ac4-0f03-4ee0-a8cd-687621cbfa85-istiod-ca-cert\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:58:17.800148 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:17.800133    4284 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b58c53d0-4acf-4de9-b322-8ead4dc9b013-workload-socket\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:58:17.800148 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:17.800147    4284 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/965d1fab-96f5-400b-bcf8-1be86ab61076-istio-podinfo\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:58:17.800448 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:17.800160    4284 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/95491ac4-0f03-4ee0-a8cd-687621cbfa85-istio-data\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:58:17.800448 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:17.800173    4284 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b58c53d0-4acf-4de9-b322-8ead4dc9b013-istiod-ca-cert\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:58:17.800448 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:17.800187    4284 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/965d1fab-96f5-400b-bcf8-1be86ab61076-istio-envoy\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:58:17.800448 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:17.800201    4284 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vx88b\" (UniqueName: \"kubernetes.io/projected/965d1fab-96f5-400b-bcf8-1be86ab61076-kube-api-access-vx88b\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:58:17.800448 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:17.800234    4284 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/95491ac4-0f03-4ee0-a8cd-687621cbfa85-workload-certs\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:58:17.800448 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:17.800249    4284 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b58c53d0-4acf-4de9-b322-8ead4dc9b013-istio-data\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:58:17.800448 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:17.800264    4284 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/965d1fab-96f5-400b-bcf8-1be86ab61076-istio-token\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:58:17.800448 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:17.800277    4284 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b58c53d0-4acf-4de9-b322-8ead4dc9b013-istio-podinfo\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:58:17.800448 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:17.800294    4284 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/95491ac4-0f03-4ee0-a8cd-687621cbfa85-credential-socket\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:58:17.800916 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:17.800681    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c381f046-a134-41bd-8e86-ce8a9d580cb9-workload-certs" pod "c381f046-a134-41bd-8e86-ce8a9d580cb9" (UID: "c381f046-a134-41bd-8e86-ce8a9d580cb9"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:58:17.801457 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:17.801429    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b58c53d0-4acf-4de9-b322-8ead4dc9b013-kube-api-access-l89vc" pod "b58c53d0-4acf-4de9-b322-8ead4dc9b013" (UID: "b58c53d0-4acf-4de9-b322-8ead4dc9b013"). InnerVolumeSpecName "kube-api-access-l89vc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 00:58:17.801785 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:17.801751    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c381f046-a134-41bd-8e86-ce8a9d580cb9-istiod-ca-cert" pod "c381f046-a134-41bd-8e86-ce8a9d580cb9" (UID: "c381f046-a134-41bd-8e86-ce8a9d580cb9"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 00:58:17.802585 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:17.802557    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/95491ac4-0f03-4ee0-a8cd-687621cbfa85-kube-api-access-hbwft" pod "95491ac4-0f03-4ee0-a8cd-687621cbfa85" (UID: "95491ac4-0f03-4ee0-a8cd-687621cbfa85"). InnerVolumeSpecName "kube-api-access-hbwft". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 00:58:17.803165 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:17.803138    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c381f046-a134-41bd-8e86-ce8a9d580cb9-workload-socket" pod "c381f046-a134-41bd-8e86-ce8a9d580cb9" (UID: "c381f046-a134-41bd-8e86-ce8a9d580cb9"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:58:17.803385 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:17.803361    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/95491ac4-0f03-4ee0-a8cd-687621cbfa85-workload-socket" pod "95491ac4-0f03-4ee0-a8cd-687621cbfa85" (UID: "95491ac4-0f03-4ee0-a8cd-687621cbfa85"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:58:17.804435 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:17.804297    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b58c53d0-4acf-4de9-b322-8ead4dc9b013-istio-token" pod "b58c53d0-4acf-4de9-b322-8ead4dc9b013" (UID: "b58c53d0-4acf-4de9-b322-8ead4dc9b013"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 00:58:17.804961 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:17.804657    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c381f046-a134-41bd-8e86-ce8a9d580cb9-istio-data" pod "c381f046-a134-41bd-8e86-ce8a9d580cb9" (UID: "c381f046-a134-41bd-8e86-ce8a9d580cb9"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:58:17.804961 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:17.804802    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/95491ac4-0f03-4ee0-a8cd-687621cbfa85-istio-podinfo" pod "95491ac4-0f03-4ee0-a8cd-687621cbfa85" (UID: "95491ac4-0f03-4ee0-a8cd-687621cbfa85"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 00:58:17.804961 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:17.804888    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/95491ac4-0f03-4ee0-a8cd-687621cbfa85-istio-envoy" pod "95491ac4-0f03-4ee0-a8cd-687621cbfa85" (UID: "95491ac4-0f03-4ee0-a8cd-687621cbfa85"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:58:17.805380 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:17.805360    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c381f046-a134-41bd-8e86-ce8a9d580cb9-kube-api-access-zxk4p" pod "c381f046-a134-41bd-8e86-ce8a9d580cb9" (UID: "c381f046-a134-41bd-8e86-ce8a9d580cb9"). InnerVolumeSpecName "kube-api-access-zxk4p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 00:58:17.805964 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:17.805942    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/95491ac4-0f03-4ee0-a8cd-687621cbfa85-istio-token" pod "95491ac4-0f03-4ee0-a8cd-687621cbfa85" (UID: "95491ac4-0f03-4ee0-a8cd-687621cbfa85"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 00:58:17.806330 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:17.806307    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c381f046-a134-41bd-8e86-ce8a9d580cb9-istio-podinfo" pod "c381f046-a134-41bd-8e86-ce8a9d580cb9" (UID: "c381f046-a134-41bd-8e86-ce8a9d580cb9"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 00:58:17.806506 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:17.806490    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b58c53d0-4acf-4de9-b322-8ead4dc9b013-istio-envoy" pod "b58c53d0-4acf-4de9-b322-8ead4dc9b013" (UID: "b58c53d0-4acf-4de9-b322-8ead4dc9b013"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:58:17.806586 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:17.806566    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c381f046-a134-41bd-8e86-ce8a9d580cb9-istio-envoy" pod "c381f046-a134-41bd-8e86-ce8a9d580cb9" (UID: "c381f046-a134-41bd-8e86-ce8a9d580cb9"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:58:17.807176 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:17.807152    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c381f046-a134-41bd-8e86-ce8a9d580cb9-istio-token" pod "c381f046-a134-41bd-8e86-ce8a9d580cb9" (UID: "c381f046-a134-41bd-8e86-ce8a9d580cb9"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 00:58:17.901409 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:17.901371    4284 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c381f046-a134-41bd-8e86-ce8a9d580cb9-istio-envoy\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:58:17.901409 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:17.901411    4284 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c381f046-a134-41bd-8e86-ce8a9d580cb9-workload-certs\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:58:17.901671 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:17.901426    4284 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c381f046-a134-41bd-8e86-ce8a9d580cb9-istiod-ca-cert\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:58:17.901671 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:17.901444    4284 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/95491ac4-0f03-4ee0-a8cd-687621cbfa85-istio-podinfo\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:58:17.901671 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:17.901459    4284 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c381f046-a134-41bd-8e86-ce8a9d580cb9-istio-data\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:58:17.901671 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:17.901474    4284 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b58c53d0-4acf-4de9-b322-8ead4dc9b013-istio-token\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:58:17.901671 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:17.901491    4284 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c381f046-a134-41bd-8e86-ce8a9d580cb9-workload-socket\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:58:17.901671 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:17.901504    4284 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/95491ac4-0f03-4ee0-a8cd-687621cbfa85-workload-socket\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:58:17.901671 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:17.901518    4284 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c381f046-a134-41bd-8e86-ce8a9d580cb9-istio-token\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:58:17.901671 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:17.901531    4284 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/95491ac4-0f03-4ee0-a8cd-687621cbfa85-istio-envoy\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:58:17.901671 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:17.901545    4284 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zxk4p\" (UniqueName: \"kubernetes.io/projected/c381f046-a134-41bd-8e86-ce8a9d580cb9-kube-api-access-zxk4p\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:58:17.901671 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:17.901560    4284 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l89vc\" (UniqueName: \"kubernetes.io/projected/b58c53d0-4acf-4de9-b322-8ead4dc9b013-kube-api-access-l89vc\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:58:17.901671 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:17.901573    4284 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/95491ac4-0f03-4ee0-a8cd-687621cbfa85-istio-token\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:58:17.901671 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:17.901587    4284 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hbwft\" (UniqueName: \"kubernetes.io/projected/95491ac4-0f03-4ee0-a8cd-687621cbfa85-kube-api-access-hbwft\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:58:17.901671 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:17.901601    4284 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b58c53d0-4acf-4de9-b322-8ead4dc9b013-istio-envoy\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:58:17.901671 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:17.901613    4284 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c381f046-a134-41bd-8e86-ce8a9d580cb9-istio-podinfo\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:58:18.540394 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:18.540303    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:58:18.574225 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:18.574177    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-7e6fe88b-openshift-default-5b64b75c89-qzgdm"
Oct 31 00:58:18.574702 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:18.574184    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-7e6fe88b-openshift-default-5b64b75c89-qzgdm" event={"ID":"b58c53d0-4acf-4de9-b322-8ead4dc9b013","Type":"ContainerDied","Data":"b73ff651d7dfa72bca7bb44b1c0d6791795457c590fb2a61f744e456517b42a6"}
Oct 31 00:58:18.574702 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:18.574426    4284 scope.go:122] "RemoveContainer" containerID="051b5ac4888bf6adde575026483832446a241a62588127de7ac51a390e228ee2"
Oct 31 00:58:18.575913 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:18.575888    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-7e6fe88b-openshift-default-5b64b75c89-vtxl5" event={"ID":"c381f046-a134-41bd-8e86-ce8a9d580cb9","Type":"ContainerDied","Data":"283124e2462a8d0995cc86fa0d94feb636df52249e95a606f458bb7fc9e35e66"}
Oct 31 00:58:18.576036 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:18.575921    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-7e6fe88b-openshift-default-5b64b75c89-vtxl5"
Oct 31 00:58:18.579368 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:18.579345    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-7e6fe88b-openshift-default-5b64b75c89-8l2zb"
Oct 31 00:58:18.579890 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:18.579871    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-7e6fe88b-openshift-default-5b64b75c89-8l2zb" event={"ID":"95491ac4-0f03-4ee0-a8cd-687621cbfa85","Type":"ContainerDied","Data":"54d29ff0addadd8cc978e650c3161752c46c63bd2f556a8cdf850def4095d306"}
Oct 31 00:58:18.581914 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:18.581886    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-7e6fe88b-openshift-default-5b64b75c89-l8cw5" event={"ID":"965d1fab-96f5-400b-bcf8-1be86ab61076","Type":"ContainerDied","Data":"ddc619ca6301459166b9d81e23e03dc2a534dc83334662e44d1528a81931c6ee"}
Oct 31 00:58:18.582022 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:18.582001    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-7e6fe88b-openshift-default-5b64b75c89-l8cw5"
Oct 31 00:58:18.588030 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:18.588014    4284 scope.go:122] "RemoveContainer" containerID="62331e7d0778d8c7d56790518744d30374d6bb135b70a871775015fb520a54aa"
Oct 31 00:58:18.597896 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:18.597823    4284 scope.go:122] "RemoveContainer" containerID="3b1157049d2cded92bfb5e01cbb09e205cba94ac6aa06477ca1c8a3ecc67f0b2"
Oct 31 00:58:18.609398 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:18.609360    4284 scope.go:122] "RemoveContainer" containerID="77b99edf1a3273033b494ff3534649c142c507990d3758bb7b6ababbef95d2e8"
Oct 31 00:58:18.735893 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:18.735856    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-7e6fe88b-openshift-default-5b64b75c89-8l2zb"]
Oct 31 00:58:18.740951 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:18.740923    4284 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-7e6fe88b-openshift-default-5b64b75c89-8l2zb"]
Oct 31 00:58:18.749187 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:18.749159    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:58:18.850339 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:18.850255    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-7e6fe88b-openshift-default-5b64b75c89-vtxl5"]
Oct 31 00:58:18.853878 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:18.853850    4284 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-7e6fe88b-openshift-default-5b64b75c89-vtxl5"]
Oct 31 00:58:18.971841 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:18.971807    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-7e6fe88b-openshift-default-5b64b75c89-qzgdm"]
Oct 31 00:58:18.981326 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:18.981291    4284 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-7e6fe88b-openshift-default-5b64b75c89-qzgdm"]
Oct 31 00:58:19.087426 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:19.087389    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-7e6fe88b-openshift-default-5b64b75c89-l8cw5"]
Oct 31 00:58:19.095545 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:19.095509    4284 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-7e6fe88b-openshift-default-5b64b75c89-l8cw5"]
Oct 31 00:58:19.964307 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:19.964270    4284 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="95491ac4-0f03-4ee0-a8cd-687621cbfa85" path="/var/lib/kubelet/pods/95491ac4-0f03-4ee0-a8cd-687621cbfa85/volumes"
Oct 31 00:58:19.964851 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:19.964835    4284 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="965d1fab-96f5-400b-bcf8-1be86ab61076" path="/var/lib/kubelet/pods/965d1fab-96f5-400b-bcf8-1be86ab61076/volumes"
Oct 31 00:58:19.965338 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:19.965319    4284 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b58c53d0-4acf-4de9-b322-8ead4dc9b013" path="/var/lib/kubelet/pods/b58c53d0-4acf-4de9-b322-8ead4dc9b013/volumes"
Oct 31 00:58:19.966110 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:19.966092    4284 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c381f046-a134-41bd-8e86-ce8a9d580cb9" path="/var/lib/kubelet/pods/c381f046-a134-41bd-8e86-ce8a9d580cb9/volumes"
Oct 31 00:58:20.435526 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:20.435487    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:58:21.606484 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:21.606454    4284 generic.go:356] "Generic (PLEG): container finished" podID="c0f54831-726b-41bd-adfb-445f8b9a39f9" containerID="fbcb1a47b41849c5e466e4e7fbbe5368ac5730a6bad7c113a530a849b67c6762" exitCode=0
Oct 31 00:58:21.606985 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:21.606506    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-f6979f-openshift-default-569f6f8dff-4k4gk" event={"ID":"c0f54831-726b-41bd-adfb-445f8b9a39f9","Type":"ContainerDied","Data":"fbcb1a47b41849c5e466e4e7fbbe5368ac5730a6bad7c113a530a849b67c6762"}
Oct 31 00:58:21.606985 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:21.606890    4284 scope.go:122] "RemoveContainer" containerID="fbcb1a47b41849c5e466e4e7fbbe5368ac5730a6bad7c113a530a849b67c6762"
Oct 31 00:58:21.613456 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:21.609875    4284 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 00:58:21.613456 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:21.610096    4284 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 00:58:21.613456 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:21.610229    4284 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 00:58:21.763276 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:21.763241    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:58:21.958763 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:21.958730    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:58:22.330437 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:22.330355    4284 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-eee7f1-77669c5769-fr9zj"]
Oct 31 00:58:22.330949 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:22.330921    4284 cpu_manager.go:414] "containerMap: removing container" podUID="b58c53d0-4acf-4de9-b322-8ead4dc9b013" containerName="istio-proxy"
Oct 31 00:58:22.330949 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:22.330939    4284 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b58c53d0-4acf-4de9-b322-8ead4dc9b013" containerName="istio-proxy"
Oct 31 00:58:22.331153 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:22.330957    4284 cpu_manager.go:414] "containerMap: removing container" podUID="965d1fab-96f5-400b-bcf8-1be86ab61076" containerName="istio-proxy"
Oct 31 00:58:22.331153 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:22.330964    4284 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="965d1fab-96f5-400b-bcf8-1be86ab61076" containerName="istio-proxy"
Oct 31 00:58:22.331153 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:22.330990    4284 cpu_manager.go:414] "containerMap: removing container" podUID="c381f046-a134-41bd-8e86-ce8a9d580cb9" containerName="istio-proxy"
Oct 31 00:58:22.331153 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:22.330997    4284 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c381f046-a134-41bd-8e86-ce8a9d580cb9" containerName="istio-proxy"
Oct 31 00:58:22.331153 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:22.331020    4284 cpu_manager.go:414] "containerMap: removing container" podUID="95491ac4-0f03-4ee0-a8cd-687621cbfa85" containerName="istio-proxy"
Oct 31 00:58:22.331153 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:22.331026    4284 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="95491ac4-0f03-4ee0-a8cd-687621cbfa85" containerName="istio-proxy"
Oct 31 00:58:22.331487 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:22.331174    4284 memory_manager.go:365] "RemoveStaleState removing state" podUID="b58c53d0-4acf-4de9-b322-8ead4dc9b013" containerName="istio-proxy"
Oct 31 00:58:22.331487 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:22.331185    4284 memory_manager.go:365] "RemoveStaleState removing state" podUID="c381f046-a134-41bd-8e86-ce8a9d580cb9" containerName="istio-proxy"
Oct 31 00:58:22.331487 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:22.331200    4284 memory_manager.go:365] "RemoveStaleState removing state" podUID="95491ac4-0f03-4ee0-a8cd-687621cbfa85" containerName="istio-proxy"
Oct 31 00:58:22.331487 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:22.331230    4284 memory_manager.go:365] "RemoveStaleState removing state" podUID="965d1fab-96f5-400b-bcf8-1be86ab61076" containerName="istio-proxy"
Oct 31 00:58:22.334325 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:22.334306    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-eee7f1-77669c5769-fr9zj"
Oct 31 00:58:22.390235 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:22.388157    4284 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-a-eee7f1-dockercfg-7s9hk\""
Oct 31 00:58:22.409249 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:22.404304    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-eee7f1-77669c5769-fr9zj"]
Oct 31 00:58:22.409249 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:22.404474    4284 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-a-eee7f1\""
Oct 31 00:58:22.478256 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:22.473372    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pqmxs\" (UniqueName: \"kubernetes.io/projected/33b97842-74a9-4d0b-9887-d65273b2bbf8-kube-api-access-pqmxs\") pod \"payload-processing-e2e-api-a-eee7f1-77669c5769-fr9zj\" (UID: \"33b97842-74a9-4d0b-9887-d65273b2bbf8\") " pod="openshift-ingress/payload-processing-e2e-api-a-eee7f1-77669c5769-fr9zj"
Oct 31 00:58:22.478256 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:22.473469    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/33b97842-74a9-4d0b-9887-d65273b2bbf8-plugins-config-volume\") pod \"payload-processing-e2e-api-a-eee7f1-77669c5769-fr9zj\" (UID: \"33b97842-74a9-4d0b-9887-d65273b2bbf8\") " pod="openshift-ingress/payload-processing-e2e-api-a-eee7f1-77669c5769-fr9zj"
Oct 31 00:58:22.588452 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:22.578881    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pqmxs\" (UniqueName: \"kubernetes.io/projected/33b97842-74a9-4d0b-9887-d65273b2bbf8-kube-api-access-pqmxs\") pod \"payload-processing-e2e-api-a-eee7f1-77669c5769-fr9zj\" (UID: \"33b97842-74a9-4d0b-9887-d65273b2bbf8\") " pod="openshift-ingress/payload-processing-e2e-api-a-eee7f1-77669c5769-fr9zj"
Oct 31 00:58:22.588452 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:22.578974    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/33b97842-74a9-4d0b-9887-d65273b2bbf8-plugins-config-volume\") pod \"payload-processing-e2e-api-a-eee7f1-77669c5769-fr9zj\" (UID: \"33b97842-74a9-4d0b-9887-d65273b2bbf8\") " pod="openshift-ingress/payload-processing-e2e-api-a-eee7f1-77669c5769-fr9zj"
Oct 31 00:58:22.588452 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:22.580169    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/33b97842-74a9-4d0b-9887-d65273b2bbf8-plugins-config-volume\") pod \"payload-processing-e2e-api-a-eee7f1-77669c5769-fr9zj\" (UID: \"33b97842-74a9-4d0b-9887-d65273b2bbf8\") " pod="openshift-ingress/payload-processing-e2e-api-a-eee7f1-77669c5769-fr9zj"
Oct 31 00:58:22.638299 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:22.628360    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-f6979f-openshift-default-569f6f8dff-4k4gk" event={"ID":"c0f54831-726b-41bd-adfb-445f8b9a39f9","Type":"ContainerStarted","Data":"055dc4b196bef410ceec47f2f5edeeb694a88f89b4bc5c445db16c0dc805c6b3"}
Oct 31 00:58:22.648880 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:22.648832    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pqmxs\" (UniqueName: \"kubernetes.io/projected/33b97842-74a9-4d0b-9887-d65273b2bbf8-kube-api-access-pqmxs\") pod \"payload-processing-e2e-api-a-eee7f1-77669c5769-fr9zj\" (UID: \"33b97842-74a9-4d0b-9887-d65273b2bbf8\") " pod="openshift-ingress/payload-processing-e2e-api-a-eee7f1-77669c5769-fr9zj"
Oct 31 00:58:22.862648 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:22.862569    4284 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-f6979f-openshift-default-569f6f8dff-4k4gk"
Oct 31 00:58:22.862808 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:22.862734    4284 patch_prober.go:29] interesting pod/e2e-ipp-b-f6979f-openshift-default-569f6f8dff-4k4gk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.108:15021/healthz/ready\": dial tcp 10.134.0.108:15021: connect: connection refused" start-of-body=
Oct 31 00:58:22.862808 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:22.862778    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-f6979f-openshift-default-569f6f8dff-4k4gk" podUID="c0f54831-726b-41bd-adfb-445f8b9a39f9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.108:15021/healthz/ready\": dial tcp 10.134.0.108:15021: connect: connection refused"
Oct 31 00:58:22.947557 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:22.947516    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-eee7f1-77669c5769-fr9zj"
Oct 31 00:58:23.349019 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:23.348990    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-eee7f1-77669c5769-fr9zj"]
Oct 31 00:58:23.349444 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:58:23.349418    4284 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod33b97842_74a9_4d0b_9887_d65273b2bbf8.slice/crio-a9c75c143347edbfa7404596281ec4059baf781d68e3edd20cad7fb6a076dc83 WatchSource:0}: Error finding container a9c75c143347edbfa7404596281ec4059baf781d68e3edd20cad7fb6a076dc83: Status 404 returned error can't find the container with id a9c75c143347edbfa7404596281ec4059baf781d68e3edd20cad7fb6a076dc83
Oct 31 00:58:23.637107 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:23.637016    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-eee7f1-77669c5769-fr9zj" event={"ID":"33b97842-74a9-4d0b-9887-d65273b2bbf8","Type":"ContainerStarted","Data":"03cc268f1ad87ae4d82e4cd8f130cae4be152c9e7118c1cc91cd9f9b540b63c7"}
Oct 31 00:58:23.637107 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:23.637070    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-eee7f1-77669c5769-fr9zj" event={"ID":"33b97842-74a9-4d0b-9887-d65273b2bbf8","Type":"ContainerStarted","Data":"a9c75c143347edbfa7404596281ec4059baf781d68e3edd20cad7fb6a076dc83"}
Oct 31 00:58:23.637339 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:23.637221    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-a-eee7f1-77669c5769-fr9zj"
Oct 31 00:58:23.762143 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:23.762091    4284 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-a-eee7f1-77669c5769-fr9zj" podStartSLOduration=1.762078275 podStartE2EDuration="1.762078275s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 00:58:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:58:23.75956335 +0000 UTC m=+1817.904570263" watchObservedRunningTime="2026-10-31 00:58:23.762078275 +0000 UTC m=+1817.907085191"
Oct 31 00:58:23.862989 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:23.862956    4284 patch_prober.go:29] interesting pod/e2e-ipp-b-f6979f-openshift-default-569f6f8dff-4k4gk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.108:15021/healthz/ready\": dial tcp 10.134.0.108:15021: connect: connection refused" start-of-body=
Oct 31 00:58:23.863161 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:23.863006    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-f6979f-openshift-default-569f6f8dff-4k4gk" podUID="c0f54831-726b-41bd-adfb-445f8b9a39f9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.108:15021/healthz/ready\": dial tcp 10.134.0.108:15021: connect: connection refused"
Oct 31 00:58:24.862947 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:24.862917    4284 patch_prober.go:29] interesting pod/e2e-ipp-b-f6979f-openshift-default-569f6f8dff-4k4gk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.108:15021/healthz/ready\": dial tcp 10.134.0.108:15021: connect: connection refused" start-of-body=
Oct 31 00:58:24.863362 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:24.862975    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-f6979f-openshift-default-569f6f8dff-4k4gk" podUID="c0f54831-726b-41bd-adfb-445f8b9a39f9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.108:15021/healthz/ready\": dial tcp 10.134.0.108:15021: connect: connection refused"
Oct 31 00:58:25.862623 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:25.862583    4284 patch_prober.go:29] interesting pod/e2e-ipp-b-f6979f-openshift-default-569f6f8dff-4k4gk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.108:15021/healthz/ready\": dial tcp 10.134.0.108:15021: connect: connection refused" start-of-body=
Oct 31 00:58:25.862807 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:25.862653    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-f6979f-openshift-default-569f6f8dff-4k4gk" podUID="c0f54831-726b-41bd-adfb-445f8b9a39f9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.108:15021/healthz/ready\": dial tcp 10.134.0.108:15021: connect: connection refused"
Oct 31 00:58:25.926366 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:25.926328    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:58:26.710978 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:26.710937    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:58:26.862899 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:26.862867    4284 patch_prober.go:29] interesting pod/e2e-ipp-b-f6979f-openshift-default-569f6f8dff-4k4gk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.108:15021/healthz/ready\": dial tcp 10.134.0.108:15021: connect: connection refused" start-of-body=
Oct 31 00:58:26.863081 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:26.862919    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-f6979f-openshift-default-569f6f8dff-4k4gk" podUID="c0f54831-726b-41bd-adfb-445f8b9a39f9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.108:15021/healthz/ready\": dial tcp 10.134.0.108:15021: connect: connection refused"
Oct 31 00:58:27.951145 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:27.951114    4284 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-f6979f-openshift-default-569f6f8dff-4k4gk"
Oct 31 00:58:27.952360 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:27.952334    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-f6979f-openshift-default-569f6f8dff-4k4gk"
Oct 31 00:58:27.953118 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:27.953097    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-f6979f-openshift-default-569f6f8dff-4k4gk"
Oct 31 00:58:28.071315 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:28.071270    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-cb2b8a-7546789495-5wzmh"]
Oct 31 00:58:28.487718 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:28.487681    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:58:28.637566 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:28.637534    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:58:31.225369 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:31.225330    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:58:33.072592 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:33.072551    4284 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-b-cb2b8a-7546789495-5wzmh" podUID="a8b1b169-c9ee-4dea-91cc-ec434f44b05c" containerName="payload-processing" containerID="cri-o://2a1707f7388819a8fdab42f235ef72a25ac9beb07a2742fdc37309ea21a60f33" gracePeriod=25
Oct 31 00:58:33.350833 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:33.350806    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-cb2b8a-7546789495-5wzmh"
Oct 31 00:58:33.446289 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:33.446239    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a8b1b169-c9ee-4dea-91cc-ec434f44b05c-kube-api-access-68vx9\" (UniqueName: \"kubernetes.io/projected/a8b1b169-c9ee-4dea-91cc-ec434f44b05c-kube-api-access-68vx9\") pod \"a8b1b169-c9ee-4dea-91cc-ec434f44b05c\" (UID: \"a8b1b169-c9ee-4dea-91cc-ec434f44b05c\") "
Oct 31 00:58:33.446538 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:33.446520    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a8b1b169-c9ee-4dea-91cc-ec434f44b05c-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a8b1b169-c9ee-4dea-91cc-ec434f44b05c-plugins-config-volume\") pod \"a8b1b169-c9ee-4dea-91cc-ec434f44b05c\" (UID: \"a8b1b169-c9ee-4dea-91cc-ec434f44b05c\") "
Oct 31 00:58:33.446975 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:33.446940    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a8b1b169-c9ee-4dea-91cc-ec434f44b05c-plugins-config-volume" pod "a8b1b169-c9ee-4dea-91cc-ec434f44b05c" (UID: "a8b1b169-c9ee-4dea-91cc-ec434f44b05c"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 00:58:33.448795 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:33.448762    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a8b1b169-c9ee-4dea-91cc-ec434f44b05c-kube-api-access-68vx9" pod "a8b1b169-c9ee-4dea-91cc-ec434f44b05c" (UID: "a8b1b169-c9ee-4dea-91cc-ec434f44b05c"). InnerVolumeSpecName "kube-api-access-68vx9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 00:58:33.548703 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:33.548667    4284 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a8b1b169-c9ee-4dea-91cc-ec434f44b05c-plugins-config-volume\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:58:33.548703 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:33.548696    4284 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-68vx9\" (UniqueName: \"kubernetes.io/projected/a8b1b169-c9ee-4dea-91cc-ec434f44b05c-kube-api-access-68vx9\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:58:33.816518 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:33.816488    4284 generic.go:356] "Generic (PLEG): container finished" podID="a8b1b169-c9ee-4dea-91cc-ec434f44b05c" containerID="2a1707f7388819a8fdab42f235ef72a25ac9beb07a2742fdc37309ea21a60f33" exitCode=0
Oct 31 00:58:33.816690 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:33.816534    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-cb2b8a-7546789495-5wzmh"
Oct 31 00:58:33.816690 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:33.816566    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-cb2b8a-7546789495-5wzmh" event={"ID":"a8b1b169-c9ee-4dea-91cc-ec434f44b05c","Type":"ContainerDied","Data":"2a1707f7388819a8fdab42f235ef72a25ac9beb07a2742fdc37309ea21a60f33"}
Oct 31 00:58:33.816690 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:33.816616    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-cb2b8a-7546789495-5wzmh" event={"ID":"a8b1b169-c9ee-4dea-91cc-ec434f44b05c","Type":"ContainerDied","Data":"ea11ed7dd33844ad9119f5fb2096907e378311f3235f0b98c6b584bfde3e0358"}
Oct 31 00:58:33.816690 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:33.816638    4284 scope.go:122] "RemoveContainer" containerID="2a1707f7388819a8fdab42f235ef72a25ac9beb07a2742fdc37309ea21a60f33"
Oct 31 00:58:33.826670 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:33.826648    4284 scope.go:122] "RemoveContainer" containerID="2a1707f7388819a8fdab42f235ef72a25ac9beb07a2742fdc37309ea21a60f33"
Oct 31 00:58:33.826960 ip-10-0-132-173 kubenswrapper[4284]: E1031 00:58:33.826942    4284 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2a1707f7388819a8fdab42f235ef72a25ac9beb07a2742fdc37309ea21a60f33\": container with ID starting with 2a1707f7388819a8fdab42f235ef72a25ac9beb07a2742fdc37309ea21a60f33 not found: ID does not exist" containerID="2a1707f7388819a8fdab42f235ef72a25ac9beb07a2742fdc37309ea21a60f33"
Oct 31 00:58:33.827020 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:33.826968    4284 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2a1707f7388819a8fdab42f235ef72a25ac9beb07a2742fdc37309ea21a60f33"} err="failed to get container status \"2a1707f7388819a8fdab42f235ef72a25ac9beb07a2742fdc37309ea21a60f33\": rpc error: code = NotFound desc = could not find container \"2a1707f7388819a8fdab42f235ef72a25ac9beb07a2742fdc37309ea21a60f33\": container with ID starting with 2a1707f7388819a8fdab42f235ef72a25ac9beb07a2742fdc37309ea21a60f33 not found: ID does not exist"
Oct 31 00:58:33.974342 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:33.974310    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-cb2b8a-7546789495-5wzmh"]
Oct 31 00:58:34.022380 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:34.022346    4284 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-cb2b8a-7546789495-5wzmh"]
Oct 31 00:58:34.646232 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:34.646190    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-a-eee7f1-77669c5769-fr9zj"
Oct 31 00:58:35.964801 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:35.964767    4284 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a8b1b169-c9ee-4dea-91cc-ec434f44b05c" path="/var/lib/kubelet/pods/a8b1b169-c9ee-4dea-91cc-ec434f44b05c/volumes"
Oct 31 00:58:36.533313 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:36.533274    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:58:41.830384 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:41.830343    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:58:43.148429 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:43.148393    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:58:43.768928 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:43.768896    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-r7hr5"]
Oct 31 00:58:43.769173 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:43.769147    4284 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-r7hr5" podUID="6cce1cfc-7109-4827-a231-fe01dd9a4bf9" containerName="istio-proxy" containerID="cri-o://7f51bafb86f4a8d560e43f5c45adb5803d37b3d0626fc00957bedeb001510d17" gracePeriod=30
Oct 31 00:58:43.817845 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:43.817798    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-hvt6d"]
Oct 31 00:58:43.818067 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:43.818044    4284 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-hvt6d" podUID="907b625f-2ccc-4de2-8181-34ff51ab2620" containerName="istio-proxy" containerID="cri-o://1207c899f144a2a77001bd83adccc1f195da972fc68fa2cf9e2379f470b4102f" gracePeriod=30
Oct 31 00:58:43.820146 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:43.820112    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-d4s6f"]
Oct 31 00:58:43.820397 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:43.820368    4284 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-d4s6f" podUID="b398d9e9-9d78-4185-ac8c-4a9eecca9f48" containerName="istio-proxy" containerID="cri-o://532045cc890f889522b7e6812eb8a00980310d5d0f7b93cc2dab220d84eb52c8" gracePeriod=30
Oct 31 00:58:46.553472 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:46.553388    4284 patch_prober.go:29] interesting pod/e2e-ipp-a-b647c3-openshift-default-674fcc8445-tvzlw container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.109:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 31 00:58:46.553844 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:46.553479    4284 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-a-b647c3-openshift-default-674fcc8445-tvzlw" podUID="caf73a48-2172-46df-9cbc-fd85b49552c1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.109:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 31 00:58:46.901326 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:46.901293    4284 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6cc458d887-r7hr5 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 31 00:58:46.904037 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:46.901359    4284 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-r7hr5" podUID="6cce1cfc-7109-4827-a231-fe01dd9a4bf9" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 31 00:58:47.027586 ip-10-0-132-173 kubenswrapper[4284]: E1031 00:58:47.027549    4284 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fd8sp" containerName="extractor"
Oct 31 00:58:47.350244 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:47.350196    4284 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6cc458d887-d4s6f container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 31 00:58:47.350442 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:47.350294    4284 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-d4s6f" podUID="b398d9e9-9d78-4185-ac8c-4a9eecca9f48" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 31 00:58:47.418774 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:47.418738    4284 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6cc458d887-hvt6d container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 31 00:58:47.418953 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:47.418799    4284 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-hvt6d" podUID="907b625f-2ccc-4de2-8181-34ff51ab2620" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 31 00:58:47.985992 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:47.985958    4284 generic.go:356] "Generic (PLEG): container finished" podID="caf73a48-2172-46df-9cbc-fd85b49552c1" containerID="200c2f3410b7b909a860d413e1b5e3459658ebce88cb879986f4357bffa51f2a" exitCode=0
Oct 31 00:58:47.986419 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:47.986028    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-b647c3-openshift-default-674fcc8445-tvzlw" event={"ID":"caf73a48-2172-46df-9cbc-fd85b49552c1","Type":"ContainerDied","Data":"200c2f3410b7b909a860d413e1b5e3459658ebce88cb879986f4357bffa51f2a"}
Oct 31 00:58:47.986486 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:47.986471    4284 scope.go:122] "RemoveContainer" containerID="200c2f3410b7b909a860d413e1b5e3459658ebce88cb879986f4357bffa51f2a"
Oct 31 00:58:47.987947 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:47.987916    4284 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 00:58:47.988026 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:47.987988    4284 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 00:58:47.988026 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:47.988022    4284 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 00:58:48.163592 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:48.163550    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:58:48.999614 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:48.999585    4284 generic.go:356] "Generic (PLEG): container finished" podID="6cce1cfc-7109-4827-a231-fe01dd9a4bf9" containerID="7f51bafb86f4a8d560e43f5c45adb5803d37b3d0626fc00957bedeb001510d17" exitCode=0
Oct 31 00:58:49.000091 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:48.999651    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-r7hr5" event={"ID":"6cce1cfc-7109-4827-a231-fe01dd9a4bf9","Type":"ContainerDied","Data":"7f51bafb86f4a8d560e43f5c45adb5803d37b3d0626fc00957bedeb001510d17"}
Oct 31 00:58:49.018759 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:49.001688    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-b647c3-openshift-default-674fcc8445-tvzlw" event={"ID":"caf73a48-2172-46df-9cbc-fd85b49552c1","Type":"ContainerStarted","Data":"867fd738ff0fb9c788a891250912b0306b7069ff4157488172a234aa175e56eb"}
Oct 31 00:58:49.018759 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:49.005901    4284 generic.go:356] "Generic (PLEG): container finished" podID="b398d9e9-9d78-4185-ac8c-4a9eecca9f48" containerID="532045cc890f889522b7e6812eb8a00980310d5d0f7b93cc2dab220d84eb52c8" exitCode=0
Oct 31 00:58:49.018759 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:49.005980    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-d4s6f" event={"ID":"b398d9e9-9d78-4185-ac8c-4a9eecca9f48","Type":"ContainerDied","Data":"532045cc890f889522b7e6812eb8a00980310d5d0f7b93cc2dab220d84eb52c8"}
Oct 31 00:58:49.018759 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:49.008501    4284 generic.go:356] "Generic (PLEG): container finished" podID="907b625f-2ccc-4de2-8181-34ff51ab2620" containerID="1207c899f144a2a77001bd83adccc1f195da972fc68fa2cf9e2379f470b4102f" exitCode=0
Oct 31 00:58:49.018759 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:49.008571    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-hvt6d" event={"ID":"907b625f-2ccc-4de2-8181-34ff51ab2620","Type":"ContainerDied","Data":"1207c899f144a2a77001bd83adccc1f195da972fc68fa2cf9e2379f470b4102f"}
Oct 31 00:58:49.128136 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:49.127810    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-r7hr5"
Oct 31 00:58:49.145357 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:49.144928    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-hvt6d"
Oct 31 00:58:49.197873 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:49.197842    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/6cce1cfc-7109-4827-a231-fe01dd9a4bf9-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6cce1cfc-7109-4827-a231-fe01dd9a4bf9-istio-podinfo\") pod \"6cce1cfc-7109-4827-a231-fe01dd9a4bf9\" (UID: \"6cce1cfc-7109-4827-a231-fe01dd9a4bf9\") "
Oct 31 00:58:49.198026 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:49.197896    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/907b625f-2ccc-4de2-8181-34ff51ab2620-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/907b625f-2ccc-4de2-8181-34ff51ab2620-workload-socket\") pod \"907b625f-2ccc-4de2-8181-34ff51ab2620\" (UID: \"907b625f-2ccc-4de2-8181-34ff51ab2620\") "
Oct 31 00:58:49.198026 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:49.197922    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6cce1cfc-7109-4827-a231-fe01dd9a4bf9-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6cce1cfc-7109-4827-a231-fe01dd9a4bf9-istio-envoy\") pod \"6cce1cfc-7109-4827-a231-fe01dd9a4bf9\" (UID: \"6cce1cfc-7109-4827-a231-fe01dd9a4bf9\") "
Oct 31 00:58:49.198026 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:49.197939    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6cce1cfc-7109-4827-a231-fe01dd9a4bf9-istio-token\" (UniqueName: \"kubernetes.io/projected/6cce1cfc-7109-4827-a231-fe01dd9a4bf9-istio-token\") pod \"6cce1cfc-7109-4827-a231-fe01dd9a4bf9\" (UID: \"6cce1cfc-7109-4827-a231-fe01dd9a4bf9\") "
Oct 31 00:58:49.198026 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:49.197991    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/907b625f-2ccc-4de2-8181-34ff51ab2620-istio-token\" (UniqueName: \"kubernetes.io/projected/907b625f-2ccc-4de2-8181-34ff51ab2620-istio-token\") pod \"907b625f-2ccc-4de2-8181-34ff51ab2620\" (UID: \"907b625f-2ccc-4de2-8181-34ff51ab2620\") "
Oct 31 00:58:49.198350 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:49.198085    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6cce1cfc-7109-4827-a231-fe01dd9a4bf9-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6cce1cfc-7109-4827-a231-fe01dd9a4bf9-workload-certs\") pod \"6cce1cfc-7109-4827-a231-fe01dd9a4bf9\" (UID: \"6cce1cfc-7109-4827-a231-fe01dd9a4bf9\") "
Oct 31 00:58:49.198350 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:49.198124    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/907b625f-2ccc-4de2-8181-34ff51ab2620-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/907b625f-2ccc-4de2-8181-34ff51ab2620-workload-certs\") pod \"907b625f-2ccc-4de2-8181-34ff51ab2620\" (UID: \"907b625f-2ccc-4de2-8181-34ff51ab2620\") "
Oct 31 00:58:49.198350 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:49.198178    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6cce1cfc-7109-4827-a231-fe01dd9a4bf9-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6cce1cfc-7109-4827-a231-fe01dd9a4bf9-istio-data\") pod \"6cce1cfc-7109-4827-a231-fe01dd9a4bf9\" (UID: \"6cce1cfc-7109-4827-a231-fe01dd9a4bf9\") "
Oct 31 00:58:49.198350 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:49.198248    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/907b625f-2ccc-4de2-8181-34ff51ab2620-workload-socket" pod "907b625f-2ccc-4de2-8181-34ff51ab2620" (UID: "907b625f-2ccc-4de2-8181-34ff51ab2620"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:58:49.198350 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:49.198298    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6cce1cfc-7109-4827-a231-fe01dd9a4bf9-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6cce1cfc-7109-4827-a231-fe01dd9a4bf9-workload-socket\") pod \"6cce1cfc-7109-4827-a231-fe01dd9a4bf9\" (UID: \"6cce1cfc-7109-4827-a231-fe01dd9a4bf9\") "
Oct 31 00:58:49.198865 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:49.198361    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/907b625f-2ccc-4de2-8181-34ff51ab2620-kube-api-access-grl28\" (UniqueName: \"kubernetes.io/projected/907b625f-2ccc-4de2-8181-34ff51ab2620-kube-api-access-grl28\") pod \"907b625f-2ccc-4de2-8181-34ff51ab2620\" (UID: \"907b625f-2ccc-4de2-8181-34ff51ab2620\") "
Oct 31 00:58:49.198865 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:49.198396    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6cce1cfc-7109-4827-a231-fe01dd9a4bf9-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6cce1cfc-7109-4827-a231-fe01dd9a4bf9-credential-socket\") pod \"6cce1cfc-7109-4827-a231-fe01dd9a4bf9\" (UID: \"6cce1cfc-7109-4827-a231-fe01dd9a4bf9\") "
Oct 31 00:58:49.198865 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:49.198453    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6cce1cfc-7109-4827-a231-fe01dd9a4bf9-workload-certs" pod "6cce1cfc-7109-4827-a231-fe01dd9a4bf9" (UID: "6cce1cfc-7109-4827-a231-fe01dd9a4bf9"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:58:49.198865 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:49.198475    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/907b625f-2ccc-4de2-8181-34ff51ab2620-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/907b625f-2ccc-4de2-8181-34ff51ab2620-istio-envoy\") pod \"907b625f-2ccc-4de2-8181-34ff51ab2620\" (UID: \"907b625f-2ccc-4de2-8181-34ff51ab2620\") "
Oct 31 00:58:49.198865 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:49.198508    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6cce1cfc-7109-4827-a231-fe01dd9a4bf9-kube-api-access-6vpff\" (UniqueName: \"kubernetes.io/projected/6cce1cfc-7109-4827-a231-fe01dd9a4bf9-kube-api-access-6vpff\") pod \"6cce1cfc-7109-4827-a231-fe01dd9a4bf9\" (UID: \"6cce1cfc-7109-4827-a231-fe01dd9a4bf9\") "
Oct 31 00:58:49.198865 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:49.198554    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/907b625f-2ccc-4de2-8181-34ff51ab2620-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/907b625f-2ccc-4de2-8181-34ff51ab2620-istiod-ca-cert\") pod \"907b625f-2ccc-4de2-8181-34ff51ab2620\" (UID: \"907b625f-2ccc-4de2-8181-34ff51ab2620\") "
Oct 31 00:58:49.198865 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:49.198591    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/907b625f-2ccc-4de2-8181-34ff51ab2620-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/907b625f-2ccc-4de2-8181-34ff51ab2620-credential-socket\") pod \"907b625f-2ccc-4de2-8181-34ff51ab2620\" (UID: \"907b625f-2ccc-4de2-8181-34ff51ab2620\") "
Oct 31 00:58:49.198865 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:49.198620    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/907b625f-2ccc-4de2-8181-34ff51ab2620-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/907b625f-2ccc-4de2-8181-34ff51ab2620-istio-data\") pod \"907b625f-2ccc-4de2-8181-34ff51ab2620\" (UID: \"907b625f-2ccc-4de2-8181-34ff51ab2620\") "
Oct 31 00:58:49.198865 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:49.198640    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/907b625f-2ccc-4de2-8181-34ff51ab2620-workload-certs" pod "907b625f-2ccc-4de2-8181-34ff51ab2620" (UID: "907b625f-2ccc-4de2-8181-34ff51ab2620"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:58:49.198865 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:49.198647    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/907b625f-2ccc-4de2-8181-34ff51ab2620-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/907b625f-2ccc-4de2-8181-34ff51ab2620-istio-podinfo\") pod \"907b625f-2ccc-4de2-8181-34ff51ab2620\" (UID: \"907b625f-2ccc-4de2-8181-34ff51ab2620\") "
Oct 31 00:58:49.198865 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:49.198653    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6cce1cfc-7109-4827-a231-fe01dd9a4bf9-istio-data" pod "6cce1cfc-7109-4827-a231-fe01dd9a4bf9" (UID: "6cce1cfc-7109-4827-a231-fe01dd9a4bf9"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:58:49.198865 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:49.198718    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/6cce1cfc-7109-4827-a231-fe01dd9a4bf9-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6cce1cfc-7109-4827-a231-fe01dd9a4bf9-istiod-ca-cert\") pod \"6cce1cfc-7109-4827-a231-fe01dd9a4bf9\" (UID: \"6cce1cfc-7109-4827-a231-fe01dd9a4bf9\") "
Oct 31 00:58:49.198865 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:49.198828    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6cce1cfc-7109-4827-a231-fe01dd9a4bf9-credential-socket" pod "6cce1cfc-7109-4827-a231-fe01dd9a4bf9" (UID: "6cce1cfc-7109-4827-a231-fe01dd9a4bf9"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:58:49.199614 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:49.198872    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/907b625f-2ccc-4de2-8181-34ff51ab2620-credential-socket" pod "907b625f-2ccc-4de2-8181-34ff51ab2620" (UID: "907b625f-2ccc-4de2-8181-34ff51ab2620"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:58:49.199614 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:49.199187    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/907b625f-2ccc-4de2-8181-34ff51ab2620-istiod-ca-cert" pod "907b625f-2ccc-4de2-8181-34ff51ab2620" (UID: "907b625f-2ccc-4de2-8181-34ff51ab2620"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 00:58:49.199614 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:49.199400    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/907b625f-2ccc-4de2-8181-34ff51ab2620-istio-data" pod "907b625f-2ccc-4de2-8181-34ff51ab2620" (UID: "907b625f-2ccc-4de2-8181-34ff51ab2620"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:58:49.199786 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:49.199701    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6cce1cfc-7109-4827-a231-fe01dd9a4bf9-istiod-ca-cert" pod "6cce1cfc-7109-4827-a231-fe01dd9a4bf9" (UID: "6cce1cfc-7109-4827-a231-fe01dd9a4bf9"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 00:58:49.199844 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:49.199809    4284 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6cce1cfc-7109-4827-a231-fe01dd9a4bf9-workload-certs\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:58:49.199844 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:49.199829    4284 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/907b625f-2ccc-4de2-8181-34ff51ab2620-workload-certs\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:58:49.199947 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:49.199842    4284 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6cce1cfc-7109-4827-a231-fe01dd9a4bf9-istio-data\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:58:49.199947 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:49.199845    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-d4s6f"
Oct 31 00:58:49.199947 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:49.199856    4284 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6cce1cfc-7109-4827-a231-fe01dd9a4bf9-credential-socket\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:58:49.199947 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:49.199869    4284 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/907b625f-2ccc-4de2-8181-34ff51ab2620-istiod-ca-cert\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:58:49.199947 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:49.199882    4284 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/907b625f-2ccc-4de2-8181-34ff51ab2620-credential-socket\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:58:49.199947 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:49.199895    4284 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/907b625f-2ccc-4de2-8181-34ff51ab2620-istio-data\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:58:49.199947 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:49.199908    4284 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/907b625f-2ccc-4de2-8181-34ff51ab2620-workload-socket\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:58:49.200518 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:49.200494    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6cce1cfc-7109-4827-a231-fe01dd9a4bf9-workload-socket" pod "6cce1cfc-7109-4827-a231-fe01dd9a4bf9" (UID: "6cce1cfc-7109-4827-a231-fe01dd9a4bf9"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:58:49.201532 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:49.201507    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/907b625f-2ccc-4de2-8181-34ff51ab2620-kube-api-access-grl28" pod "907b625f-2ccc-4de2-8181-34ff51ab2620" (UID: "907b625f-2ccc-4de2-8181-34ff51ab2620"). InnerVolumeSpecName "kube-api-access-grl28". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 00:58:49.206633 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:49.206568    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/907b625f-2ccc-4de2-8181-34ff51ab2620-istio-podinfo" pod "907b625f-2ccc-4de2-8181-34ff51ab2620" (UID: "907b625f-2ccc-4de2-8181-34ff51ab2620"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 00:58:49.206633 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:49.206596    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/907b625f-2ccc-4de2-8181-34ff51ab2620-istio-envoy" pod "907b625f-2ccc-4de2-8181-34ff51ab2620" (UID: "907b625f-2ccc-4de2-8181-34ff51ab2620"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:58:49.206633 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:49.206627    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/907b625f-2ccc-4de2-8181-34ff51ab2620-istio-token" pod "907b625f-2ccc-4de2-8181-34ff51ab2620" (UID: "907b625f-2ccc-4de2-8181-34ff51ab2620"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 00:58:49.206923 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:49.206902    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6cce1cfc-7109-4827-a231-fe01dd9a4bf9-kube-api-access-6vpff" pod "6cce1cfc-7109-4827-a231-fe01dd9a4bf9" (UID: "6cce1cfc-7109-4827-a231-fe01dd9a4bf9"). InnerVolumeSpecName "kube-api-access-6vpff". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 00:58:49.206923 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:49.206902    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6cce1cfc-7109-4827-a231-fe01dd9a4bf9-istio-envoy" pod "6cce1cfc-7109-4827-a231-fe01dd9a4bf9" (UID: "6cce1cfc-7109-4827-a231-fe01dd9a4bf9"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:58:49.207103 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:49.206956    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6cce1cfc-7109-4827-a231-fe01dd9a4bf9-istio-token" pod "6cce1cfc-7109-4827-a231-fe01dd9a4bf9" (UID: "6cce1cfc-7109-4827-a231-fe01dd9a4bf9"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 00:58:49.207993 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:49.207975    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/6cce1cfc-7109-4827-a231-fe01dd9a4bf9-istio-podinfo" pod "6cce1cfc-7109-4827-a231-fe01dd9a4bf9" (UID: "6cce1cfc-7109-4827-a231-fe01dd9a4bf9"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 00:58:49.314269 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:49.312676    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b398d9e9-9d78-4185-ac8c-4a9eecca9f48-kube-api-access-xkwp6\" (UniqueName: \"kubernetes.io/projected/b398d9e9-9d78-4185-ac8c-4a9eecca9f48-kube-api-access-xkwp6\") pod \"b398d9e9-9d78-4185-ac8c-4a9eecca9f48\" (UID: \"b398d9e9-9d78-4185-ac8c-4a9eecca9f48\") "
Oct 31 00:58:49.314269 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:49.312768    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b398d9e9-9d78-4185-ac8c-4a9eecca9f48-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b398d9e9-9d78-4185-ac8c-4a9eecca9f48-istio-envoy\") pod \"b398d9e9-9d78-4185-ac8c-4a9eecca9f48\" (UID: \"b398d9e9-9d78-4185-ac8c-4a9eecca9f48\") "
Oct 31 00:58:49.314269 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:49.312834    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/b398d9e9-9d78-4185-ac8c-4a9eecca9f48-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b398d9e9-9d78-4185-ac8c-4a9eecca9f48-istio-podinfo\") pod \"b398d9e9-9d78-4185-ac8c-4a9eecca9f48\" (UID: \"b398d9e9-9d78-4185-ac8c-4a9eecca9f48\") "
Oct 31 00:58:49.314269 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:49.312895    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b398d9e9-9d78-4185-ac8c-4a9eecca9f48-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b398d9e9-9d78-4185-ac8c-4a9eecca9f48-istio-data\") pod \"b398d9e9-9d78-4185-ac8c-4a9eecca9f48\" (UID: \"b398d9e9-9d78-4185-ac8c-4a9eecca9f48\") "
Oct 31 00:58:49.314269 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:49.312929    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b398d9e9-9d78-4185-ac8c-4a9eecca9f48-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b398d9e9-9d78-4185-ac8c-4a9eecca9f48-workload-certs\") pod \"b398d9e9-9d78-4185-ac8c-4a9eecca9f48\" (UID: \"b398d9e9-9d78-4185-ac8c-4a9eecca9f48\") "
Oct 31 00:58:49.314269 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:49.312993    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b398d9e9-9d78-4185-ac8c-4a9eecca9f48-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b398d9e9-9d78-4185-ac8c-4a9eecca9f48-istiod-ca-cert\") pod \"b398d9e9-9d78-4185-ac8c-4a9eecca9f48\" (UID: \"b398d9e9-9d78-4185-ac8c-4a9eecca9f48\") "
Oct 31 00:58:49.314269 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:49.313118    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b398d9e9-9d78-4185-ac8c-4a9eecca9f48-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b398d9e9-9d78-4185-ac8c-4a9eecca9f48-credential-socket\") pod \"b398d9e9-9d78-4185-ac8c-4a9eecca9f48\" (UID: \"b398d9e9-9d78-4185-ac8c-4a9eecca9f48\") "
Oct 31 00:58:49.314269 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:49.313152    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b398d9e9-9d78-4185-ac8c-4a9eecca9f48-istio-token\" (UniqueName: \"kubernetes.io/projected/b398d9e9-9d78-4185-ac8c-4a9eecca9f48-istio-token\") pod \"b398d9e9-9d78-4185-ac8c-4a9eecca9f48\" (UID: \"b398d9e9-9d78-4185-ac8c-4a9eecca9f48\") "
Oct 31 00:58:49.314269 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:49.313288    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b398d9e9-9d78-4185-ac8c-4a9eecca9f48-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b398d9e9-9d78-4185-ac8c-4a9eecca9f48-workload-socket\") pod \"b398d9e9-9d78-4185-ac8c-4a9eecca9f48\" (UID: \"b398d9e9-9d78-4185-ac8c-4a9eecca9f48\") "
Oct 31 00:58:49.314269 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:49.313869    4284 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-grl28\" (UniqueName: \"kubernetes.io/projected/907b625f-2ccc-4de2-8181-34ff51ab2620-kube-api-access-grl28\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:58:49.314269 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:49.313896    4284 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/907b625f-2ccc-4de2-8181-34ff51ab2620-istio-envoy\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:58:49.314269 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:49.313909    4284 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6vpff\" (UniqueName: \"kubernetes.io/projected/6cce1cfc-7109-4827-a231-fe01dd9a4bf9-kube-api-access-6vpff\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:58:49.314269 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:49.313921    4284 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/907b625f-2ccc-4de2-8181-34ff51ab2620-istio-podinfo\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:58:49.314269 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:49.313936    4284 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6cce1cfc-7109-4827-a231-fe01dd9a4bf9-istiod-ca-cert\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:58:49.314269 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:49.313950    4284 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6cce1cfc-7109-4827-a231-fe01dd9a4bf9-istio-podinfo\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:58:49.314269 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:49.313962    4284 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6cce1cfc-7109-4827-a231-fe01dd9a4bf9-istio-envoy\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:58:49.314269 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:49.313975    4284 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6cce1cfc-7109-4827-a231-fe01dd9a4bf9-istio-token\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:58:49.314269 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:49.313990    4284 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/907b625f-2ccc-4de2-8181-34ff51ab2620-istio-token\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:58:49.314269 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:49.314003    4284 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6cce1cfc-7109-4827-a231-fe01dd9a4bf9-workload-socket\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:58:49.316071 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:49.315657    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b398d9e9-9d78-4185-ac8c-4a9eecca9f48-credential-socket" pod "b398d9e9-9d78-4185-ac8c-4a9eecca9f48" (UID: "b398d9e9-9d78-4185-ac8c-4a9eecca9f48"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:58:49.316071 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:49.315668    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b398d9e9-9d78-4185-ac8c-4a9eecca9f48-istio-data" pod "b398d9e9-9d78-4185-ac8c-4a9eecca9f48" (UID: "b398d9e9-9d78-4185-ac8c-4a9eecca9f48"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:58:49.316071 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:49.315736    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b398d9e9-9d78-4185-ac8c-4a9eecca9f48-workload-socket" pod "b398d9e9-9d78-4185-ac8c-4a9eecca9f48" (UID: "b398d9e9-9d78-4185-ac8c-4a9eecca9f48"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:58:49.316376 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:49.316117    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b398d9e9-9d78-4185-ac8c-4a9eecca9f48-istiod-ca-cert" pod "b398d9e9-9d78-4185-ac8c-4a9eecca9f48" (UID: "b398d9e9-9d78-4185-ac8c-4a9eecca9f48"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 00:58:49.316376 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:49.316299    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b398d9e9-9d78-4185-ac8c-4a9eecca9f48-workload-certs" pod "b398d9e9-9d78-4185-ac8c-4a9eecca9f48" (UID: "b398d9e9-9d78-4185-ac8c-4a9eecca9f48"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:58:49.319652 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:49.319543    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b398d9e9-9d78-4185-ac8c-4a9eecca9f48-kube-api-access-xkwp6" pod "b398d9e9-9d78-4185-ac8c-4a9eecca9f48" (UID: "b398d9e9-9d78-4185-ac8c-4a9eecca9f48"). InnerVolumeSpecName "kube-api-access-xkwp6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 00:58:49.319652 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:49.319617    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b398d9e9-9d78-4185-ac8c-4a9eecca9f48-istio-envoy" pod "b398d9e9-9d78-4185-ac8c-4a9eecca9f48" (UID: "b398d9e9-9d78-4185-ac8c-4a9eecca9f48"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:58:49.319850 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:49.319679    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b398d9e9-9d78-4185-ac8c-4a9eecca9f48-istio-podinfo" pod "b398d9e9-9d78-4185-ac8c-4a9eecca9f48" (UID: "b398d9e9-9d78-4185-ac8c-4a9eecca9f48"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 00:58:49.320058 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:49.320031    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b398d9e9-9d78-4185-ac8c-4a9eecca9f48-istio-token" pod "b398d9e9-9d78-4185-ac8c-4a9eecca9f48" (UID: "b398d9e9-9d78-4185-ac8c-4a9eecca9f48"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 00:58:49.419609 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:49.415934    4284 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b398d9e9-9d78-4185-ac8c-4a9eecca9f48-istio-podinfo\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:58:49.419609 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:49.415979    4284 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b398d9e9-9d78-4185-ac8c-4a9eecca9f48-istio-data\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:58:49.419609 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:49.415994    4284 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b398d9e9-9d78-4185-ac8c-4a9eecca9f48-workload-certs\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:58:49.419609 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:49.416009    4284 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b398d9e9-9d78-4185-ac8c-4a9eecca9f48-istiod-ca-cert\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:58:49.419609 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:49.416023    4284 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b398d9e9-9d78-4185-ac8c-4a9eecca9f48-credential-socket\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:58:49.419609 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:49.416038    4284 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b398d9e9-9d78-4185-ac8c-4a9eecca9f48-istio-token\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:58:49.419609 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:49.416052    4284 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b398d9e9-9d78-4185-ac8c-4a9eecca9f48-workload-socket\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:58:49.419609 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:49.416067    4284 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xkwp6\" (UniqueName: \"kubernetes.io/projected/b398d9e9-9d78-4185-ac8c-4a9eecca9f48-kube-api-access-xkwp6\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:58:49.419609 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:49.416082    4284 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b398d9e9-9d78-4185-ac8c-4a9eecca9f48-istio-envoy\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:58:49.752243 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:49.752143    4284 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-b647c3-openshift-default-674fcc8445-tvzlw"
Oct 31 00:58:49.752438 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:49.752418    4284 patch_prober.go:29] interesting pod/e2e-ipp-a-b647c3-openshift-default-674fcc8445-tvzlw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.109:15021/healthz/ready\": dial tcp 10.134.0.109:15021: connect: connection refused" start-of-body=
Oct 31 00:58:49.752489 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:49.752470    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-b647c3-openshift-default-674fcc8445-tvzlw" podUID="caf73a48-2172-46df-9cbc-fd85b49552c1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.109:15021/healthz/ready\": dial tcp 10.134.0.109:15021: connect: connection refused"
Oct 31 00:58:50.018686 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:50.018601    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-r7hr5"
Oct 31 00:58:50.019103 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:50.018601    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-r7hr5" event={"ID":"6cce1cfc-7109-4827-a231-fe01dd9a4bf9","Type":"ContainerDied","Data":"9da644d0541ae5f1391e81f9db8d0790c32cedc5f4125f2bbe34b83f7759914d"}
Oct 31 00:58:50.019103 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:50.018766    4284 scope.go:122] "RemoveContainer" containerID="7f51bafb86f4a8d560e43f5c45adb5803d37b3d0626fc00957bedeb001510d17"
Oct 31 00:58:50.020601 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:50.020329    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-d4s6f"
Oct 31 00:58:50.020601 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:50.020328    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-d4s6f" event={"ID":"b398d9e9-9d78-4185-ac8c-4a9eecca9f48","Type":"ContainerDied","Data":"05794f8793eea658e70157b252cf72341a1beda36ecb4e6ab561020aec311618"}
Oct 31 00:58:50.022583 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:50.022124    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-hvt6d"
Oct 31 00:58:50.022583 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:50.022148    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-hvt6d" event={"ID":"907b625f-2ccc-4de2-8181-34ff51ab2620","Type":"ContainerDied","Data":"eb62cd8f227970dd8f6e04676452a583440adcde365ebe8fd044a04d800e4bed"}
Oct 31 00:58:50.030734 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:50.030711    4284 scope.go:122] "RemoveContainer" containerID="532045cc890f889522b7e6812eb8a00980310d5d0f7b93cc2dab220d84eb52c8"
Oct 31 00:58:50.039995 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:50.039977    4284 scope.go:122] "RemoveContainer" containerID="1207c899f144a2a77001bd83adccc1f195da972fc68fa2cf9e2379f470b4102f"
Oct 31 00:58:50.177906 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:50.177872    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-d4s6f"]
Oct 31 00:58:50.225803 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:50.225758    4284 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-d4s6f"]
Oct 31 00:58:50.314962 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:50.314841    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-hvt6d"]
Oct 31 00:58:50.317873 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:50.317849    4284 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-hvt6d"]
Oct 31 00:58:50.415724 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:50.415689    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-r7hr5"]
Oct 31 00:58:50.418283 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:50.418264    4284 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-r7hr5"]
Oct 31 00:58:50.752742 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:50.752708    4284 patch_prober.go:29] interesting pod/e2e-ipp-a-b647c3-openshift-default-674fcc8445-tvzlw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.109:15021/healthz/ready\": dial tcp 10.134.0.109:15021: connect: connection refused" start-of-body=
Oct 31 00:58:50.752911 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:50.752788    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-b647c3-openshift-default-674fcc8445-tvzlw" podUID="caf73a48-2172-46df-9cbc-fd85b49552c1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.109:15021/healthz/ready\": dial tcp 10.134.0.109:15021: connect: connection refused"
Oct 31 00:58:51.752745 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:51.752708    4284 patch_prober.go:29] interesting pod/e2e-ipp-a-b647c3-openshift-default-674fcc8445-tvzlw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.109:15021/healthz/ready\": dial tcp 10.134.0.109:15021: connect: connection refused" start-of-body=
Oct 31 00:58:51.753145 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:51.752760    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-b647c3-openshift-default-674fcc8445-tvzlw" podUID="caf73a48-2172-46df-9cbc-fd85b49552c1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.109:15021/healthz/ready\": dial tcp 10.134.0.109:15021: connect: connection refused"
Oct 31 00:58:51.971143 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:51.971099    4284 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6cce1cfc-7109-4827-a231-fe01dd9a4bf9" path="/var/lib/kubelet/pods/6cce1cfc-7109-4827-a231-fe01dd9a4bf9/volumes"
Oct 31 00:58:51.971668 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:51.971647    4284 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="907b625f-2ccc-4de2-8181-34ff51ab2620" path="/var/lib/kubelet/pods/907b625f-2ccc-4de2-8181-34ff51ab2620/volumes"
Oct 31 00:58:51.972115 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:51.972096    4284 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b398d9e9-9d78-4185-ac8c-4a9eecca9f48" path="/var/lib/kubelet/pods/b398d9e9-9d78-4185-ac8c-4a9eecca9f48/volumes"
Oct 31 00:58:52.270888 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:52.270854    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-e942ab-586494d55c-84g9q"]
Oct 31 00:58:52.271115 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:52.271074    4284 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e942ab-586494d55c-84g9q" podUID="288ecf4a-e496-46e4-9139-b026bca0157d" containerName="maas-api" containerID="cri-o://402bab6943c504c437f23b402d720897fc94c031563898b1961fce1e4fe1cd48" gracePeriod=30
Oct 31 00:58:52.369696 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:52.369660    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:58:52.682161 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:52.682131    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e942ab-586494d55c-84g9q"
Oct 31 00:58:52.751081 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:52.751049    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/288ecf4a-e496-46e4-9139-b026bca0157d-metrics-certs\" (UniqueName: \"kubernetes.io/secret/288ecf4a-e496-46e4-9139-b026bca0157d-metrics-certs\") pod \"288ecf4a-e496-46e4-9139-b026bca0157d\" (UID: \"288ecf4a-e496-46e4-9139-b026bca0157d\") "
Oct 31 00:58:52.751301 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:52.751097    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/288ecf4a-e496-46e4-9139-b026bca0157d-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/288ecf4a-e496-46e4-9139-b026bca0157d-openshift-service-ca\") pod \"288ecf4a-e496-46e4-9139-b026bca0157d\" (UID: \"288ecf4a-e496-46e4-9139-b026bca0157d\") "
Oct 31 00:58:52.751301 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:52.751243    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/288ecf4a-e496-46e4-9139-b026bca0157d-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/288ecf4a-e496-46e4-9139-b026bca0157d-maas-api-tls\") pod \"288ecf4a-e496-46e4-9139-b026bca0157d\" (UID: \"288ecf4a-e496-46e4-9139-b026bca0157d\") "
Oct 31 00:58:52.751301 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:52.751284    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/288ecf4a-e496-46e4-9139-b026bca0157d-kube-api-access-tjjqk\" (UniqueName: \"kubernetes.io/projected/288ecf4a-e496-46e4-9139-b026bca0157d-kube-api-access-tjjqk\") pod \"288ecf4a-e496-46e4-9139-b026bca0157d\" (UID: \"288ecf4a-e496-46e4-9139-b026bca0157d\") "
Oct 31 00:58:52.751519 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:52.751496    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/288ecf4a-e496-46e4-9139-b026bca0157d-openshift-service-ca" pod "288ecf4a-e496-46e4-9139-b026bca0157d" (UID: "288ecf4a-e496-46e4-9139-b026bca0157d"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 00:58:52.751677 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:52.751659    4284 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/288ecf4a-e496-46e4-9139-b026bca0157d-openshift-service-ca\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:58:52.752371 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:52.752344    4284 patch_prober.go:29] interesting pod/e2e-ipp-a-b647c3-openshift-default-674fcc8445-tvzlw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.109:15021/healthz/ready\": dial tcp 10.134.0.109:15021: connect: connection refused" start-of-body=
Oct 31 00:58:52.752480 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:52.752403    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-b647c3-openshift-default-674fcc8445-tvzlw" podUID="caf73a48-2172-46df-9cbc-fd85b49552c1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.109:15021/healthz/ready\": dial tcp 10.134.0.109:15021: connect: connection refused"
Oct 31 00:58:52.753740 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:52.753700    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/288ecf4a-e496-46e4-9139-b026bca0157d-metrics-certs" pod "288ecf4a-e496-46e4-9139-b026bca0157d" (UID: "288ecf4a-e496-46e4-9139-b026bca0157d"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 00:58:52.754075 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:52.753740    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/288ecf4a-e496-46e4-9139-b026bca0157d-kube-api-access-tjjqk" pod "288ecf4a-e496-46e4-9139-b026bca0157d" (UID: "288ecf4a-e496-46e4-9139-b026bca0157d"). InnerVolumeSpecName "kube-api-access-tjjqk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 00:58:52.754075 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:52.753785    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/288ecf4a-e496-46e4-9139-b026bca0157d-maas-api-tls" pod "288ecf4a-e496-46e4-9139-b026bca0157d" (UID: "288ecf4a-e496-46e4-9139-b026bca0157d"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 00:58:52.872618 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:52.871961    4284 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/288ecf4a-e496-46e4-9139-b026bca0157d-maas-api-tls\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:58:52.872618 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:52.871994    4284 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tjjqk\" (UniqueName: \"kubernetes.io/projected/288ecf4a-e496-46e4-9139-b026bca0157d-kube-api-access-tjjqk\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:58:52.872618 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:52.872005    4284 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/288ecf4a-e496-46e4-9139-b026bca0157d-metrics-certs\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:58:53.043241 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:53.043120    4284 generic.go:356] "Generic (PLEG): container finished" podID="288ecf4a-e496-46e4-9139-b026bca0157d" containerID="402bab6943c504c437f23b402d720897fc94c031563898b1961fce1e4fe1cd48" exitCode=0
Oct 31 00:58:53.043241 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:53.043165    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e942ab-586494d55c-84g9q"
Oct 31 00:58:53.043241 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:53.043197    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e942ab-586494d55c-84g9q" event={"ID":"288ecf4a-e496-46e4-9139-b026bca0157d","Type":"ContainerDied","Data":"402bab6943c504c437f23b402d720897fc94c031563898b1961fce1e4fe1cd48"}
Oct 31 00:58:53.043511 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:53.043270    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e942ab-586494d55c-84g9q" event={"ID":"288ecf4a-e496-46e4-9139-b026bca0157d","Type":"ContainerDied","Data":"8f47dd74dae175dcec0bbf1c69601e659ede8b51dd676313cc116b8daa5c6fcf"}
Oct 31 00:58:53.043511 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:53.043295    4284 scope.go:122] "RemoveContainer" containerID="402bab6943c504c437f23b402d720897fc94c031563898b1961fce1e4fe1cd48"
Oct 31 00:58:53.054435 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:53.054418    4284 scope.go:122] "RemoveContainer" containerID="402bab6943c504c437f23b402d720897fc94c031563898b1961fce1e4fe1cd48"
Oct 31 00:58:53.054722 ip-10-0-132-173 kubenswrapper[4284]: E1031 00:58:53.054699    4284 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"402bab6943c504c437f23b402d720897fc94c031563898b1961fce1e4fe1cd48\": container with ID starting with 402bab6943c504c437f23b402d720897fc94c031563898b1961fce1e4fe1cd48 not found: ID does not exist" containerID="402bab6943c504c437f23b402d720897fc94c031563898b1961fce1e4fe1cd48"
Oct 31 00:58:53.054795 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:53.054730    4284 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"402bab6943c504c437f23b402d720897fc94c031563898b1961fce1e4fe1cd48"} err="failed to get container status \"402bab6943c504c437f23b402d720897fc94c031563898b1961fce1e4fe1cd48\": rpc error: code = NotFound desc = could not find container \"402bab6943c504c437f23b402d720897fc94c031563898b1961fce1e4fe1cd48\": container with ID starting with 402bab6943c504c437f23b402d720897fc94c031563898b1961fce1e4fe1cd48 not found: ID does not exist"
Oct 31 00:58:53.216289 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:53.216201    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-e942ab-586494d55c-84g9q"]
Oct 31 00:58:53.234820 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:53.234788    4284 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-e942ab-586494d55c-84g9q"]
Oct 31 00:58:53.752645 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:53.752611    4284 patch_prober.go:29] interesting pod/e2e-ipp-a-b647c3-openshift-default-674fcc8445-tvzlw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.109:15021/healthz/ready\": dial tcp 10.134.0.109:15021: connect: connection refused" start-of-body=
Oct 31 00:58:53.752825 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:53.752680    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-b647c3-openshift-default-674fcc8445-tvzlw" podUID="caf73a48-2172-46df-9cbc-fd85b49552c1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.109:15021/healthz/ready\": dial tcp 10.134.0.109:15021: connect: connection refused"
Oct 31 00:58:53.866921 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:53.866884    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:58:53.964322 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:53.964291    4284 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="288ecf4a-e496-46e4-9139-b026bca0157d" path="/var/lib/kubelet/pods/288ecf4a-e496-46e4-9139-b026bca0157d/volumes"
Oct 31 00:58:54.077237 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:54.077124    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:58:54.268238 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:54.267821    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:58:54.753158 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:54.753109    4284 patch_prober.go:29] interesting pod/e2e-ipp-a-b647c3-openshift-default-674fcc8445-tvzlw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.109:15021/healthz/ready\": dial tcp 10.134.0.109:15021: connect: connection refused" start-of-body=
Oct 31 00:58:54.753413 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:54.753168    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-b647c3-openshift-default-674fcc8445-tvzlw" podUID="caf73a48-2172-46df-9cbc-fd85b49552c1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.109:15021/healthz/ready\": dial tcp 10.134.0.109:15021: connect: connection refused"
Oct 31 00:58:54.776882 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:54.776848    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:58:55.755946 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:55.755915    4284 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-b647c3-openshift-default-674fcc8445-tvzlw"
Oct 31 00:58:55.756450 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:55.756235    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-b647c3-openshift-default-674fcc8445-tvzlw"
Oct 31 00:58:55.756886 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:55.756871    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-b647c3-openshift-default-674fcc8445-tvzlw"
Oct 31 00:58:59.258966 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:59.258928    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:58:59.444488 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:58:59.444455    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:59:01.188577 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:01.188536    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:59:01.380667 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:01.380581    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:59:01.542794 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:01.542756    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:59:02.233100 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:02.233057    4284 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-ipp-a-ac952a/ipp-route-ac952a-kserve-75c45558d9-fxngk"]
Oct 31 00:59:02.233889 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:02.233870    4284 cpu_manager.go:414] "containerMap: removing container" podUID="288ecf4a-e496-46e4-9139-b026bca0157d" containerName="maas-api"
Oct 31 00:59:02.233934 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:02.233891    4284 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="288ecf4a-e496-46e4-9139-b026bca0157d" containerName="maas-api"
Oct 31 00:59:02.233934 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:02.233907    4284 cpu_manager.go:414] "containerMap: removing container" podUID="a8b1b169-c9ee-4dea-91cc-ec434f44b05c" containerName="payload-processing"
Oct 31 00:59:02.233934 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:02.233912    4284 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a8b1b169-c9ee-4dea-91cc-ec434f44b05c" containerName="payload-processing"
Oct 31 00:59:02.233934 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:02.233922    4284 cpu_manager.go:414] "containerMap: removing container" podUID="6cce1cfc-7109-4827-a231-fe01dd9a4bf9" containerName="istio-proxy"
Oct 31 00:59:02.233934 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:02.233927    4284 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6cce1cfc-7109-4827-a231-fe01dd9a4bf9" containerName="istio-proxy"
Oct 31 00:59:02.234094 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:02.233940    4284 cpu_manager.go:414] "containerMap: removing container" podUID="b58c53d0-4acf-4de9-b322-8ead4dc9b013" containerName="istio-proxy"
Oct 31 00:59:02.234094 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:02.233945    4284 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b58c53d0-4acf-4de9-b322-8ead4dc9b013" containerName="istio-proxy"
Oct 31 00:59:02.234094 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:02.233964    4284 cpu_manager.go:414] "containerMap: removing container" podUID="b398d9e9-9d78-4185-ac8c-4a9eecca9f48" containerName="istio-proxy"
Oct 31 00:59:02.234094 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:02.233970    4284 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b398d9e9-9d78-4185-ac8c-4a9eecca9f48" containerName="istio-proxy"
Oct 31 00:59:02.234094 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:02.233985    4284 cpu_manager.go:414] "containerMap: removing container" podUID="907b625f-2ccc-4de2-8181-34ff51ab2620" containerName="istio-proxy"
Oct 31 00:59:02.234094 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:02.233989    4284 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="907b625f-2ccc-4de2-8181-34ff51ab2620" containerName="istio-proxy"
Oct 31 00:59:02.234094 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:02.234067    4284 memory_manager.go:365] "RemoveStaleState removing state" podUID="288ecf4a-e496-46e4-9139-b026bca0157d" containerName="maas-api"
Oct 31 00:59:02.234094 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:02.234074    4284 memory_manager.go:365] "RemoveStaleState removing state" podUID="b398d9e9-9d78-4185-ac8c-4a9eecca9f48" containerName="istio-proxy"
Oct 31 00:59:02.234094 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:02.234080    4284 memory_manager.go:365] "RemoveStaleState removing state" podUID="b58c53d0-4acf-4de9-b322-8ead4dc9b013" containerName="istio-proxy"
Oct 31 00:59:02.234094 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:02.234089    4284 memory_manager.go:365] "RemoveStaleState removing state" podUID="a8b1b169-c9ee-4dea-91cc-ec434f44b05c" containerName="payload-processing"
Oct 31 00:59:02.234094 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:02.234094    4284 memory_manager.go:365] "RemoveStaleState removing state" podUID="6cce1cfc-7109-4827-a231-fe01dd9a4bf9" containerName="istio-proxy"
Oct 31 00:59:02.234434 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:02.234100    4284 memory_manager.go:365] "RemoveStaleState removing state" podUID="907b625f-2ccc-4de2-8181-34ff51ab2620" containerName="istio-proxy"
Oct 31 00:59:02.236800 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:02.236783    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-ac952a/ipp-route-ac952a-kserve-75c45558d9-fxngk"
Oct 31 00:59:02.290346 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:02.290315    4284 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-ipp-a-ac952a\"/\"default-dockercfg-st9vc\""
Oct 31 00:59:02.290346 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:02.290332    4284 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-ipp-a-ac952a\"/\"openshift-service-ca.crt\""
Oct 31 00:59:02.335127 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:02.335090    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5e194b3e-7d44-437b-a50a-0b64a656d855-home\") pod \"ipp-route-ac952a-kserve-75c45558d9-fxngk\" (UID: \"5e194b3e-7d44-437b-a50a-0b64a656d855\") " pod="ai-tenant-e2e-ipp-a-ac952a/ipp-route-ac952a-kserve-75c45558d9-fxngk"
Oct 31 00:59:02.335325 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:02.335189    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5e194b3e-7d44-437b-a50a-0b64a656d855-tmp-dir\") pod \"ipp-route-ac952a-kserve-75c45558d9-fxngk\" (UID: \"5e194b3e-7d44-437b-a50a-0b64a656d855\") " pod="ai-tenant-e2e-ipp-a-ac952a/ipp-route-ac952a-kserve-75c45558d9-fxngk"
Oct 31 00:59:02.335325 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:02.335261    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5e194b3e-7d44-437b-a50a-0b64a656d855-model-cache\") pod \"ipp-route-ac952a-kserve-75c45558d9-fxngk\" (UID: \"5e194b3e-7d44-437b-a50a-0b64a656d855\") " pod="ai-tenant-e2e-ipp-a-ac952a/ipp-route-ac952a-kserve-75c45558d9-fxngk"
Oct 31 00:59:02.335325 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:02.335305    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5e194b3e-7d44-437b-a50a-0b64a656d855-tls-certs\") pod \"ipp-route-ac952a-kserve-75c45558d9-fxngk\" (UID: \"5e194b3e-7d44-437b-a50a-0b64a656d855\") " pod="ai-tenant-e2e-ipp-a-ac952a/ipp-route-ac952a-kserve-75c45558d9-fxngk"
Oct 31 00:59:02.335488 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:02.335443    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5e194b3e-7d44-437b-a50a-0b64a656d855-dshm\") pod \"ipp-route-ac952a-kserve-75c45558d9-fxngk\" (UID: \"5e194b3e-7d44-437b-a50a-0b64a656d855\") " pod="ai-tenant-e2e-ipp-a-ac952a/ipp-route-ac952a-kserve-75c45558d9-fxngk"
Oct 31 00:59:02.335686 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:02.335664    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jjxvg\" (UniqueName: \"kubernetes.io/projected/5e194b3e-7d44-437b-a50a-0b64a656d855-kube-api-access-jjxvg\") pod \"ipp-route-ac952a-kserve-75c45558d9-fxngk\" (UID: \"5e194b3e-7d44-437b-a50a-0b64a656d855\") " pod="ai-tenant-e2e-ipp-a-ac952a/ipp-route-ac952a-kserve-75c45558d9-fxngk"
Oct 31 00:59:02.338223 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:02.338176    4284 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-ipp-a-ac952a\"/\"kube-root-ca.crt\""
Oct 31 00:59:02.338365 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:02.338344    4284 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-ipp-a-ac952a\"/\"ipp-route-ac952a-kserve-self-signed-certs\""
Oct 31 00:59:02.338731 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:02.338708    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:59:02.341475 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:02.341455    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-ipp-a-ac952a/ipp-route-ac952a-kserve-75c45558d9-fxngk"]
Oct 31 00:59:02.436633 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:02.436592    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jjxvg\" (UniqueName: \"kubernetes.io/projected/5e194b3e-7d44-437b-a50a-0b64a656d855-kube-api-access-jjxvg\") pod \"ipp-route-ac952a-kserve-75c45558d9-fxngk\" (UID: \"5e194b3e-7d44-437b-a50a-0b64a656d855\") " pod="ai-tenant-e2e-ipp-a-ac952a/ipp-route-ac952a-kserve-75c45558d9-fxngk"
Oct 31 00:59:02.436860 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:02.436662    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5e194b3e-7d44-437b-a50a-0b64a656d855-home\") pod \"ipp-route-ac952a-kserve-75c45558d9-fxngk\" (UID: \"5e194b3e-7d44-437b-a50a-0b64a656d855\") " pod="ai-tenant-e2e-ipp-a-ac952a/ipp-route-ac952a-kserve-75c45558d9-fxngk"
Oct 31 00:59:02.436860 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:02.436707    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5e194b3e-7d44-437b-a50a-0b64a656d855-tmp-dir\") pod \"ipp-route-ac952a-kserve-75c45558d9-fxngk\" (UID: \"5e194b3e-7d44-437b-a50a-0b64a656d855\") " pod="ai-tenant-e2e-ipp-a-ac952a/ipp-route-ac952a-kserve-75c45558d9-fxngk"
Oct 31 00:59:02.436860 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:02.436762    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5e194b3e-7d44-437b-a50a-0b64a656d855-model-cache\") pod \"ipp-route-ac952a-kserve-75c45558d9-fxngk\" (UID: \"5e194b3e-7d44-437b-a50a-0b64a656d855\") " pod="ai-tenant-e2e-ipp-a-ac952a/ipp-route-ac952a-kserve-75c45558d9-fxngk"
Oct 31 00:59:02.436860 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:02.436813    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5e194b3e-7d44-437b-a50a-0b64a656d855-tls-certs\") pod \"ipp-route-ac952a-kserve-75c45558d9-fxngk\" (UID: \"5e194b3e-7d44-437b-a50a-0b64a656d855\") " pod="ai-tenant-e2e-ipp-a-ac952a/ipp-route-ac952a-kserve-75c45558d9-fxngk"
Oct 31 00:59:02.436860 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:02.436839    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5e194b3e-7d44-437b-a50a-0b64a656d855-dshm\") pod \"ipp-route-ac952a-kserve-75c45558d9-fxngk\" (UID: \"5e194b3e-7d44-437b-a50a-0b64a656d855\") " pod="ai-tenant-e2e-ipp-a-ac952a/ipp-route-ac952a-kserve-75c45558d9-fxngk"
Oct 31 00:59:02.437619 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:02.437590    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5e194b3e-7d44-437b-a50a-0b64a656d855-tmp-dir\") pod \"ipp-route-ac952a-kserve-75c45558d9-fxngk\" (UID: \"5e194b3e-7d44-437b-a50a-0b64a656d855\") " pod="ai-tenant-e2e-ipp-a-ac952a/ipp-route-ac952a-kserve-75c45558d9-fxngk"
Oct 31 00:59:02.438111 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:02.438086    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5e194b3e-7d44-437b-a50a-0b64a656d855-home\") pod \"ipp-route-ac952a-kserve-75c45558d9-fxngk\" (UID: \"5e194b3e-7d44-437b-a50a-0b64a656d855\") " pod="ai-tenant-e2e-ipp-a-ac952a/ipp-route-ac952a-kserve-75c45558d9-fxngk"
Oct 31 00:59:02.438268 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:02.438250    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5e194b3e-7d44-437b-a50a-0b64a656d855-model-cache\") pod \"ipp-route-ac952a-kserve-75c45558d9-fxngk\" (UID: \"5e194b3e-7d44-437b-a50a-0b64a656d855\") " pod="ai-tenant-e2e-ipp-a-ac952a/ipp-route-ac952a-kserve-75c45558d9-fxngk"
Oct 31 00:59:02.439654 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:02.439630    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5e194b3e-7d44-437b-a50a-0b64a656d855-dshm\") pod \"ipp-route-ac952a-kserve-75c45558d9-fxngk\" (UID: \"5e194b3e-7d44-437b-a50a-0b64a656d855\") " pod="ai-tenant-e2e-ipp-a-ac952a/ipp-route-ac952a-kserve-75c45558d9-fxngk"
Oct 31 00:59:02.440721 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:02.440704    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5e194b3e-7d44-437b-a50a-0b64a656d855-tls-certs\") pod \"ipp-route-ac952a-kserve-75c45558d9-fxngk\" (UID: \"5e194b3e-7d44-437b-a50a-0b64a656d855\") " pod="ai-tenant-e2e-ipp-a-ac952a/ipp-route-ac952a-kserve-75c45558d9-fxngk"
Oct 31 00:59:02.500268 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:02.500185    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jjxvg\" (UniqueName: \"kubernetes.io/projected/5e194b3e-7d44-437b-a50a-0b64a656d855-kube-api-access-jjxvg\") pod \"ipp-route-ac952a-kserve-75c45558d9-fxngk\" (UID: \"5e194b3e-7d44-437b-a50a-0b64a656d855\") " pod="ai-tenant-e2e-ipp-a-ac952a/ipp-route-ac952a-kserve-75c45558d9-fxngk"
Oct 31 00:59:02.548141 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:02.548098    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-ac952a/ipp-route-ac952a-kserve-75c45558d9-fxngk"
Oct 31 00:59:02.922851 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:02.922825    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-ipp-a-ac952a/ipp-route-ac952a-kserve-75c45558d9-fxngk"]
Oct 31 00:59:02.923427 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:59:02.923405    4284 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5e194b3e_7d44_437b_a50a_0b64a656d855.slice/crio-4d3056937dc8374606e09d68c8b5bbfdc28ab999a6b7d17df7c3b8282cfc97f1 WatchSource:0}: Error finding container 4d3056937dc8374606e09d68c8b5bbfdc28ab999a6b7d17df7c3b8282cfc97f1: Status 404 returned error can't find the container with id 4d3056937dc8374606e09d68c8b5bbfdc28ab999a6b7d17df7c3b8282cfc97f1
Oct 31 00:59:03.109651 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:03.109614    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-ac952a/ipp-route-ac952a-kserve-75c45558d9-fxngk" event={"ID":"5e194b3e-7d44-437b-a50a-0b64a656d855","Type":"ContainerStarted","Data":"b2d1d7e11c0337b0f222660ff04e257e1fc42c322f8986c7c84b7a9029463657"}
Oct 31 00:59:03.109651 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:03.109657    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-ac952a/ipp-route-ac952a-kserve-75c45558d9-fxngk" event={"ID":"5e194b3e-7d44-437b-a50a-0b64a656d855","Type":"ContainerStarted","Data":"4d3056937dc8374606e09d68c8b5bbfdc28ab999a6b7d17df7c3b8282cfc97f1"}
Oct 31 00:59:03.229408 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:03.229297    4284 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="ai-tenant-e2e-ipp-a-ac952a/ipp-route-ac952a-kserve-75c45558d9-fxngk" podStartSLOduration=1.229281292 podStartE2EDuration="1.229281292s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 00:59:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:59:03.227058709 +0000 UTC m=+1857.372065624" watchObservedRunningTime="2026-10-31 00:59:03.229281292 +0000 UTC m=+1857.374288207"
Oct 31 00:59:03.464431 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:03.464389    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:59:03.548996 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:03.548920    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-ipp-a-ac952a/ipp-route-ac952a-kserve-75c45558d9-fxngk"
Oct 31 00:59:04.034368 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:04.034333    4284 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-praxis-a-8c08e7-openshift-default-5d56d76c65-qnh6x"]
Oct 31 00:59:04.037743 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:04.037716    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-praxis-a-8c08e7-openshift-default-5d56d76c65-qnh6x"
Oct 31 00:59:04.090921 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:04.090890    4284 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-praxis-a-8c08e7-openshift-default-dockercfg-76t7c\""
Oct 31 00:59:04.119282 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:04.119252    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:59:04.142896 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:04.142861    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-praxis-a-8c08e7-openshift-default-5d56d76c65-qnh6x"]
Oct 31 00:59:04.233240 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:04.199272    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/16131784-3cb6-4d01-a27f-6ce599437f3c-workload-certs\") pod \"e2e-praxis-a-8c08e7-openshift-default-5d56d76c65-qnh6x\" (UID: \"16131784-3cb6-4d01-a27f-6ce599437f3c\") " pod="openshift-ingress/e2e-praxis-a-8c08e7-openshift-default-5d56d76c65-qnh6x"
Oct 31 00:59:04.233240 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:04.225345    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/16131784-3cb6-4d01-a27f-6ce599437f3c-credential-socket\") pod \"e2e-praxis-a-8c08e7-openshift-default-5d56d76c65-qnh6x\" (UID: \"16131784-3cb6-4d01-a27f-6ce599437f3c\") " pod="openshift-ingress/e2e-praxis-a-8c08e7-openshift-default-5d56d76c65-qnh6x"
Oct 31 00:59:04.233240 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:04.225413    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gh8xd\" (UniqueName: \"kubernetes.io/projected/16131784-3cb6-4d01-a27f-6ce599437f3c-kube-api-access-gh8xd\") pod \"e2e-praxis-a-8c08e7-openshift-default-5d56d76c65-qnh6x\" (UID: \"16131784-3cb6-4d01-a27f-6ce599437f3c\") " pod="openshift-ingress/e2e-praxis-a-8c08e7-openshift-default-5d56d76c65-qnh6x"
Oct 31 00:59:04.233240 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:04.225451    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/16131784-3cb6-4d01-a27f-6ce599437f3c-istiod-ca-cert\") pod \"e2e-praxis-a-8c08e7-openshift-default-5d56d76c65-qnh6x\" (UID: \"16131784-3cb6-4d01-a27f-6ce599437f3c\") " pod="openshift-ingress/e2e-praxis-a-8c08e7-openshift-default-5d56d76c65-qnh6x"
Oct 31 00:59:04.233240 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:04.225515    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/16131784-3cb6-4d01-a27f-6ce599437f3c-istio-data\") pod \"e2e-praxis-a-8c08e7-openshift-default-5d56d76c65-qnh6x\" (UID: \"16131784-3cb6-4d01-a27f-6ce599437f3c\") " pod="openshift-ingress/e2e-praxis-a-8c08e7-openshift-default-5d56d76c65-qnh6x"
Oct 31 00:59:04.233240 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:04.225551    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/16131784-3cb6-4d01-a27f-6ce599437f3c-istio-envoy\") pod \"e2e-praxis-a-8c08e7-openshift-default-5d56d76c65-qnh6x\" (UID: \"16131784-3cb6-4d01-a27f-6ce599437f3c\") " pod="openshift-ingress/e2e-praxis-a-8c08e7-openshift-default-5d56d76c65-qnh6x"
Oct 31 00:59:04.233240 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:04.225626    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/16131784-3cb6-4d01-a27f-6ce599437f3c-istio-podinfo\") pod \"e2e-praxis-a-8c08e7-openshift-default-5d56d76c65-qnh6x\" (UID: \"16131784-3cb6-4d01-a27f-6ce599437f3c\") " pod="openshift-ingress/e2e-praxis-a-8c08e7-openshift-default-5d56d76c65-qnh6x"
Oct 31 00:59:04.233240 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:04.225655    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/16131784-3cb6-4d01-a27f-6ce599437f3c-istio-token\") pod \"e2e-praxis-a-8c08e7-openshift-default-5d56d76c65-qnh6x\" (UID: \"16131784-3cb6-4d01-a27f-6ce599437f3c\") " pod="openshift-ingress/e2e-praxis-a-8c08e7-openshift-default-5d56d76c65-qnh6x"
Oct 31 00:59:04.233240 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:04.225789    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/16131784-3cb6-4d01-a27f-6ce599437f3c-workload-socket\") pod \"e2e-praxis-a-8c08e7-openshift-default-5d56d76c65-qnh6x\" (UID: \"16131784-3cb6-4d01-a27f-6ce599437f3c\") " pod="openshift-ingress/e2e-praxis-a-8c08e7-openshift-default-5d56d76c65-qnh6x"
Oct 31 00:59:04.309740 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:04.309649    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:59:04.329309 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:04.329253    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/16131784-3cb6-4d01-a27f-6ce599437f3c-istio-envoy\") pod \"e2e-praxis-a-8c08e7-openshift-default-5d56d76c65-qnh6x\" (UID: \"16131784-3cb6-4d01-a27f-6ce599437f3c\") " pod="openshift-ingress/e2e-praxis-a-8c08e7-openshift-default-5d56d76c65-qnh6x"
Oct 31 00:59:04.330303 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:04.330275    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/16131784-3cb6-4d01-a27f-6ce599437f3c-istio-podinfo\") pod \"e2e-praxis-a-8c08e7-openshift-default-5d56d76c65-qnh6x\" (UID: \"16131784-3cb6-4d01-a27f-6ce599437f3c\") " pod="openshift-ingress/e2e-praxis-a-8c08e7-openshift-default-5d56d76c65-qnh6x"
Oct 31 00:59:04.330973 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:04.330953    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/16131784-3cb6-4d01-a27f-6ce599437f3c-istio-token\") pod \"e2e-praxis-a-8c08e7-openshift-default-5d56d76c65-qnh6x\" (UID: \"16131784-3cb6-4d01-a27f-6ce599437f3c\") " pod="openshift-ingress/e2e-praxis-a-8c08e7-openshift-default-5d56d76c65-qnh6x"
Oct 31 00:59:04.337831 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:04.337771    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/16131784-3cb6-4d01-a27f-6ce599437f3c-workload-socket\") pod \"e2e-praxis-a-8c08e7-openshift-default-5d56d76c65-qnh6x\" (UID: \"16131784-3cb6-4d01-a27f-6ce599437f3c\") " pod="openshift-ingress/e2e-praxis-a-8c08e7-openshift-default-5d56d76c65-qnh6x"
Oct 31 00:59:04.338755 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:04.338732    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/16131784-3cb6-4d01-a27f-6ce599437f3c-workload-certs\") pod \"e2e-praxis-a-8c08e7-openshift-default-5d56d76c65-qnh6x\" (UID: \"16131784-3cb6-4d01-a27f-6ce599437f3c\") " pod="openshift-ingress/e2e-praxis-a-8c08e7-openshift-default-5d56d76c65-qnh6x"
Oct 31 00:59:04.339229 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:04.338470    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/16131784-3cb6-4d01-a27f-6ce599437f3c-workload-socket\") pod \"e2e-praxis-a-8c08e7-openshift-default-5d56d76c65-qnh6x\" (UID: \"16131784-3cb6-4d01-a27f-6ce599437f3c\") " pod="openshift-ingress/e2e-praxis-a-8c08e7-openshift-default-5d56d76c65-qnh6x"
Oct 31 00:59:04.339347 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:04.339327    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/16131784-3cb6-4d01-a27f-6ce599437f3c-credential-socket\") pod \"e2e-praxis-a-8c08e7-openshift-default-5d56d76c65-qnh6x\" (UID: \"16131784-3cb6-4d01-a27f-6ce599437f3c\") " pod="openshift-ingress/e2e-praxis-a-8c08e7-openshift-default-5d56d76c65-qnh6x"
Oct 31 00:59:04.339426 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:04.339406    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gh8xd\" (UniqueName: \"kubernetes.io/projected/16131784-3cb6-4d01-a27f-6ce599437f3c-kube-api-access-gh8xd\") pod \"e2e-praxis-a-8c08e7-openshift-default-5d56d76c65-qnh6x\" (UID: \"16131784-3cb6-4d01-a27f-6ce599437f3c\") " pod="openshift-ingress/e2e-praxis-a-8c08e7-openshift-default-5d56d76c65-qnh6x"
Oct 31 00:59:04.339501 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:04.339442    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/16131784-3cb6-4d01-a27f-6ce599437f3c-istiod-ca-cert\") pod \"e2e-praxis-a-8c08e7-openshift-default-5d56d76c65-qnh6x\" (UID: \"16131784-3cb6-4d01-a27f-6ce599437f3c\") " pod="openshift-ingress/e2e-praxis-a-8c08e7-openshift-default-5d56d76c65-qnh6x"
Oct 31 00:59:04.339553 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:04.339517    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/16131784-3cb6-4d01-a27f-6ce599437f3c-istio-data\") pod \"e2e-praxis-a-8c08e7-openshift-default-5d56d76c65-qnh6x\" (UID: \"16131784-3cb6-4d01-a27f-6ce599437f3c\") " pod="openshift-ingress/e2e-praxis-a-8c08e7-openshift-default-5d56d76c65-qnh6x"
Oct 31 00:59:04.339839 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:04.339806    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/16131784-3cb6-4d01-a27f-6ce599437f3c-istio-data\") pod \"e2e-praxis-a-8c08e7-openshift-default-5d56d76c65-qnh6x\" (UID: \"16131784-3cb6-4d01-a27f-6ce599437f3c\") " pod="openshift-ingress/e2e-praxis-a-8c08e7-openshift-default-5d56d76c65-qnh6x"
Oct 31 00:59:04.339839 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:04.339138    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/16131784-3cb6-4d01-a27f-6ce599437f3c-workload-certs\") pod \"e2e-praxis-a-8c08e7-openshift-default-5d56d76c65-qnh6x\" (UID: \"16131784-3cb6-4d01-a27f-6ce599437f3c\") " pod="openshift-ingress/e2e-praxis-a-8c08e7-openshift-default-5d56d76c65-qnh6x"
Oct 31 00:59:04.340195 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:04.340066    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/16131784-3cb6-4d01-a27f-6ce599437f3c-credential-socket\") pod \"e2e-praxis-a-8c08e7-openshift-default-5d56d76c65-qnh6x\" (UID: \"16131784-3cb6-4d01-a27f-6ce599437f3c\") " pod="openshift-ingress/e2e-praxis-a-8c08e7-openshift-default-5d56d76c65-qnh6x"
Oct 31 00:59:04.340822 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:04.340801    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/16131784-3cb6-4d01-a27f-6ce599437f3c-istiod-ca-cert\") pod \"e2e-praxis-a-8c08e7-openshift-default-5d56d76c65-qnh6x\" (UID: \"16131784-3cb6-4d01-a27f-6ce599437f3c\") " pod="openshift-ingress/e2e-praxis-a-8c08e7-openshift-default-5d56d76c65-qnh6x"
Oct 31 00:59:04.360465 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:04.358993    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/16131784-3cb6-4d01-a27f-6ce599437f3c-istio-envoy\") pod \"e2e-praxis-a-8c08e7-openshift-default-5d56d76c65-qnh6x\" (UID: \"16131784-3cb6-4d01-a27f-6ce599437f3c\") " pod="openshift-ingress/e2e-praxis-a-8c08e7-openshift-default-5d56d76c65-qnh6x"
Oct 31 00:59:04.360465 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:04.360226    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/16131784-3cb6-4d01-a27f-6ce599437f3c-istio-podinfo\") pod \"e2e-praxis-a-8c08e7-openshift-default-5d56d76c65-qnh6x\" (UID: \"16131784-3cb6-4d01-a27f-6ce599437f3c\") " pod="openshift-ingress/e2e-praxis-a-8c08e7-openshift-default-5d56d76c65-qnh6x"
Oct 31 00:59:04.416240 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:04.412440    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gh8xd\" (UniqueName: \"kubernetes.io/projected/16131784-3cb6-4d01-a27f-6ce599437f3c-kube-api-access-gh8xd\") pod \"e2e-praxis-a-8c08e7-openshift-default-5d56d76c65-qnh6x\" (UID: \"16131784-3cb6-4d01-a27f-6ce599437f3c\") " pod="openshift-ingress/e2e-praxis-a-8c08e7-openshift-default-5d56d76c65-qnh6x"
Oct 31 00:59:04.423240 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:04.421727    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/16131784-3cb6-4d01-a27f-6ce599437f3c-istio-token\") pod \"e2e-praxis-a-8c08e7-openshift-default-5d56d76c65-qnh6x\" (UID: \"16131784-3cb6-4d01-a27f-6ce599437f3c\") " pod="openshift-ingress/e2e-praxis-a-8c08e7-openshift-default-5d56d76c65-qnh6x"
Oct 31 00:59:04.651959 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:04.651862    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-praxis-a-8c08e7-openshift-default-5d56d76c65-qnh6x"
Oct 31 00:59:04.728376 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:04.728341    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:59:05.073926 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:05.073889    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-praxis-a-8c08e7-openshift-default-5d56d76c65-qnh6x"]
Oct 31 00:59:05.075373 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:59:05.075346    4284 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod16131784_3cb6_4d01_a27f_6ce599437f3c.slice/crio-200c93936c05cb305667973fb5ea2a61061c49abffbd860bc4fe91959d41620e WatchSource:0}: Error finding container 200c93936c05cb305667973fb5ea2a61061c49abffbd860bc4fe91959d41620e: Status 404 returned error can't find the container with id 200c93936c05cb305667973fb5ea2a61061c49abffbd860bc4fe91959d41620e
Oct 31 00:59:05.078021 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:05.077989    4284 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 00:59:05.078124 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:05.078071    4284 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 00:59:05.078184 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:05.078120    4284 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 00:59:05.123111 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:05.123082    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-a-8c08e7-openshift-default-5d56d76c65-qnh6x" event={"ID":"16131784-3cb6-4d01-a27f-6ce599437f3c","Type":"ContainerStarted","Data":"200c93936c05cb305667973fb5ea2a61061c49abffbd860bc4fe91959d41620e"}
Oct 31 00:59:06.130485 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:06.130442    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-a-8c08e7-openshift-default-5d56d76c65-qnh6x" event={"ID":"16131784-3cb6-4d01-a27f-6ce599437f3c","Type":"ContainerStarted","Data":"bae8086e21fa68bb91f5084ddbb3811c359f00373ecd02892c41bad22b4625dc"}
Oct 31 00:59:06.299959 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:06.299905    4284 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-praxis-a-8c08e7-openshift-default-5d56d76c65-qnh6x" podStartSLOduration=3.299892587 podStartE2EDuration="3.299892587s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 00:59:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:59:06.299584555 +0000 UTC m=+1860.444591471" watchObservedRunningTime="2026-10-31 00:59:06.299892587 +0000 UTC m=+1860.444899502"
Oct 31 00:59:06.653537 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:06.653499    4284 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-praxis-a-8c08e7-openshift-default-5d56d76c65-qnh6x"
Oct 31 00:59:06.659031 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:06.659007    4284 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-praxis-a-8c08e7-openshift-default-5d56d76c65-qnh6x"
Oct 31 00:59:06.920009 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:06.919975    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:59:07.110920 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:07.110880    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:59:07.135966 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:07.135940    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-praxis-a-8c08e7-openshift-default-5d56d76c65-qnh6x"
Oct 31 00:59:07.136986 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:07.136969    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-praxis-a-8c08e7-openshift-default-5d56d76c65-qnh6x"
Oct 31 00:59:09.158622 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:09.158572    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:59:09.364583 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:09.364543    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:59:09.538734 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:09.538653    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:59:10.190275 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:10.190241    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:59:10.397721 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:10.397688    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:59:11.332558 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:11.332522    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:59:11.934721 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:11.934685    4284 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-b-eb41d3-openshift-default-5bdbd7758c-lqfzl"]
Oct 31 00:59:11.939772 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:11.939752    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-eb41d3-openshift-default-5bdbd7758c-lqfzl"
Oct 31 00:59:11.994374 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:11.994347    4284 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-b-eb41d3-openshift-default-dockercfg-2sxlz\""
Oct 31 00:59:12.045472 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:12.045437    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-eb41d3-openshift-default-5bdbd7758c-lqfzl"]
Oct 31 00:59:12.047970 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:12.047943    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:59:12.145238 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:12.143414    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2728e644-cebe-42fe-88ef-3d17a8a5817e-workload-certs\") pod \"e2e-ipp-b-eb41d3-openshift-default-5bdbd7758c-lqfzl\" (UID: \"2728e644-cebe-42fe-88ef-3d17a8a5817e\") " pod="openshift-ingress/e2e-ipp-b-eb41d3-openshift-default-5bdbd7758c-lqfzl"
Oct 31 00:59:12.145238 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:12.143510    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2728e644-cebe-42fe-88ef-3d17a8a5817e-istio-podinfo\") pod \"e2e-ipp-b-eb41d3-openshift-default-5bdbd7758c-lqfzl\" (UID: \"2728e644-cebe-42fe-88ef-3d17a8a5817e\") " pod="openshift-ingress/e2e-ipp-b-eb41d3-openshift-default-5bdbd7758c-lqfzl"
Oct 31 00:59:12.145238 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:12.143668    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2728e644-cebe-42fe-88ef-3d17a8a5817e-credential-socket\") pod \"e2e-ipp-b-eb41d3-openshift-default-5bdbd7758c-lqfzl\" (UID: \"2728e644-cebe-42fe-88ef-3d17a8a5817e\") " pod="openshift-ingress/e2e-ipp-b-eb41d3-openshift-default-5bdbd7758c-lqfzl"
Oct 31 00:59:12.145238 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:12.143747    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2728e644-cebe-42fe-88ef-3d17a8a5817e-istio-token\") pod \"e2e-ipp-b-eb41d3-openshift-default-5bdbd7758c-lqfzl\" (UID: \"2728e644-cebe-42fe-88ef-3d17a8a5817e\") " pod="openshift-ingress/e2e-ipp-b-eb41d3-openshift-default-5bdbd7758c-lqfzl"
Oct 31 00:59:12.145238 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:12.143818    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2728e644-cebe-42fe-88ef-3d17a8a5817e-workload-socket\") pod \"e2e-ipp-b-eb41d3-openshift-default-5bdbd7758c-lqfzl\" (UID: \"2728e644-cebe-42fe-88ef-3d17a8a5817e\") " pod="openshift-ingress/e2e-ipp-b-eb41d3-openshift-default-5bdbd7758c-lqfzl"
Oct 31 00:59:12.145238 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:12.143854    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2728e644-cebe-42fe-88ef-3d17a8a5817e-istio-data\") pod \"e2e-ipp-b-eb41d3-openshift-default-5bdbd7758c-lqfzl\" (UID: \"2728e644-cebe-42fe-88ef-3d17a8a5817e\") " pod="openshift-ingress/e2e-ipp-b-eb41d3-openshift-default-5bdbd7758c-lqfzl"
Oct 31 00:59:12.145238 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:12.143894    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2728e644-cebe-42fe-88ef-3d17a8a5817e-istiod-ca-cert\") pod \"e2e-ipp-b-eb41d3-openshift-default-5bdbd7758c-lqfzl\" (UID: \"2728e644-cebe-42fe-88ef-3d17a8a5817e\") " pod="openshift-ingress/e2e-ipp-b-eb41d3-openshift-default-5bdbd7758c-lqfzl"
Oct 31 00:59:12.145238 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:12.143926    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2728e644-cebe-42fe-88ef-3d17a8a5817e-istio-envoy\") pod \"e2e-ipp-b-eb41d3-openshift-default-5bdbd7758c-lqfzl\" (UID: \"2728e644-cebe-42fe-88ef-3d17a8a5817e\") " pod="openshift-ingress/e2e-ipp-b-eb41d3-openshift-default-5bdbd7758c-lqfzl"
Oct 31 00:59:12.145238 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:12.144059    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fpmfv\" (UniqueName: \"kubernetes.io/projected/2728e644-cebe-42fe-88ef-3d17a8a5817e-kube-api-access-fpmfv\") pod \"e2e-ipp-b-eb41d3-openshift-default-5bdbd7758c-lqfzl\" (UID: \"2728e644-cebe-42fe-88ef-3d17a8a5817e\") " pod="openshift-ingress/e2e-ipp-b-eb41d3-openshift-default-5bdbd7758c-lqfzl"
Oct 31 00:59:12.258285 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:12.255841    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2728e644-cebe-42fe-88ef-3d17a8a5817e-workload-certs\") pod \"e2e-ipp-b-eb41d3-openshift-default-5bdbd7758c-lqfzl\" (UID: \"2728e644-cebe-42fe-88ef-3d17a8a5817e\") " pod="openshift-ingress/e2e-ipp-b-eb41d3-openshift-default-5bdbd7758c-lqfzl"
Oct 31 00:59:12.258285 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:12.255902    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2728e644-cebe-42fe-88ef-3d17a8a5817e-istio-podinfo\") pod \"e2e-ipp-b-eb41d3-openshift-default-5bdbd7758c-lqfzl\" (UID: \"2728e644-cebe-42fe-88ef-3d17a8a5817e\") " pod="openshift-ingress/e2e-ipp-b-eb41d3-openshift-default-5bdbd7758c-lqfzl"
Oct 31 00:59:12.258285 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:12.255995    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2728e644-cebe-42fe-88ef-3d17a8a5817e-credential-socket\") pod \"e2e-ipp-b-eb41d3-openshift-default-5bdbd7758c-lqfzl\" (UID: \"2728e644-cebe-42fe-88ef-3d17a8a5817e\") " pod="openshift-ingress/e2e-ipp-b-eb41d3-openshift-default-5bdbd7758c-lqfzl"
Oct 31 00:59:12.258285 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:12.256062    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2728e644-cebe-42fe-88ef-3d17a8a5817e-istio-token\") pod \"e2e-ipp-b-eb41d3-openshift-default-5bdbd7758c-lqfzl\" (UID: \"2728e644-cebe-42fe-88ef-3d17a8a5817e\") " pod="openshift-ingress/e2e-ipp-b-eb41d3-openshift-default-5bdbd7758c-lqfzl"
Oct 31 00:59:12.258285 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:12.256115    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2728e644-cebe-42fe-88ef-3d17a8a5817e-workload-socket\") pod \"e2e-ipp-b-eb41d3-openshift-default-5bdbd7758c-lqfzl\" (UID: \"2728e644-cebe-42fe-88ef-3d17a8a5817e\") " pod="openshift-ingress/e2e-ipp-b-eb41d3-openshift-default-5bdbd7758c-lqfzl"
Oct 31 00:59:12.258285 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:12.256147    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2728e644-cebe-42fe-88ef-3d17a8a5817e-istio-data\") pod \"e2e-ipp-b-eb41d3-openshift-default-5bdbd7758c-lqfzl\" (UID: \"2728e644-cebe-42fe-88ef-3d17a8a5817e\") " pod="openshift-ingress/e2e-ipp-b-eb41d3-openshift-default-5bdbd7758c-lqfzl"
Oct 31 00:59:12.258285 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:12.256184    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2728e644-cebe-42fe-88ef-3d17a8a5817e-istiod-ca-cert\") pod \"e2e-ipp-b-eb41d3-openshift-default-5bdbd7758c-lqfzl\" (UID: \"2728e644-cebe-42fe-88ef-3d17a8a5817e\") " pod="openshift-ingress/e2e-ipp-b-eb41d3-openshift-default-5bdbd7758c-lqfzl"
Oct 31 00:59:12.258285 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:12.256263    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2728e644-cebe-42fe-88ef-3d17a8a5817e-istio-envoy\") pod \"e2e-ipp-b-eb41d3-openshift-default-5bdbd7758c-lqfzl\" (UID: \"2728e644-cebe-42fe-88ef-3d17a8a5817e\") " pod="openshift-ingress/e2e-ipp-b-eb41d3-openshift-default-5bdbd7758c-lqfzl"
Oct 31 00:59:12.258285 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:12.256350    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fpmfv\" (UniqueName: \"kubernetes.io/projected/2728e644-cebe-42fe-88ef-3d17a8a5817e-kube-api-access-fpmfv\") pod \"e2e-ipp-b-eb41d3-openshift-default-5bdbd7758c-lqfzl\" (UID: \"2728e644-cebe-42fe-88ef-3d17a8a5817e\") " pod="openshift-ingress/e2e-ipp-b-eb41d3-openshift-default-5bdbd7758c-lqfzl"
Oct 31 00:59:12.258285 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:12.257203    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2728e644-cebe-42fe-88ef-3d17a8a5817e-workload-certs\") pod \"e2e-ipp-b-eb41d3-openshift-default-5bdbd7758c-lqfzl\" (UID: \"2728e644-cebe-42fe-88ef-3d17a8a5817e\") " pod="openshift-ingress/e2e-ipp-b-eb41d3-openshift-default-5bdbd7758c-lqfzl"
Oct 31 00:59:12.258745 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:12.258348    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2728e644-cebe-42fe-88ef-3d17a8a5817e-istio-data\") pod \"e2e-ipp-b-eb41d3-openshift-default-5bdbd7758c-lqfzl\" (UID: \"2728e644-cebe-42fe-88ef-3d17a8a5817e\") " pod="openshift-ingress/e2e-ipp-b-eb41d3-openshift-default-5bdbd7758c-lqfzl"
Oct 31 00:59:12.258745 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:12.258605    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2728e644-cebe-42fe-88ef-3d17a8a5817e-credential-socket\") pod \"e2e-ipp-b-eb41d3-openshift-default-5bdbd7758c-lqfzl\" (UID: \"2728e644-cebe-42fe-88ef-3d17a8a5817e\") " pod="openshift-ingress/e2e-ipp-b-eb41d3-openshift-default-5bdbd7758c-lqfzl"
Oct 31 00:59:12.259014 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:12.258990    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2728e644-cebe-42fe-88ef-3d17a8a5817e-istiod-ca-cert\") pod \"e2e-ipp-b-eb41d3-openshift-default-5bdbd7758c-lqfzl\" (UID: \"2728e644-cebe-42fe-88ef-3d17a8a5817e\") " pod="openshift-ingress/e2e-ipp-b-eb41d3-openshift-default-5bdbd7758c-lqfzl"
Oct 31 00:59:12.266553 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:12.266524    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2728e644-cebe-42fe-88ef-3d17a8a5817e-workload-socket\") pod \"e2e-ipp-b-eb41d3-openshift-default-5bdbd7758c-lqfzl\" (UID: \"2728e644-cebe-42fe-88ef-3d17a8a5817e\") " pod="openshift-ingress/e2e-ipp-b-eb41d3-openshift-default-5bdbd7758c-lqfzl"
Oct 31 00:59:12.267324 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:12.267284    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2728e644-cebe-42fe-88ef-3d17a8a5817e-istio-podinfo\") pod \"e2e-ipp-b-eb41d3-openshift-default-5bdbd7758c-lqfzl\" (UID: \"2728e644-cebe-42fe-88ef-3d17a8a5817e\") " pod="openshift-ingress/e2e-ipp-b-eb41d3-openshift-default-5bdbd7758c-lqfzl"
Oct 31 00:59:12.268455 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:12.268429    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2728e644-cebe-42fe-88ef-3d17a8a5817e-istio-envoy\") pod \"e2e-ipp-b-eb41d3-openshift-default-5bdbd7758c-lqfzl\" (UID: \"2728e644-cebe-42fe-88ef-3d17a8a5817e\") " pod="openshift-ingress/e2e-ipp-b-eb41d3-openshift-default-5bdbd7758c-lqfzl"
Oct 31 00:59:12.316788 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:12.316754    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fpmfv\" (UniqueName: \"kubernetes.io/projected/2728e644-cebe-42fe-88ef-3d17a8a5817e-kube-api-access-fpmfv\") pod \"e2e-ipp-b-eb41d3-openshift-default-5bdbd7758c-lqfzl\" (UID: \"2728e644-cebe-42fe-88ef-3d17a8a5817e\") " pod="openshift-ingress/e2e-ipp-b-eb41d3-openshift-default-5bdbd7758c-lqfzl"
Oct 31 00:59:12.317106 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:12.317086    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2728e644-cebe-42fe-88ef-3d17a8a5817e-istio-token\") pod \"e2e-ipp-b-eb41d3-openshift-default-5bdbd7758c-lqfzl\" (UID: \"2728e644-cebe-42fe-88ef-3d17a8a5817e\") " pod="openshift-ingress/e2e-ipp-b-eb41d3-openshift-default-5bdbd7758c-lqfzl"
Oct 31 00:59:12.473378 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:12.473337    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:59:12.549191 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:12.549111    4284 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-ipp-a-ac952a/ipp-route-ac952a-kserve-75c45558d9-fxngk"
Oct 31 00:59:12.553277 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:12.553143    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-eb41d3-openshift-default-5bdbd7758c-lqfzl"
Oct 31 00:59:12.571881 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:12.571855    4284 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-ipp-a-ac952a/ipp-route-ac952a-kserve-75c45558d9-fxngk"
Oct 31 00:59:12.976983 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:12.976955    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-eb41d3-openshift-default-5bdbd7758c-lqfzl"]
Oct 31 00:59:12.977498 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:59:12.977476    4284 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2728e644_cebe_42fe_88ef_3d17a8a5817e.slice/crio-ae713e97ffcad839b15942904e04134213e5c7e9974e0a1ab4bd5f4abdda2f28 WatchSource:0}: Error finding container ae713e97ffcad839b15942904e04134213e5c7e9974e0a1ab4bd5f4abdda2f28: Status 404 returned error can't find the container with id ae713e97ffcad839b15942904e04134213e5c7e9974e0a1ab4bd5f4abdda2f28
Oct 31 00:59:12.980096 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:12.980064    4284 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 00:59:12.980192 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:12.980139    4284 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 00:59:12.980192 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:12.980182    4284 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 00:59:13.281898 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:13.281097    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-eb41d3-openshift-default-5bdbd7758c-lqfzl" event={"ID":"2728e644-cebe-42fe-88ef-3d17a8a5817e","Type":"ContainerStarted","Data":"7d2e3989067ab9ebc481325739a6a6843c952e478c9b181b3e522b9af1458faf"}
Oct 31 00:59:13.281898 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:13.281170    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-eb41d3-openshift-default-5bdbd7758c-lqfzl" event={"ID":"2728e644-cebe-42fe-88ef-3d17a8a5817e","Type":"ContainerStarted","Data":"ae713e97ffcad839b15942904e04134213e5c7e9974e0a1ab4bd5f4abdda2f28"}
Oct 31 00:59:13.299001 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:13.298975    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-ipp-a-ac952a/ipp-route-ac952a-kserve-75c45558d9-fxngk"
Oct 31 00:59:13.313265 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:13.313235    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:59:13.544887 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:13.544764    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:59:13.551836 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:13.551784    4284 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-b-eb41d3-openshift-default-5bdbd7758c-lqfzl" podStartSLOduration=2.551764075 podStartE2EDuration="2.551764075s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 00:59:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:59:13.432849203 +0000 UTC m=+1867.577856133" watchObservedRunningTime="2026-10-31 00:59:13.551764075 +0000 UTC m=+1867.696770995"
Oct 31 00:59:13.554450 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:13.554357    4284 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-eb41d3-openshift-default-5bdbd7758c-lqfzl"
Oct 31 00:59:14.028360 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:14.028326    4284 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-praxis-a-8c08e7-5668d6bd7c-btcgk"]
Oct 31 00:59:14.036269 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:14.035238    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-praxis-a-8c08e7-5668d6bd7c-btcgk"]
Oct 31 00:59:14.036269 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:14.035374    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-praxis-a-8c08e7-5668d6bd7c-btcgk"
Oct 31 00:59:14.087237 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:14.087182    4284 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-praxis-a-8c08e7-dockercfg-fp8ln\""
Oct 31 00:59:14.087429 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:14.087294    4284 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-praxis-a-8c08e7\""
Oct 31 00:59:14.143782 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:14.143748    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c724bcaf-457c-4b0b-8f2c-7ed25c4c8fb5-plugins-config-volume\") pod \"payload-processing-e2e-praxis-a-8c08e7-5668d6bd7c-btcgk\" (UID: \"c724bcaf-457c-4b0b-8f2c-7ed25c4c8fb5\") " pod="openshift-ingress/payload-processing-e2e-praxis-a-8c08e7-5668d6bd7c-btcgk"
Oct 31 00:59:14.143936 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:14.143885    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t7qzq\" (UniqueName: \"kubernetes.io/projected/c724bcaf-457c-4b0b-8f2c-7ed25c4c8fb5-kube-api-access-t7qzq\") pod \"payload-processing-e2e-praxis-a-8c08e7-5668d6bd7c-btcgk\" (UID: \"c724bcaf-457c-4b0b-8f2c-7ed25c4c8fb5\") " pod="openshift-ingress/payload-processing-e2e-praxis-a-8c08e7-5668d6bd7c-btcgk"
Oct 31 00:59:14.245501 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:14.245468    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c724bcaf-457c-4b0b-8f2c-7ed25c4c8fb5-plugins-config-volume\") pod \"payload-processing-e2e-praxis-a-8c08e7-5668d6bd7c-btcgk\" (UID: \"c724bcaf-457c-4b0b-8f2c-7ed25c4c8fb5\") " pod="openshift-ingress/payload-processing-e2e-praxis-a-8c08e7-5668d6bd7c-btcgk"
Oct 31 00:59:14.245677 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:14.245573    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t7qzq\" (UniqueName: \"kubernetes.io/projected/c724bcaf-457c-4b0b-8f2c-7ed25c4c8fb5-kube-api-access-t7qzq\") pod \"payload-processing-e2e-praxis-a-8c08e7-5668d6bd7c-btcgk\" (UID: \"c724bcaf-457c-4b0b-8f2c-7ed25c4c8fb5\") " pod="openshift-ingress/payload-processing-e2e-praxis-a-8c08e7-5668d6bd7c-btcgk"
Oct 31 00:59:14.246187 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:14.246169    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c724bcaf-457c-4b0b-8f2c-7ed25c4c8fb5-plugins-config-volume\") pod \"payload-processing-e2e-praxis-a-8c08e7-5668d6bd7c-btcgk\" (UID: \"c724bcaf-457c-4b0b-8f2c-7ed25c4c8fb5\") " pod="openshift-ingress/payload-processing-e2e-praxis-a-8c08e7-5668d6bd7c-btcgk"
Oct 31 00:59:14.312957 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:14.312887    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t7qzq\" (UniqueName: \"kubernetes.io/projected/c724bcaf-457c-4b0b-8f2c-7ed25c4c8fb5-kube-api-access-t7qzq\") pod \"payload-processing-e2e-praxis-a-8c08e7-5668d6bd7c-btcgk\" (UID: \"c724bcaf-457c-4b0b-8f2c-7ed25c4c8fb5\") " pod="openshift-ingress/payload-processing-e2e-praxis-a-8c08e7-5668d6bd7c-btcgk"
Oct 31 00:59:14.367236 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:14.359513    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-praxis-a-8c08e7-5668d6bd7c-btcgk"
Oct 31 00:59:14.561332 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:14.561302    4284 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-eb41d3-openshift-default-5bdbd7758c-lqfzl"
Oct 31 00:59:14.796803 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:14.796765    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-b647c3-5d89c86dd7-s999b"]
Oct 31 00:59:14.839028 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:14.839002    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-praxis-a-8c08e7-5668d6bd7c-btcgk"]
Oct 31 00:59:14.839531 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:59:14.839508    4284 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc724bcaf_457c_4b0b_8f2c_7ed25c4c8fb5.slice/crio-2ba3402c29094ebaeb0a8b23c505c27da47c47e472b9e1a101bfad09276e9f18 WatchSource:0}: Error finding container 2ba3402c29094ebaeb0a8b23c505c27da47c47e472b9e1a101bfad09276e9f18: Status 404 returned error can't find the container with id 2ba3402c29094ebaeb0a8b23c505c27da47c47e472b9e1a101bfad09276e9f18
Oct 31 00:59:14.986993 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:14.986955    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:59:15.210239 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:15.198494    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:59:15.294165 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:15.294130    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-praxis-a-8c08e7-5668d6bd7c-btcgk" event={"ID":"c724bcaf-457c-4b0b-8f2c-7ed25c4c8fb5","Type":"ContainerStarted","Data":"6c499c1f0b2818da050f41649ff3d918d0eb641a6ab009d06726dd643f2b3761"}
Oct 31 00:59:15.294379 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:15.294191    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-praxis-a-8c08e7-5668d6bd7c-btcgk" event={"ID":"c724bcaf-457c-4b0b-8f2c-7ed25c4c8fb5","Type":"ContainerStarted","Data":"2ba3402c29094ebaeb0a8b23c505c27da47c47e472b9e1a101bfad09276e9f18"}
Oct 31 00:59:15.294379 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:15.294253    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-praxis-a-8c08e7-5668d6bd7c-btcgk"
Oct 31 00:59:15.294516 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:15.294460    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-eb41d3-openshift-default-5bdbd7758c-lqfzl"
Oct 31 00:59:15.295386 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:15.295358    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-eb41d3-openshift-default-5bdbd7758c-lqfzl"
Oct 31 00:59:15.408002 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:15.407943    4284 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-praxis-a-8c08e7-5668d6bd7c-btcgk" podStartSLOduration=2.407924197 podStartE2EDuration="2.407924197s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 00:59:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:59:15.406381594 +0000 UTC m=+1869.551388512" watchObservedRunningTime="2026-10-31 00:59:15.407924197 +0000 UTC m=+1869.552931114"
Oct 31 00:59:16.887081 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:16.887042    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:59:17.017717 ip-10-0-132-173 kubenswrapper[4284]: E1031 00:59:17.017680    4284 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fd8sp" containerName="extractor"
Oct 31 00:59:17.106794 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:17.106758    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:59:17.532845 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:17.532809    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:59:19.797783 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:19.797738    4284 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-b647c3-5d89c86dd7-s999b" podUID="8e9971e4-4766-4644-bf14-341b643426d9" containerName="payload-pre-processing" containerID="cri-o://c8fbf267f6a7e73cd18abd0d74d6f8cda70734e2c386cb72e04e2f67d3fba0ff" gracePeriod=25
Oct 31 00:59:20.080552 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:20.080528    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-b647c3-5d89c86dd7-s999b"
Oct 31 00:59:20.145183 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:20.145147    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/8e9971e4-4766-4644-bf14-341b643426d9-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8e9971e4-4766-4644-bf14-341b643426d9-plugins-config-volume\") pod \"8e9971e4-4766-4644-bf14-341b643426d9\" (UID: \"8e9971e4-4766-4644-bf14-341b643426d9\") "
Oct 31 00:59:20.145364 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:20.145201    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8e9971e4-4766-4644-bf14-341b643426d9-kube-api-access-z24ns\" (UniqueName: \"kubernetes.io/projected/8e9971e4-4766-4644-bf14-341b643426d9-kube-api-access-z24ns\") pod \"8e9971e4-4766-4644-bf14-341b643426d9\" (UID: \"8e9971e4-4766-4644-bf14-341b643426d9\") "
Oct 31 00:59:20.145622 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:20.145587    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8e9971e4-4766-4644-bf14-341b643426d9-plugins-config-volume" pod "8e9971e4-4766-4644-bf14-341b643426d9" (UID: "8e9971e4-4766-4644-bf14-341b643426d9"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 00:59:20.145737 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:20.145723    4284 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8e9971e4-4766-4644-bf14-341b643426d9-plugins-config-volume\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:59:20.147494 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:20.147457    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8e9971e4-4766-4644-bf14-341b643426d9-kube-api-access-z24ns" pod "8e9971e4-4766-4644-bf14-341b643426d9" (UID: "8e9971e4-4766-4644-bf14-341b643426d9"). InnerVolumeSpecName "kube-api-access-z24ns". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 00:59:20.247090 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:20.247059    4284 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z24ns\" (UniqueName: \"kubernetes.io/projected/8e9971e4-4766-4644-bf14-341b643426d9-kube-api-access-z24ns\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:59:20.327276 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:20.327168    4284 generic.go:356] "Generic (PLEG): container finished" podID="8e9971e4-4766-4644-bf14-341b643426d9" containerID="c8fbf267f6a7e73cd18abd0d74d6f8cda70734e2c386cb72e04e2f67d3fba0ff" exitCode=0
Oct 31 00:59:20.327276 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:20.327246    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-b647c3-5d89c86dd7-s999b"
Oct 31 00:59:20.327526 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:20.327249    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-b647c3-5d89c86dd7-s999b" event={"ID":"8e9971e4-4766-4644-bf14-341b643426d9","Type":"ContainerDied","Data":"c8fbf267f6a7e73cd18abd0d74d6f8cda70734e2c386cb72e04e2f67d3fba0ff"}
Oct 31 00:59:20.327526 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:20.327396    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-b647c3-5d89c86dd7-s999b" event={"ID":"8e9971e4-4766-4644-bf14-341b643426d9","Type":"ContainerDied","Data":"028ac02c29584f1b391f236e96a5760362d32fed72db9fb94959abada1ec70be"}
Oct 31 00:59:20.327526 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:20.327428    4284 scope.go:122] "RemoveContainer" containerID="c8fbf267f6a7e73cd18abd0d74d6f8cda70734e2c386cb72e04e2f67d3fba0ff"
Oct 31 00:59:20.337301 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:20.337279    4284 scope.go:122] "RemoveContainer" containerID="c8fbf267f6a7e73cd18abd0d74d6f8cda70734e2c386cb72e04e2f67d3fba0ff"
Oct 31 00:59:20.337569 ip-10-0-132-173 kubenswrapper[4284]: E1031 00:59:20.337549    4284 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c8fbf267f6a7e73cd18abd0d74d6f8cda70734e2c386cb72e04e2f67d3fba0ff\": container with ID starting with c8fbf267f6a7e73cd18abd0d74d6f8cda70734e2c386cb72e04e2f67d3fba0ff not found: ID does not exist" containerID="c8fbf267f6a7e73cd18abd0d74d6f8cda70734e2c386cb72e04e2f67d3fba0ff"
Oct 31 00:59:20.337638 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:20.337577    4284 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c8fbf267f6a7e73cd18abd0d74d6f8cda70734e2c386cb72e04e2f67d3fba0ff"} err="failed to get container status \"c8fbf267f6a7e73cd18abd0d74d6f8cda70734e2c386cb72e04e2f67d3fba0ff\": rpc error: code = NotFound desc = could not find container \"c8fbf267f6a7e73cd18abd0d74d6f8cda70734e2c386cb72e04e2f67d3fba0ff\": container with ID starting with c8fbf267f6a7e73cd18abd0d74d6f8cda70734e2c386cb72e04e2f67d3fba0ff not found: ID does not exist"
Oct 31 00:59:20.486165 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:20.486130    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-b647c3-5d89c86dd7-s999b"]
Oct 31 00:59:20.489517 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:20.489491    4284 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-b647c3-5d89c86dd7-s999b"]
Oct 31 00:59:21.341617 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:21.341580    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:59:21.528676 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:21.528635    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:59:21.988665 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:21.988634    4284 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="8e9971e4-4766-4644-bf14-341b643426d9" path="/var/lib/kubelet/pods/8e9971e4-4766-4644-bf14-341b643426d9/volumes"
Oct 31 00:59:22.695039 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:22.695006    4284 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-eee7f1-d9bf6db74-ggvqp"]
Oct 31 00:59:22.695692 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:22.695675    4284 cpu_manager.go:414] "containerMap: removing container" podUID="8e9971e4-4766-4644-bf14-341b643426d9" containerName="payload-pre-processing"
Oct 31 00:59:22.695752 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:22.695694    4284 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8e9971e4-4766-4644-bf14-341b643426d9" containerName="payload-pre-processing"
Oct 31 00:59:22.695808 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:22.695799    4284 memory_manager.go:365] "RemoveStaleState removing state" podUID="8e9971e4-4766-4644-bf14-341b643426d9" containerName="payload-pre-processing"
Oct 31 00:59:22.700921 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:22.700280    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-eee7f1-d9bf6db74-ggvqp"
Oct 31 00:59:22.755553 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:22.755525    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-eee7f1-d9bf6db74-ggvqp"]
Oct 31 00:59:22.798631 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:22.798601    4284 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-eee7f1-serving-cert\""
Oct 31 00:59:22.834293 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:22.834256    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d5b7cbb4-de93-4a9a-abd3-b69636f595dd-openshift-service-ca\") pod \"maas-api-e2e-api-a-eee7f1-d9bf6db74-ggvqp\" (UID: \"d5b7cbb4-de93-4a9a-abd3-b69636f595dd\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-eee7f1-d9bf6db74-ggvqp"
Oct 31 00:59:22.834477 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:22.834367    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d5b7cbb4-de93-4a9a-abd3-b69636f595dd-metrics-certs\") pod \"maas-api-e2e-api-a-eee7f1-d9bf6db74-ggvqp\" (UID: \"d5b7cbb4-de93-4a9a-abd3-b69636f595dd\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-eee7f1-d9bf6db74-ggvqp"
Oct 31 00:59:22.834477 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:22.834424    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d5b7cbb4-de93-4a9a-abd3-b69636f595dd-maas-api-tls\") pod \"maas-api-e2e-api-a-eee7f1-d9bf6db74-ggvqp\" (UID: \"d5b7cbb4-de93-4a9a-abd3-b69636f595dd\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-eee7f1-d9bf6db74-ggvqp"
Oct 31 00:59:22.834477 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:22.834466    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2hssj\" (UniqueName: \"kubernetes.io/projected/d5b7cbb4-de93-4a9a-abd3-b69636f595dd-kube-api-access-2hssj\") pod \"maas-api-e2e-api-a-eee7f1-d9bf6db74-ggvqp\" (UID: \"d5b7cbb4-de93-4a9a-abd3-b69636f595dd\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-eee7f1-d9bf6db74-ggvqp"
Oct 31 00:59:22.900492 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:22.900458    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:59:22.960048 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:22.959963    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d5b7cbb4-de93-4a9a-abd3-b69636f595dd-openshift-service-ca\") pod \"maas-api-e2e-api-a-eee7f1-d9bf6db74-ggvqp\" (UID: \"d5b7cbb4-de93-4a9a-abd3-b69636f595dd\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-eee7f1-d9bf6db74-ggvqp"
Oct 31 00:59:22.967238 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:22.964414    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d5b7cbb4-de93-4a9a-abd3-b69636f595dd-metrics-certs\") pod \"maas-api-e2e-api-a-eee7f1-d9bf6db74-ggvqp\" (UID: \"d5b7cbb4-de93-4a9a-abd3-b69636f595dd\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-eee7f1-d9bf6db74-ggvqp"
Oct 31 00:59:22.967238 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:22.964495    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d5b7cbb4-de93-4a9a-abd3-b69636f595dd-maas-api-tls\") pod \"maas-api-e2e-api-a-eee7f1-d9bf6db74-ggvqp\" (UID: \"d5b7cbb4-de93-4a9a-abd3-b69636f595dd\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-eee7f1-d9bf6db74-ggvqp"
Oct 31 00:59:22.967238 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:22.964544    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2hssj\" (UniqueName: \"kubernetes.io/projected/d5b7cbb4-de93-4a9a-abd3-b69636f595dd-kube-api-access-2hssj\") pod \"maas-api-e2e-api-a-eee7f1-d9bf6db74-ggvqp\" (UID: \"d5b7cbb4-de93-4a9a-abd3-b69636f595dd\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-eee7f1-d9bf6db74-ggvqp"
Oct 31 00:59:22.967238 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:22.961296    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d5b7cbb4-de93-4a9a-abd3-b69636f595dd-openshift-service-ca\") pod \"maas-api-e2e-api-a-eee7f1-d9bf6db74-ggvqp\" (UID: \"d5b7cbb4-de93-4a9a-abd3-b69636f595dd\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-eee7f1-d9bf6db74-ggvqp"
Oct 31 00:59:22.967238 ip-10-0-132-173 kubenswrapper[4284]: E1031 00:59:22.965631    4284 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-a-eee7f1-serving-cert: secret "maas-api-e2e-api-a-eee7f1-serving-cert" not found
Oct 31 00:59:22.967238 ip-10-0-132-173 kubenswrapper[4284]: E1031 00:59:22.965711    4284 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d5b7cbb4-de93-4a9a-abd3-b69636f595dd-maas-api-tls podName:d5b7cbb4-de93-4a9a-abd3-b69636f595dd nodeName:}" failed. No retries permitted until 2026-10-31 00:59:23.465690864 +0000 UTC m=+1877.610697758 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/d5b7cbb4-de93-4a9a-abd3-b69636f595dd-maas-api-tls") pod "maas-api-e2e-api-a-eee7f1-d9bf6db74-ggvqp" (UID: "d5b7cbb4-de93-4a9a-abd3-b69636f595dd") : secret "maas-api-e2e-api-a-eee7f1-serving-cert" not found
Oct 31 00:59:22.989514 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:22.989478    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d5b7cbb4-de93-4a9a-abd3-b69636f595dd-metrics-certs\") pod \"maas-api-e2e-api-a-eee7f1-d9bf6db74-ggvqp\" (UID: \"d5b7cbb4-de93-4a9a-abd3-b69636f595dd\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-eee7f1-d9bf6db74-ggvqp"
Oct 31 00:59:23.024623 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:23.024584    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2hssj\" (UniqueName: \"kubernetes.io/projected/d5b7cbb4-de93-4a9a-abd3-b69636f595dd-kube-api-access-2hssj\") pod \"maas-api-e2e-api-a-eee7f1-d9bf6db74-ggvqp\" (UID: \"d5b7cbb4-de93-4a9a-abd3-b69636f595dd\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-eee7f1-d9bf6db74-ggvqp"
Oct 31 00:59:23.050828 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:23.050795    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:59:23.251071 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:23.250987    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:59:23.472171 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:23.472137    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d5b7cbb4-de93-4a9a-abd3-b69636f595dd-maas-api-tls\") pod \"maas-api-e2e-api-a-eee7f1-d9bf6db74-ggvqp\" (UID: \"d5b7cbb4-de93-4a9a-abd3-b69636f595dd\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-eee7f1-d9bf6db74-ggvqp"
Oct 31 00:59:23.474687 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:23.474660    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d5b7cbb4-de93-4a9a-abd3-b69636f595dd-maas-api-tls\") pod \"maas-api-e2e-api-a-eee7f1-d9bf6db74-ggvqp\" (UID: \"d5b7cbb4-de93-4a9a-abd3-b69636f595dd\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-eee7f1-d9bf6db74-ggvqp"
Oct 31 00:59:23.617510 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:23.617421    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-eee7f1-d9bf6db74-ggvqp"
Oct 31 00:59:23.988662 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:23.988637    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-eee7f1-d9bf6db74-ggvqp"]
Oct 31 00:59:23.989509 ip-10-0-132-173 kubenswrapper[4284]: W1031 00:59:23.989485    4284 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd5b7cbb4_de93_4a9a_abd3_b69636f595dd.slice/crio-d431b8e5d2aa6c3abfd8181b7d98240bf4f993a48df600b0a3df0d8576c141bb WatchSource:0}: Error finding container d431b8e5d2aa6c3abfd8181b7d98240bf4f993a48df600b0a3df0d8576c141bb: Status 404 returned error can't find the container with id d431b8e5d2aa6c3abfd8181b7d98240bf4f993a48df600b0a3df0d8576c141bb
Oct 31 00:59:24.354073 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:24.353992    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-eee7f1-d9bf6db74-ggvqp" event={"ID":"d5b7cbb4-de93-4a9a-abd3-b69636f595dd","Type":"ContainerStarted","Data":"d431b8e5d2aa6c3abfd8181b7d98240bf4f993a48df600b0a3df0d8576c141bb"}
Oct 31 00:59:25.362186 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:25.362101    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-eee7f1-d9bf6db74-ggvqp" event={"ID":"d5b7cbb4-de93-4a9a-abd3-b69636f595dd","Type":"ContainerStarted","Data":"1ee37fad3f92b4642d7b48b8f35f45711b99249469ab146987ada980b9b06ab0"}
Oct 31 00:59:25.362652 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:25.362286    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-eee7f1-d9bf6db74-ggvqp"
Oct 31 00:59:25.478633 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:25.478579    4284 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-eee7f1-d9bf6db74-ggvqp" podStartSLOduration=2.503143552 podStartE2EDuration="3.478564004s" totalImagesPullingTime="975.420452ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 00:59:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:59:25.476161433 +0000 UTC m=+1879.621168350" watchObservedRunningTime="2026-10-31 00:59:25.478564004 +0000 UTC m=+1879.623570920"
Oct 31 00:59:26.303655 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:26.303627    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-praxis-a-8c08e7-5668d6bd7c-btcgk"
Oct 31 00:59:28.340562 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:28.340528    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:59:31.374150 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:31.374124    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-eee7f1-d9bf6db74-ggvqp"
Oct 31 00:59:31.821294 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:31.821263    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:59:32.303404 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:32.303351    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:59:35.004685 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:35.004650    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:59:35.661015 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:35.660981    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:59:39.371021 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:39.370981    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:59:39.513694 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:39.513653    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:59:39.736146 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:39.736110    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:59:44.613704 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:44.613660    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:59:47.038500 ip-10-0-132-173 kubenswrapper[4284]: E1031 00:59:47.038465    4284 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fd8sp" containerName="extractor"
Oct 31 00:59:48.738963 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:48.738926    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-f66180-5c5d986cbf-7hvkv"]
Oct 31 00:59:48.739468 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:48.739203    4284 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-f66180-5c5d986cbf-7hvkv" podUID="d9d70c1c-9fbd-48cc-b100-24fe6c5836d8" containerName="maas-api" containerID="cri-o://eaf3f53d12259dc8b6bf69802db0f4fb268754d0af80ebfc94131eb51c101afc" gracePeriod=30
Oct 31 00:59:48.863460 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:48.863424    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:59:49.087835 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:49.087799    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-f66180-5c5d986cbf-7hvkv"
Oct 31 00:59:49.185779 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:49.185744    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/d9d70c1c-9fbd-48cc-b100-24fe6c5836d8-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d9d70c1c-9fbd-48cc-b100-24fe6c5836d8-maas-api-tls\") pod \"d9d70c1c-9fbd-48cc-b100-24fe6c5836d8\" (UID: \"d9d70c1c-9fbd-48cc-b100-24fe6c5836d8\") "
Oct 31 00:59:49.185990 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:49.185807    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d9d70c1c-9fbd-48cc-b100-24fe6c5836d8-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d9d70c1c-9fbd-48cc-b100-24fe6c5836d8-openshift-service-ca\") pod \"d9d70c1c-9fbd-48cc-b100-24fe6c5836d8\" (UID: \"d9d70c1c-9fbd-48cc-b100-24fe6c5836d8\") "
Oct 31 00:59:49.185990 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:49.185882    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d9d70c1c-9fbd-48cc-b100-24fe6c5836d8-kube-api-access-rgd29\" (UniqueName: \"kubernetes.io/projected/d9d70c1c-9fbd-48cc-b100-24fe6c5836d8-kube-api-access-rgd29\") pod \"d9d70c1c-9fbd-48cc-b100-24fe6c5836d8\" (UID: \"d9d70c1c-9fbd-48cc-b100-24fe6c5836d8\") "
Oct 31 00:59:49.185990 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:49.185950    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/d9d70c1c-9fbd-48cc-b100-24fe6c5836d8-metrics-certs\" (UniqueName: \"kubernetes.io/secret/d9d70c1c-9fbd-48cc-b100-24fe6c5836d8-metrics-certs\") pod \"d9d70c1c-9fbd-48cc-b100-24fe6c5836d8\" (UID: \"d9d70c1c-9fbd-48cc-b100-24fe6c5836d8\") "
Oct 31 00:59:49.186538 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:49.186508    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d9d70c1c-9fbd-48cc-b100-24fe6c5836d8-openshift-service-ca" pod "d9d70c1c-9fbd-48cc-b100-24fe6c5836d8" (UID: "d9d70c1c-9fbd-48cc-b100-24fe6c5836d8"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 00:59:49.188662 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:49.188622    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d9d70c1c-9fbd-48cc-b100-24fe6c5836d8-maas-api-tls" pod "d9d70c1c-9fbd-48cc-b100-24fe6c5836d8" (UID: "d9d70c1c-9fbd-48cc-b100-24fe6c5836d8"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 00:59:49.189061 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:49.189030    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d9d70c1c-9fbd-48cc-b100-24fe6c5836d8-kube-api-access-rgd29" pod "d9d70c1c-9fbd-48cc-b100-24fe6c5836d8" (UID: "d9d70c1c-9fbd-48cc-b100-24fe6c5836d8"). InnerVolumeSpecName "kube-api-access-rgd29". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 00:59:49.189362 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:49.189320    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d9d70c1c-9fbd-48cc-b100-24fe6c5836d8-metrics-certs" pod "d9d70c1c-9fbd-48cc-b100-24fe6c5836d8" (UID: "d9d70c1c-9fbd-48cc-b100-24fe6c5836d8"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 00:59:49.251584 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:49.251546    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-f66180-84487bc787-gjmlg"]
Oct 31 00:59:49.251776 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:49.251743    4284 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-f66180-84487bc787-gjmlg" podUID="f2cb10e0-cf06-4c51-b383-498ea90f2052" containerName="payload-pre-processing" containerID="cri-o://f4c54fe6db42294aecf7c2cd73993e69c0b3e9bf8e197057b67ee25f8cfd2073" gracePeriod=30
Oct 31 00:59:49.288364 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:49.288186    4284 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rgd29\" (UniqueName: \"kubernetes.io/projected/d9d70c1c-9fbd-48cc-b100-24fe6c5836d8-kube-api-access-rgd29\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:59:49.288364 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:49.288246    4284 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d9d70c1c-9fbd-48cc-b100-24fe6c5836d8-metrics-certs\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:59:49.288364 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:49.288262    4284 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d9d70c1c-9fbd-48cc-b100-24fe6c5836d8-maas-api-tls\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:59:49.288364 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:49.288276    4284 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d9d70c1c-9fbd-48cc-b100-24fe6c5836d8-openshift-service-ca\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:59:49.347792 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:49.347671    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:59:49.493900 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:49.493864    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:59:49.518356 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:49.518333    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-f66180-84487bc787-gjmlg"
Oct 31 00:59:49.554291 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:49.554264    4284 generic.go:356] "Generic (PLEG): container finished" podID="f2cb10e0-cf06-4c51-b383-498ea90f2052" containerID="f4c54fe6db42294aecf7c2cd73993e69c0b3e9bf8e197057b67ee25f8cfd2073" exitCode=0
Oct 31 00:59:49.554458 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:49.554323    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-f66180-84487bc787-gjmlg"
Oct 31 00:59:49.554458 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:49.554352    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-f66180-84487bc787-gjmlg" event={"ID":"f2cb10e0-cf06-4c51-b383-498ea90f2052","Type":"ContainerDied","Data":"f4c54fe6db42294aecf7c2cd73993e69c0b3e9bf8e197057b67ee25f8cfd2073"}
Oct 31 00:59:49.554458 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:49.554408    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-f66180-84487bc787-gjmlg" event={"ID":"f2cb10e0-cf06-4c51-b383-498ea90f2052","Type":"ContainerDied","Data":"2592808cd02827fa7b0636d41e50dc255b1311b2f2e287e24962a3f685264b75"}
Oct 31 00:59:49.554458 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:49.554438    4284 scope.go:122] "RemoveContainer" containerID="f4c54fe6db42294aecf7c2cd73993e69c0b3e9bf8e197057b67ee25f8cfd2073"
Oct 31 00:59:49.555976 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:49.555949    4284 generic.go:356] "Generic (PLEG): container finished" podID="d9d70c1c-9fbd-48cc-b100-24fe6c5836d8" containerID="eaf3f53d12259dc8b6bf69802db0f4fb268754d0af80ebfc94131eb51c101afc" exitCode=0
Oct 31 00:59:49.556085 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:49.555987    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-f66180-5c5d986cbf-7hvkv"
Oct 31 00:59:49.556085 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:49.556034    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-f66180-5c5d986cbf-7hvkv" event={"ID":"d9d70c1c-9fbd-48cc-b100-24fe6c5836d8","Type":"ContainerDied","Data":"eaf3f53d12259dc8b6bf69802db0f4fb268754d0af80ebfc94131eb51c101afc"}
Oct 31 00:59:49.556184 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:49.556094    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-f66180-5c5d986cbf-7hvkv" event={"ID":"d9d70c1c-9fbd-48cc-b100-24fe6c5836d8","Type":"ContainerDied","Data":"7d141c3ff2d4ea3725d935322fe39eebb1a9bac95701688826b8900e9d4007ea"}
Oct 31 00:59:49.565495 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:49.565475    4284 scope.go:122] "RemoveContainer" containerID="f4c54fe6db42294aecf7c2cd73993e69c0b3e9bf8e197057b67ee25f8cfd2073"
Oct 31 00:59:49.565781 ip-10-0-132-173 kubenswrapper[4284]: E1031 00:59:49.565755    4284 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f4c54fe6db42294aecf7c2cd73993e69c0b3e9bf8e197057b67ee25f8cfd2073\": container with ID starting with f4c54fe6db42294aecf7c2cd73993e69c0b3e9bf8e197057b67ee25f8cfd2073 not found: ID does not exist" containerID="f4c54fe6db42294aecf7c2cd73993e69c0b3e9bf8e197057b67ee25f8cfd2073"
Oct 31 00:59:49.565862 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:49.565793    4284 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f4c54fe6db42294aecf7c2cd73993e69c0b3e9bf8e197057b67ee25f8cfd2073"} err="failed to get container status \"f4c54fe6db42294aecf7c2cd73993e69c0b3e9bf8e197057b67ee25f8cfd2073\": rpc error: code = NotFound desc = could not find container \"f4c54fe6db42294aecf7c2cd73993e69c0b3e9bf8e197057b67ee25f8cfd2073\": container with ID starting with f4c54fe6db42294aecf7c2cd73993e69c0b3e9bf8e197057b67ee25f8cfd2073 not found: ID does not exist"
Oct 31 00:59:49.565862 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:49.565814    4284 scope.go:122] "RemoveContainer" containerID="eaf3f53d12259dc8b6bf69802db0f4fb268754d0af80ebfc94131eb51c101afc"
Oct 31 00:59:49.574567 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:49.574551    4284 scope.go:122] "RemoveContainer" containerID="eaf3f53d12259dc8b6bf69802db0f4fb268754d0af80ebfc94131eb51c101afc"
Oct 31 00:59:49.574812 ip-10-0-132-173 kubenswrapper[4284]: E1031 00:59:49.574795    4284 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eaf3f53d12259dc8b6bf69802db0f4fb268754d0af80ebfc94131eb51c101afc\": container with ID starting with eaf3f53d12259dc8b6bf69802db0f4fb268754d0af80ebfc94131eb51c101afc not found: ID does not exist" containerID="eaf3f53d12259dc8b6bf69802db0f4fb268754d0af80ebfc94131eb51c101afc"
Oct 31 00:59:49.574868 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:49.574817    4284 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eaf3f53d12259dc8b6bf69802db0f4fb268754d0af80ebfc94131eb51c101afc"} err="failed to get container status \"eaf3f53d12259dc8b6bf69802db0f4fb268754d0af80ebfc94131eb51c101afc\": rpc error: code = NotFound desc = could not find container \"eaf3f53d12259dc8b6bf69802db0f4fb268754d0af80ebfc94131eb51c101afc\": container with ID starting with eaf3f53d12259dc8b6bf69802db0f4fb268754d0af80ebfc94131eb51c101afc not found: ID does not exist"
Oct 31 00:59:49.592470 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:49.592447    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f2cb10e0-cf06-4c51-b383-498ea90f2052-kube-api-access-lkcjf\" (UniqueName: \"kubernetes.io/projected/f2cb10e0-cf06-4c51-b383-498ea90f2052-kube-api-access-lkcjf\") pod \"f2cb10e0-cf06-4c51-b383-498ea90f2052\" (UID: \"f2cb10e0-cf06-4c51-b383-498ea90f2052\") "
Oct 31 00:59:49.592566 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:49.592513    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f2cb10e0-cf06-4c51-b383-498ea90f2052-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f2cb10e0-cf06-4c51-b383-498ea90f2052-plugins-config-volume\") pod \"f2cb10e0-cf06-4c51-b383-498ea90f2052\" (UID: \"f2cb10e0-cf06-4c51-b383-498ea90f2052\") "
Oct 31 00:59:49.592910 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:49.592886    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f2cb10e0-cf06-4c51-b383-498ea90f2052-plugins-config-volume" pod "f2cb10e0-cf06-4c51-b383-498ea90f2052" (UID: "f2cb10e0-cf06-4c51-b383-498ea90f2052"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 00:59:49.593032 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:49.593014    4284 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f2cb10e0-cf06-4c51-b383-498ea90f2052-plugins-config-volume\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:59:49.594674 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:49.594651    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f2cb10e0-cf06-4c51-b383-498ea90f2052-kube-api-access-lkcjf" pod "f2cb10e0-cf06-4c51-b383-498ea90f2052" (UID: "f2cb10e0-cf06-4c51-b383-498ea90f2052"). InnerVolumeSpecName "kube-api-access-lkcjf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 00:59:49.732239 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:49.710849    4284 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lkcjf\" (UniqueName: \"kubernetes.io/projected/f2cb10e0-cf06-4c51-b383-498ea90f2052-kube-api-access-lkcjf\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 00:59:49.827686 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:49.827650    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-f66180-5c5d986cbf-7hvkv"]
Oct 31 00:59:49.833318 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:49.832232    4284 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-f66180-5c5d986cbf-7hvkv"]
Oct 31 00:59:49.965602 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:49.965564    4284 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d9d70c1c-9fbd-48cc-b100-24fe6c5836d8" path="/var/lib/kubelet/pods/d9d70c1c-9fbd-48cc-b100-24fe6c5836d8/volumes"
Oct 31 00:59:49.974132 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:49.974102    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-f66180-84487bc787-gjmlg"]
Oct 31 00:59:49.977309 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:49.977287    4284 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-f66180-84487bc787-gjmlg"]
Oct 31 00:59:50.363015 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:50.362978    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:59:50.580176 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:50.580143    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:59:50.736012 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:50.735980    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:59:51.316314 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:51.316283    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:59:51.573465 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:51.573365    4284 generic.go:356] "Generic (PLEG): container finished" podID="4fe7b9bb-dcce-4bb5-b37a-82fe1639b77e" containerID="dbae75592cdb87487f1720b192cbb998410a7c09334712b8ca46c5e262be5ebe" exitCode=0
Oct 31 00:59:51.573465 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:51.573426    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-eee7f1-openshift-default-78d7b57c9-cn4g6" event={"ID":"4fe7b9bb-dcce-4bb5-b37a-82fe1639b77e","Type":"ContainerDied","Data":"dbae75592cdb87487f1720b192cbb998410a7c09334712b8ca46c5e262be5ebe"}
Oct 31 00:59:51.573953 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:51.573934    4284 scope.go:122] "RemoveContainer" containerID="dbae75592cdb87487f1720b192cbb998410a7c09334712b8ca46c5e262be5ebe"
Oct 31 00:59:51.575771 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:51.575744    4284 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 00:59:51.575873 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:51.575823    4284 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 00:59:51.575873 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:51.575865    4284 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 00:59:51.965486 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:51.965456    4284 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f2cb10e0-cf06-4c51-b383-498ea90f2052" path="/var/lib/kubelet/pods/f2cb10e0-cf06-4c51-b383-498ea90f2052/volumes"
Oct 31 00:59:52.582097 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:52.582062    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-eee7f1-openshift-default-78d7b57c9-cn4g6" event={"ID":"4fe7b9bb-dcce-4bb5-b37a-82fe1639b77e","Type":"ContainerStarted","Data":"0c5d8907e60cf1cf12a88d576187cbcaf5d85130c988488329a87151c5a76960"}
Oct 31 00:59:53.247110 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:53.247070    4284 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-eee7f1-openshift-default-78d7b57c9-cn4g6"
Oct 31 00:59:53.247332 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:53.247310    4284 patch_prober.go:29] interesting pod/e2e-api-a-eee7f1-openshift-default-78d7b57c9-cn4g6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.116:15021/healthz/ready\": dial tcp 10.134.0.116:15021: connect: connection refused" start-of-body=
Oct 31 00:59:53.247394 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:53.247366    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-eee7f1-openshift-default-78d7b57c9-cn4g6" podUID="4fe7b9bb-dcce-4bb5-b37a-82fe1639b77e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.116:15021/healthz/ready\": dial tcp 10.134.0.116:15021: connect: connection refused"
Oct 31 00:59:54.247151 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:54.247114    4284 patch_prober.go:29] interesting pod/e2e-api-a-eee7f1-openshift-default-78d7b57c9-cn4g6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.116:15021/healthz/ready\": dial tcp 10.134.0.116:15021: connect: connection refused" start-of-body=
Oct 31 00:59:54.247623 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:54.247163    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-eee7f1-openshift-default-78d7b57c9-cn4g6" podUID="4fe7b9bb-dcce-4bb5-b37a-82fe1639b77e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.116:15021/healthz/ready\": dial tcp 10.134.0.116:15021: connect: connection refused"
Oct 31 00:59:55.247398 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:55.247362    4284 patch_prober.go:29] interesting pod/e2e-api-a-eee7f1-openshift-default-78d7b57c9-cn4g6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.116:15021/healthz/ready\": dial tcp 10.134.0.116:15021: connect: connection refused" start-of-body=
Oct 31 00:59:55.247818 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:55.247427    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-eee7f1-openshift-default-78d7b57c9-cn4g6" podUID="4fe7b9bb-dcce-4bb5-b37a-82fe1639b77e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.116:15021/healthz/ready\": dial tcp 10.134.0.116:15021: connect: connection refused"
Oct 31 00:59:55.993247 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:55.983966    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:59:56.247827 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:56.247742    4284 patch_prober.go:29] interesting pod/e2e-api-a-eee7f1-openshift-default-78d7b57c9-cn4g6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.116:15021/healthz/ready\": dial tcp 10.134.0.116:15021: connect: connection refused" start-of-body=
Oct 31 00:59:56.247827 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:56.247798    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-eee7f1-openshift-default-78d7b57c9-cn4g6" podUID="4fe7b9bb-dcce-4bb5-b37a-82fe1639b77e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.116:15021/healthz/ready\": dial tcp 10.134.0.116:15021: connect: connection refused"
Oct 31 00:59:56.512747 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:56.512662    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:59:56.710783 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:56.710736    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:59:57.248152 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:57.248111    4284 patch_prober.go:29] interesting pod/e2e-api-a-eee7f1-openshift-default-78d7b57c9-cn4g6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.116:15021/healthz/ready\": dial tcp 10.134.0.116:15021: connect: connection refused" start-of-body=
Oct 31 00:59:57.248637 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:57.248187    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-eee7f1-openshift-default-78d7b57c9-cn4g6" podUID="4fe7b9bb-dcce-4bb5-b37a-82fe1639b77e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.116:15021/healthz/ready\": dial tcp 10.134.0.116:15021: connect: connection refused"
Oct 31 00:59:58.203074 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:58.203041    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:59:58.252034 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:58.252003    4284 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-eee7f1-openshift-default-78d7b57c9-cn4g6"
Oct 31 00:59:58.252896 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:58.252875    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-eee7f1-openshift-default-78d7b57c9-cn4g6"
Oct 31 00:59:58.253881 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:58.253855    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-eee7f1-openshift-default-78d7b57c9-cn4g6"
Oct 31 00:59:58.400914 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:58.400879    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:59:58.530611 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:58.530498    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-f66180-openshift-default-55cfc8577c-lfq7v"]
Oct 31 00:59:58.531468 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:58.531433    4284 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-a-f66180-openshift-default-55cfc8577c-lfq7v" podUID="1e029b42-c66c-4ef0-b6e4-0a5ef5670758" containerName="istio-proxy" containerID="cri-o://cd8e6d6347680c0a4421384a11472c5da9bcacbaafb9153b701402aaa2e28f6e" gracePeriod=30
Oct 31 00:59:58.569977 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:58.569923    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:59:59.485936 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:59.485894    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 00:59:59.635890 ip-10-0-132-173 kubenswrapper[4284]: I1031 00:59:59.635858    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:00:00.031547 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:00.031502    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:00:00.084165 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:00.084098    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-29ece5-openshift-default-6fcb86dbc-hl96f"]
Oct 31 01:00:00.084410 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:00.084345    4284 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w2-29ece5-openshift-default-6fcb86dbc-hl96f" podUID="19acc006-2388-488d-bfff-6ce6bab123a8" containerName="istio-proxy" containerID="cri-o://cc637542466f434173e4554b90b9e78e20e03f3c87b39f900985c90e5e524c3a" gracePeriod=30
Oct 31 01:00:00.205719 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:00.205689    4284 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-b63c24-29890140-jxtzl"]
Oct 31 01:00:00.206258 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:00.206246    4284 cpu_manager.go:414] "containerMap: removing container" podUID="d9d70c1c-9fbd-48cc-b100-24fe6c5836d8" containerName="maas-api"
Oct 31 01:00:00.206312 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:00.206259    4284 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d9d70c1c-9fbd-48cc-b100-24fe6c5836d8" containerName="maas-api"
Oct 31 01:00:00.206312 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:00.206279    4284 cpu_manager.go:414] "containerMap: removing container" podUID="f2cb10e0-cf06-4c51-b383-498ea90f2052" containerName="payload-pre-processing"
Oct 31 01:00:00.206312 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:00.206284    4284 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f2cb10e0-cf06-4c51-b383-498ea90f2052" containerName="payload-pre-processing"
Oct 31 01:00:00.206408 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:00.206374    4284 memory_manager.go:365] "RemoveStaleState removing state" podUID="d9d70c1c-9fbd-48cc-b100-24fe6c5836d8" containerName="maas-api"
Oct 31 01:00:00.206408 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:00.206385    4284 memory_manager.go:365] "RemoveStaleState removing state" podUID="f2cb10e0-cf06-4c51-b383-498ea90f2052" containerName="payload-pre-processing"
Oct 31 01:00:00.209808 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:00.209786    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-b63c24-29890140-jxtzl"
Oct 31 01:00:00.234017 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:00.233978    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6af0f6fa-a965-4083-af26-8338eeb6e796-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w1-b63c24-29890140-jxtzl\" (UID: \"6af0f6fa-a965-4083-af26-8338eeb6e796\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-b63c24-29890140-jxtzl"
Oct 31 01:00:00.236646 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:00.236610    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4r5kq\" (UniqueName: \"kubernetes.io/projected/6af0f6fa-a965-4083-af26-8338eeb6e796-kube-api-access-4r5kq\") pod \"maas-api-key-cleanup-e2e-worker-w1-b63c24-29890140-jxtzl\" (UID: \"6af0f6fa-a965-4083-af26-8338eeb6e796\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-b63c24-29890140-jxtzl"
Oct 31 01:00:00.267112 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:00.267076    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-b63c24-29890140-jxtzl"]
Oct 31 01:00:00.352285 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:00.341244    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6af0f6fa-a965-4083-af26-8338eeb6e796-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w1-b63c24-29890140-jxtzl\" (UID: \"6af0f6fa-a965-4083-af26-8338eeb6e796\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-b63c24-29890140-jxtzl"
Oct 31 01:00:00.352285 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:00.341432    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4r5kq\" (UniqueName: \"kubernetes.io/projected/6af0f6fa-a965-4083-af26-8338eeb6e796-kube-api-access-4r5kq\") pod \"maas-api-key-cleanup-e2e-worker-w1-b63c24-29890140-jxtzl\" (UID: \"6af0f6fa-a965-4083-af26-8338eeb6e796\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-b63c24-29890140-jxtzl"
Oct 31 01:00:00.352285 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:00.345697    4284 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-8pzqp\""
Oct 31 01:00:00.406651 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:00.405401    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6af0f6fa-a965-4083-af26-8338eeb6e796-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w1-b63c24-29890140-jxtzl\" (UID: \"6af0f6fa-a965-4083-af26-8338eeb6e796\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-b63c24-29890140-jxtzl"
Oct 31 01:00:00.432042 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:00.432009    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4r5kq\" (UniqueName: \"kubernetes.io/projected/6af0f6fa-a965-4083-af26-8338eeb6e796-kube-api-access-4r5kq\") pod \"maas-api-key-cleanup-e2e-worker-w1-b63c24-29890140-jxtzl\" (UID: \"6af0f6fa-a965-4083-af26-8338eeb6e796\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-b63c24-29890140-jxtzl"
Oct 31 01:00:00.532236 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:00.526448    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-b63c24-29890140-jxtzl"
Oct 31 01:00:00.967164 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:00.967135    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-b63c24-29890140-jxtzl"]
Oct 31 01:00:00.967788 ip-10-0-132-173 kubenswrapper[4284]: W1031 01:00:00.967758    4284 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6af0f6fa_a965_4083_af26_8338eeb6e796.slice/crio-67070e7da659e916532086284b172ef58c69d6f972430b5ce11101d1bb04a7a1 WatchSource:0}: Error finding container 67070e7da659e916532086284b172ef58c69d6f972430b5ce11101d1bb04a7a1: Status 404 returned error can't find the container with id 67070e7da659e916532086284b172ef58c69d6f972430b5ce11101d1bb04a7a1
Oct 31 01:00:01.142377 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:01.142344    4284 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-f6979f-7754cc869f-gmkpn"]
Oct 31 01:00:01.147686 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:01.147665    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-f6979f-7754cc869f-gmkpn"
Oct 31 01:00:01.179691 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:01.179659    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b5d4b079-fbfb-4b9a-9027-47ab2b92aae5-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-f6979f-7754cc869f-gmkpn\" (UID: \"b5d4b079-fbfb-4b9a-9027-47ab2b92aae5\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-f6979f-7754cc869f-gmkpn"
Oct 31 01:00:01.179853 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:01.179699    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zv57q\" (UniqueName: \"kubernetes.io/projected/b5d4b079-fbfb-4b9a-9027-47ab2b92aae5-kube-api-access-zv57q\") pod \"payload-pre-processing-e2e-ipp-b-f6979f-7754cc869f-gmkpn\" (UID: \"b5d4b079-fbfb-4b9a-9027-47ab2b92aae5\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-f6979f-7754cc869f-gmkpn"
Oct 31 01:00:01.200798 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:01.200769    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-f6979f-7754cc869f-gmkpn"]
Oct 31 01:00:01.244408 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:01.244323    4284 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-b-f6979f-dockercfg-lqpl4\""
Oct 31 01:00:01.244408 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:01.244352    4284 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-b-f6979f\""
Oct 31 01:00:01.280971 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:01.280939    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b5d4b079-fbfb-4b9a-9027-47ab2b92aae5-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-f6979f-7754cc869f-gmkpn\" (UID: \"b5d4b079-fbfb-4b9a-9027-47ab2b92aae5\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-f6979f-7754cc869f-gmkpn"
Oct 31 01:00:01.281156 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:01.280979    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zv57q\" (UniqueName: \"kubernetes.io/projected/b5d4b079-fbfb-4b9a-9027-47ab2b92aae5-kube-api-access-zv57q\") pod \"payload-pre-processing-e2e-ipp-b-f6979f-7754cc869f-gmkpn\" (UID: \"b5d4b079-fbfb-4b9a-9027-47ab2b92aae5\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-f6979f-7754cc869f-gmkpn"
Oct 31 01:00:01.281661 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:01.281642    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b5d4b079-fbfb-4b9a-9027-47ab2b92aae5-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-f6979f-7754cc869f-gmkpn\" (UID: \"b5d4b079-fbfb-4b9a-9027-47ab2b92aae5\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-f6979f-7754cc869f-gmkpn"
Oct 31 01:00:01.342499 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:01.342471    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zv57q\" (UniqueName: \"kubernetes.io/projected/b5d4b079-fbfb-4b9a-9027-47ab2b92aae5-kube-api-access-zv57q\") pod \"payload-pre-processing-e2e-ipp-b-f6979f-7754cc869f-gmkpn\" (UID: \"b5d4b079-fbfb-4b9a-9027-47ab2b92aae5\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-f6979f-7754cc869f-gmkpn"
Oct 31 01:00:01.463055 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:01.463019    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-f6979f-7754cc869f-gmkpn"
Oct 31 01:00:01.662247 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:01.662198    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-b63c24-29890140-jxtzl" event={"ID":"6af0f6fa-a965-4083-af26-8338eeb6e796","Type":"ContainerStarted","Data":"e4aca4418293c151faa823b68c6ab96ad67f56c72291b8fefef9db293b429319"}
Oct 31 01:00:01.662726 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:01.662269    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-b63c24-29890140-jxtzl" event={"ID":"6af0f6fa-a965-4083-af26-8338eeb6e796","Type":"ContainerStarted","Data":"67070e7da659e916532086284b172ef58c69d6f972430b5ce11101d1bb04a7a1"}
Oct 31 01:00:01.774621 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:01.774513    4284 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-b63c24-29890140-jxtzl" podStartSLOduration=1.774498748 podStartE2EDuration="1.774498748s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 01:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 01:00:01.772428964 +0000 UTC m=+1915.917435881" watchObservedRunningTime="2026-10-31 01:00:01.774498748 +0000 UTC m=+1915.919505664"
Oct 31 01:00:01.836542 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:01.836508    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-f6979f-7754cc869f-gmkpn"]
Oct 31 01:00:01.836855 ip-10-0-132-173 kubenswrapper[4284]: W1031 01:00:01.836830    4284 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb5d4b079_fbfb_4b9a_9027_47ab2b92aae5.slice/crio-5afd021614af06d83f5525bd66bc186c91a27057722d4ad56de59ba440ac83f0 WatchSource:0}: Error finding container 5afd021614af06d83f5525bd66bc186c91a27057722d4ad56de59ba440ac83f0: Status 404 returned error can't find the container with id 5afd021614af06d83f5525bd66bc186c91a27057722d4ad56de59ba440ac83f0
Oct 31 01:00:01.872888 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:01.872859    4284 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-f6979f-78bd886898-gf67r"]
Oct 31 01:00:01.878341 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:01.878321    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-f6979f-78bd886898-gf67r"
Oct 31 01:00:01.889714 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:01.889690    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8a1ca904-c09a-4b92-ab80-72241680c116-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-f6979f-78bd886898-gf67r\" (UID: \"8a1ca904-c09a-4b92-ab80-72241680c116\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-f6979f-78bd886898-gf67r"
Oct 31 01:00:01.889889 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:01.889866    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t85hx\" (UniqueName: \"kubernetes.io/projected/8a1ca904-c09a-4b92-ab80-72241680c116-kube-api-access-t85hx\") pod \"payload-processing-e2e-ipp-b-f6979f-78bd886898-gf67r\" (UID: \"8a1ca904-c09a-4b92-ab80-72241680c116\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-f6979f-78bd886898-gf67r"
Oct 31 01:00:01.934432 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:01.934401    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-f6979f-78bd886898-gf67r"]
Oct 31 01:00:01.991048 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:01.991013    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8a1ca904-c09a-4b92-ab80-72241680c116-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-f6979f-78bd886898-gf67r\" (UID: \"8a1ca904-c09a-4b92-ab80-72241680c116\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-f6979f-78bd886898-gf67r"
Oct 31 01:00:01.991241 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:01.991155    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t85hx\" (UniqueName: \"kubernetes.io/projected/8a1ca904-c09a-4b92-ab80-72241680c116-kube-api-access-t85hx\") pod \"payload-processing-e2e-ipp-b-f6979f-78bd886898-gf67r\" (UID: \"8a1ca904-c09a-4b92-ab80-72241680c116\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-f6979f-78bd886898-gf67r"
Oct 31 01:00:01.992281 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:01.992254    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8a1ca904-c09a-4b92-ab80-72241680c116-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-f6979f-78bd886898-gf67r\" (UID: \"8a1ca904-c09a-4b92-ab80-72241680c116\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-f6979f-78bd886898-gf67r"
Oct 31 01:00:02.050641 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:02.050568    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t85hx\" (UniqueName: \"kubernetes.io/projected/8a1ca904-c09a-4b92-ab80-72241680c116-kube-api-access-t85hx\") pod \"payload-processing-e2e-ipp-b-f6979f-78bd886898-gf67r\" (UID: \"8a1ca904-c09a-4b92-ab80-72241680c116\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-f6979f-78bd886898-gf67r"
Oct 31 01:00:02.257857 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:02.257825    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-f6979f-78bd886898-gf67r"
Oct 31 01:00:02.640192 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:02.640162    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-f6979f-78bd886898-gf67r"]
Oct 31 01:00:02.640580 ip-10-0-132-173 kubenswrapper[4284]: W1031 01:00:02.640549    4284 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8a1ca904_c09a_4b92_ab80_72241680c116.slice/crio-cbfa6505e86269d313240ae65311054164c39a424391d7af138b5b20e98d05b5 WatchSource:0}: Error finding container cbfa6505e86269d313240ae65311054164c39a424391d7af138b5b20e98d05b5: Status 404 returned error can't find the container with id cbfa6505e86269d313240ae65311054164c39a424391d7af138b5b20e98d05b5
Oct 31 01:00:02.670048 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:02.670013    4284 generic.go:356] "Generic (PLEG): container finished" podID="6af0f6fa-a965-4083-af26-8338eeb6e796" containerID="e4aca4418293c151faa823b68c6ab96ad67f56c72291b8fefef9db293b429319" exitCode=60
Oct 31 01:00:02.670470 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:02.670098    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-b63c24-29890140-jxtzl" event={"ID":"6af0f6fa-a965-4083-af26-8338eeb6e796","Type":"ContainerDied","Data":"e4aca4418293c151faa823b68c6ab96ad67f56c72291b8fefef9db293b429319"}
Oct 31 01:00:02.670470 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:02.670379    4284 scope.go:122] "RemoveContainer" containerID="e4aca4418293c151faa823b68c6ab96ad67f56c72291b8fefef9db293b429319"
Oct 31 01:00:02.671451 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:02.671429    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-f6979f-78bd886898-gf67r" event={"ID":"8a1ca904-c09a-4b92-ab80-72241680c116","Type":"ContainerStarted","Data":"cbfa6505e86269d313240ae65311054164c39a424391d7af138b5b20e98d05b5"}
Oct 31 01:00:02.673240 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:02.673183    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-f6979f-7754cc869f-gmkpn" event={"ID":"b5d4b079-fbfb-4b9a-9027-47ab2b92aae5","Type":"ContainerStarted","Data":"f53da27474169195c494765752239be6973b48d29961d0f418ac32f2e34cbd89"}
Oct 31 01:00:02.673345 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:02.673247    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-f6979f-7754cc869f-gmkpn" event={"ID":"b5d4b079-fbfb-4b9a-9027-47ab2b92aae5","Type":"ContainerStarted","Data":"5afd021614af06d83f5525bd66bc186c91a27057722d4ad56de59ba440ac83f0"}
Oct 31 01:00:02.673392 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:02.673363    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-f6979f-7754cc869f-gmkpn"
Oct 31 01:00:02.895381 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:02.895284    4284 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-f6979f-7754cc869f-gmkpn" podStartSLOduration=1.8952682109999999 podStartE2EDuration="1.895268211s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 01:00:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 01:00:02.892591146 +0000 UTC m=+1917.037598075" watchObservedRunningTime="2026-10-31 01:00:02.895268211 +0000 UTC m=+1917.040275127"
Oct 31 01:00:03.167891 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:03.167861    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:00:03.682137 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:03.682104    4284 generic.go:356] "Generic (PLEG): container finished" podID="6af0f6fa-a965-4083-af26-8338eeb6e796" containerID="f72d4aa46c91f9740971bd2799b03ca34835810acfe00ae590cb399ff5a4b6d6" exitCode=60
Oct 31 01:00:03.682643 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:03.682186    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-b63c24-29890140-jxtzl" event={"ID":"6af0f6fa-a965-4083-af26-8338eeb6e796","Type":"ContainerDied","Data":"f72d4aa46c91f9740971bd2799b03ca34835810acfe00ae590cb399ff5a4b6d6"}
Oct 31 01:00:03.682643 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:03.682275    4284 scope.go:122] "RemoveContainer" containerID="e4aca4418293c151faa823b68c6ab96ad67f56c72291b8fefef9db293b429319"
Oct 31 01:00:03.682643 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:03.682568    4284 scope.go:122] "RemoveContainer" containerID="f72d4aa46c91f9740971bd2799b03ca34835810acfe00ae590cb399ff5a4b6d6"
Oct 31 01:00:03.682953 ip-10-0-132-173 kubenswrapper[4284]: E1031 01:00:03.682928    4284 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w1-b63c24-29890140-jxtzl_odh-ai-gateway-infra(6af0f6fa-a965-4083-af26-8338eeb6e796)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-b63c24-29890140-jxtzl" podUID="6af0f6fa-a965-4083-af26-8338eeb6e796"
Oct 31 01:00:03.684383 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:03.684353    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-f6979f-78bd886898-gf67r" event={"ID":"8a1ca904-c09a-4b92-ab80-72241680c116","Type":"ContainerStarted","Data":"e84b1f8636bc89199d26c5a57a57975dd4929c263b690963b9a4fa74f6fb8e31"}
Oct 31 01:00:03.684833 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:03.684505    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-f6979f-78bd886898-gf67r"
Oct 31 01:00:03.686477 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:03.686457    4284 generic.go:356] "Generic (PLEG): container finished" podID="1e029b42-c66c-4ef0-b6e4-0a5ef5670758" containerID="cd8e6d6347680c0a4421384a11472c5da9bcacbaafb9153b701402aaa2e28f6e" exitCode=0
Oct 31 01:00:03.686588 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:03.686533    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-f66180-openshift-default-55cfc8577c-lfq7v" event={"ID":"1e029b42-c66c-4ef0-b6e4-0a5ef5670758","Type":"ContainerDied","Data":"cd8e6d6347680c0a4421384a11472c5da9bcacbaafb9153b701402aaa2e28f6e"}
Oct 31 01:00:03.713002 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:03.712972    4284 scope.go:122] "RemoveContainer" containerID="75bee10e954827824a90ad4ba61a1a6327708e401bb1c1759bff7b703d745e5d"
Oct 31 01:00:03.818715 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:03.818683    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-f66180-openshift-default-55cfc8577c-lfq7v"
Oct 31 01:00:03.907655 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:03.907591    4284 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-b-f6979f-78bd886898-gf67r" podStartSLOduration=2.907573463 podStartE2EDuration="2.907573463s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 01:00:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 01:00:03.905642147 +0000 UTC m=+1918.050649064" watchObservedRunningTime="2026-10-31 01:00:03.907573463 +0000 UTC m=+1918.052580380"
Oct 31 01:00:03.950074 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:03.950038    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1e029b42-c66c-4ef0-b6e4-0a5ef5670758-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1e029b42-c66c-4ef0-b6e4-0a5ef5670758-istiod-ca-cert\") pod \"1e029b42-c66c-4ef0-b6e4-0a5ef5670758\" (UID: \"1e029b42-c66c-4ef0-b6e4-0a5ef5670758\") "
Oct 31 01:00:03.950265 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:03.950133    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/1e029b42-c66c-4ef0-b6e4-0a5ef5670758-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1e029b42-c66c-4ef0-b6e4-0a5ef5670758-istio-podinfo\") pod \"1e029b42-c66c-4ef0-b6e4-0a5ef5670758\" (UID: \"1e029b42-c66c-4ef0-b6e4-0a5ef5670758\") "
Oct 31 01:00:03.950265 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:03.950158    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1e029b42-c66c-4ef0-b6e4-0a5ef5670758-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1e029b42-c66c-4ef0-b6e4-0a5ef5670758-workload-socket\") pod \"1e029b42-c66c-4ef0-b6e4-0a5ef5670758\" (UID: \"1e029b42-c66c-4ef0-b6e4-0a5ef5670758\") "
Oct 31 01:00:03.950265 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:03.950179    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1e029b42-c66c-4ef0-b6e4-0a5ef5670758-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1e029b42-c66c-4ef0-b6e4-0a5ef5670758-istio-envoy\") pod \"1e029b42-c66c-4ef0-b6e4-0a5ef5670758\" (UID: \"1e029b42-c66c-4ef0-b6e4-0a5ef5670758\") "
Oct 31 01:00:03.950265 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:03.950242    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1e029b42-c66c-4ef0-b6e4-0a5ef5670758-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1e029b42-c66c-4ef0-b6e4-0a5ef5670758-istio-data\") pod \"1e029b42-c66c-4ef0-b6e4-0a5ef5670758\" (UID: \"1e029b42-c66c-4ef0-b6e4-0a5ef5670758\") "
Oct 31 01:00:03.950427 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:03.950326    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1e029b42-c66c-4ef0-b6e4-0a5ef5670758-kube-api-access-8kjss\" (UniqueName: \"kubernetes.io/projected/1e029b42-c66c-4ef0-b6e4-0a5ef5670758-kube-api-access-8kjss\") pod \"1e029b42-c66c-4ef0-b6e4-0a5ef5670758\" (UID: \"1e029b42-c66c-4ef0-b6e4-0a5ef5670758\") "
Oct 31 01:00:03.950427 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:03.950410    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1e029b42-c66c-4ef0-b6e4-0a5ef5670758-istio-token\" (UniqueName: \"kubernetes.io/projected/1e029b42-c66c-4ef0-b6e4-0a5ef5670758-istio-token\") pod \"1e029b42-c66c-4ef0-b6e4-0a5ef5670758\" (UID: \"1e029b42-c66c-4ef0-b6e4-0a5ef5670758\") "
Oct 31 01:00:03.950513 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:03.950494    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1e029b42-c66c-4ef0-b6e4-0a5ef5670758-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1e029b42-c66c-4ef0-b6e4-0a5ef5670758-workload-certs\") pod \"1e029b42-c66c-4ef0-b6e4-0a5ef5670758\" (UID: \"1e029b42-c66c-4ef0-b6e4-0a5ef5670758\") "
Oct 31 01:00:03.950513 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:03.950489    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1e029b42-c66c-4ef0-b6e4-0a5ef5670758-workload-socket" pod "1e029b42-c66c-4ef0-b6e4-0a5ef5670758" (UID: "1e029b42-c66c-4ef0-b6e4-0a5ef5670758"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:00:03.950587 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:03.950516    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1e029b42-c66c-4ef0-b6e4-0a5ef5670758-istiod-ca-cert" pod "1e029b42-c66c-4ef0-b6e4-0a5ef5670758" (UID: "1e029b42-c66c-4ef0-b6e4-0a5ef5670758"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 01:00:03.950631 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:03.950613    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1e029b42-c66c-4ef0-b6e4-0a5ef5670758-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1e029b42-c66c-4ef0-b6e4-0a5ef5670758-credential-socket\") pod \"1e029b42-c66c-4ef0-b6e4-0a5ef5670758\" (UID: \"1e029b42-c66c-4ef0-b6e4-0a5ef5670758\") "
Oct 31 01:00:03.950631 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:03.950619    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1e029b42-c66c-4ef0-b6e4-0a5ef5670758-istio-data" pod "1e029b42-c66c-4ef0-b6e4-0a5ef5670758" (UID: "1e029b42-c66c-4ef0-b6e4-0a5ef5670758"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:00:03.950796 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:03.950774    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1e029b42-c66c-4ef0-b6e4-0a5ef5670758-workload-certs" pod "1e029b42-c66c-4ef0-b6e4-0a5ef5670758" (UID: "1e029b42-c66c-4ef0-b6e4-0a5ef5670758"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:00:03.951245 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:03.950937    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1e029b42-c66c-4ef0-b6e4-0a5ef5670758-credential-socket" pod "1e029b42-c66c-4ef0-b6e4-0a5ef5670758" (UID: "1e029b42-c66c-4ef0-b6e4-0a5ef5670758"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:00:03.951359 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:03.951309    4284 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1e029b42-c66c-4ef0-b6e4-0a5ef5670758-workload-certs\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:00:03.951359 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:03.951333    4284 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1e029b42-c66c-4ef0-b6e4-0a5ef5670758-credential-socket\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:00:03.951359 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:03.951349    4284 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1e029b42-c66c-4ef0-b6e4-0a5ef5670758-istiod-ca-cert\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:00:03.951552 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:03.951363    4284 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1e029b42-c66c-4ef0-b6e4-0a5ef5670758-workload-socket\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:00:03.951552 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:03.951377    4284 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1e029b42-c66c-4ef0-b6e4-0a5ef5670758-istio-data\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:00:03.952750 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:03.952725    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/1e029b42-c66c-4ef0-b6e4-0a5ef5670758-istio-podinfo" pod "1e029b42-c66c-4ef0-b6e4-0a5ef5670758" (UID: "1e029b42-c66c-4ef0-b6e4-0a5ef5670758"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 01:00:03.952850 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:03.952826    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1e029b42-c66c-4ef0-b6e4-0a5ef5670758-istio-token" pod "1e029b42-c66c-4ef0-b6e4-0a5ef5670758" (UID: "1e029b42-c66c-4ef0-b6e4-0a5ef5670758"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:00:03.952902 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:03.952847    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1e029b42-c66c-4ef0-b6e4-0a5ef5670758-istio-envoy" pod "1e029b42-c66c-4ef0-b6e4-0a5ef5670758" (UID: "1e029b42-c66c-4ef0-b6e4-0a5ef5670758"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:00:03.952902 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:03.952865    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1e029b42-c66c-4ef0-b6e4-0a5ef5670758-kube-api-access-8kjss" pod "1e029b42-c66c-4ef0-b6e4-0a5ef5670758" (UID: "1e029b42-c66c-4ef0-b6e4-0a5ef5670758"). InnerVolumeSpecName "kube-api-access-8kjss". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:00:04.056233 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:04.053719    4284 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1e029b42-c66c-4ef0-b6e4-0a5ef5670758-istio-token\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:00:04.056233 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:04.053760    4284 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1e029b42-c66c-4ef0-b6e4-0a5ef5670758-istio-podinfo\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:00:04.056233 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:04.053777    4284 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1e029b42-c66c-4ef0-b6e4-0a5ef5670758-istio-envoy\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:00:04.056233 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:04.053793    4284 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8kjss\" (UniqueName: \"kubernetes.io/projected/1e029b42-c66c-4ef0-b6e4-0a5ef5670758-kube-api-access-8kjss\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:00:04.694906 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:04.694874    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-f66180-openshift-default-55cfc8577c-lfq7v"
Oct 31 01:00:04.694906 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:04.694888    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-f66180-openshift-default-55cfc8577c-lfq7v" event={"ID":"1e029b42-c66c-4ef0-b6e4-0a5ef5670758","Type":"ContainerDied","Data":"b6c915f018d7e7a15d3bf5f363484fb004438067e42ea735a38f631fff743612"}
Oct 31 01:00:04.695521 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:04.694950    4284 scope.go:122] "RemoveContainer" containerID="cd8e6d6347680c0a4421384a11472c5da9bcacbaafb9153b701402aaa2e28f6e"
Oct 31 01:00:04.697762 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:04.697739    4284 scope.go:122] "RemoveContainer" containerID="f72d4aa46c91f9740971bd2799b03ca34835810acfe00ae590cb399ff5a4b6d6"
Oct 31 01:00:04.698132 ip-10-0-132-173 kubenswrapper[4284]: E1031 01:00:04.698105    4284 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w1-b63c24-29890140-jxtzl_odh-ai-gateway-infra(6af0f6fa-a965-4083-af26-8338eeb6e796)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-b63c24-29890140-jxtzl" podUID="6af0f6fa-a965-4083-af26-8338eeb6e796"
Oct 31 01:00:04.809766 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:04.809723    4284 patch_prober.go:29] interesting pod/e2e-ipp-a-f66180-openshift-default-55cfc8577c-lfq7v container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.97:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 31 01:00:04.809951 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:04.809784    4284 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-a-f66180-openshift-default-55cfc8577c-lfq7v" podUID="1e029b42-c66c-4ef0-b6e4-0a5ef5670758" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.97:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 31 01:00:04.967194 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:04.967107    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-f66180-openshift-default-55cfc8577c-lfq7v"]
Oct 31 01:00:04.972824 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:04.972792    4284 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-a-f66180-openshift-default-55cfc8577c-lfq7v"]
Oct 31 01:00:05.356974 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:05.356948    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-29ece5-openshift-default-6fcb86dbc-hl96f"
Oct 31 01:00:05.470654 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:05.470613    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/19acc006-2388-488d-bfff-6ce6bab123a8-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/19acc006-2388-488d-bfff-6ce6bab123a8-istio-data\") pod \"19acc006-2388-488d-bfff-6ce6bab123a8\" (UID: \"19acc006-2388-488d-bfff-6ce6bab123a8\") "
Oct 31 01:00:05.470839 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:05.470699    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/19acc006-2388-488d-bfff-6ce6bab123a8-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/19acc006-2388-488d-bfff-6ce6bab123a8-istiod-ca-cert\") pod \"19acc006-2388-488d-bfff-6ce6bab123a8\" (UID: \"19acc006-2388-488d-bfff-6ce6bab123a8\") "
Oct 31 01:00:05.470839 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:05.470731    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/19acc006-2388-488d-bfff-6ce6bab123a8-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/19acc006-2388-488d-bfff-6ce6bab123a8-credential-socket\") pod \"19acc006-2388-488d-bfff-6ce6bab123a8\" (UID: \"19acc006-2388-488d-bfff-6ce6bab123a8\") "
Oct 31 01:00:05.470839 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:05.470756    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/19acc006-2388-488d-bfff-6ce6bab123a8-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/19acc006-2388-488d-bfff-6ce6bab123a8-workload-certs\") pod \"19acc006-2388-488d-bfff-6ce6bab123a8\" (UID: \"19acc006-2388-488d-bfff-6ce6bab123a8\") "
Oct 31 01:00:05.470839 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:05.470782    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/19acc006-2388-488d-bfff-6ce6bab123a8-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/19acc006-2388-488d-bfff-6ce6bab123a8-istio-podinfo\") pod \"19acc006-2388-488d-bfff-6ce6bab123a8\" (UID: \"19acc006-2388-488d-bfff-6ce6bab123a8\") "
Oct 31 01:00:05.471000 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:05.470904    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/19acc006-2388-488d-bfff-6ce6bab123a8-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/19acc006-2388-488d-bfff-6ce6bab123a8-istio-envoy\") pod \"19acc006-2388-488d-bfff-6ce6bab123a8\" (UID: \"19acc006-2388-488d-bfff-6ce6bab123a8\") "
Oct 31 01:00:05.471000 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:05.470955    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/19acc006-2388-488d-bfff-6ce6bab123a8-istio-data" pod "19acc006-2388-488d-bfff-6ce6bab123a8" (UID: "19acc006-2388-488d-bfff-6ce6bab123a8"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:00:05.471000 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:05.470990    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/19acc006-2388-488d-bfff-6ce6bab123a8-kube-api-access-jrs5w\" (UniqueName: \"kubernetes.io/projected/19acc006-2388-488d-bfff-6ce6bab123a8-kube-api-access-jrs5w\") pod \"19acc006-2388-488d-bfff-6ce6bab123a8\" (UID: \"19acc006-2388-488d-bfff-6ce6bab123a8\") "
Oct 31 01:00:05.471153 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:05.470999    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/19acc006-2388-488d-bfff-6ce6bab123a8-workload-certs" pod "19acc006-2388-488d-bfff-6ce6bab123a8" (UID: "19acc006-2388-488d-bfff-6ce6bab123a8"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:00:05.471153 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:05.471059    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/19acc006-2388-488d-bfff-6ce6bab123a8-istio-token\" (UniqueName: \"kubernetes.io/projected/19acc006-2388-488d-bfff-6ce6bab123a8-istio-token\") pod \"19acc006-2388-488d-bfff-6ce6bab123a8\" (UID: \"19acc006-2388-488d-bfff-6ce6bab123a8\") "
Oct 31 01:00:05.471153 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:05.471117    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/19acc006-2388-488d-bfff-6ce6bab123a8-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/19acc006-2388-488d-bfff-6ce6bab123a8-workload-socket\") pod \"19acc006-2388-488d-bfff-6ce6bab123a8\" (UID: \"19acc006-2388-488d-bfff-6ce6bab123a8\") "
Oct 31 01:00:05.471153 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:05.471140    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/19acc006-2388-488d-bfff-6ce6bab123a8-credential-socket" pod "19acc006-2388-488d-bfff-6ce6bab123a8" (UID: "19acc006-2388-488d-bfff-6ce6bab123a8"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:00:05.471415 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:05.471191    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/19acc006-2388-488d-bfff-6ce6bab123a8-istiod-ca-cert" pod "19acc006-2388-488d-bfff-6ce6bab123a8" (UID: "19acc006-2388-488d-bfff-6ce6bab123a8"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 01:00:05.471489 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:05.471419    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/19acc006-2388-488d-bfff-6ce6bab123a8-workload-socket" pod "19acc006-2388-488d-bfff-6ce6bab123a8" (UID: "19acc006-2388-488d-bfff-6ce6bab123a8"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:00:05.471851 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:05.471735    4284 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/19acc006-2388-488d-bfff-6ce6bab123a8-workload-socket\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:00:05.471851 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:05.471761    4284 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/19acc006-2388-488d-bfff-6ce6bab123a8-istio-data\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:00:05.471851 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:05.471817    4284 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/19acc006-2388-488d-bfff-6ce6bab123a8-istiod-ca-cert\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:00:05.472060 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:05.471917    4284 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/19acc006-2388-488d-bfff-6ce6bab123a8-credential-socket\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:00:05.472060 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:05.471964    4284 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/19acc006-2388-488d-bfff-6ce6bab123a8-workload-certs\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:00:05.473954 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:05.473923    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/19acc006-2388-488d-bfff-6ce6bab123a8-istio-envoy" pod "19acc006-2388-488d-bfff-6ce6bab123a8" (UID: "19acc006-2388-488d-bfff-6ce6bab123a8"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:00:05.474397 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:05.474362    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/19acc006-2388-488d-bfff-6ce6bab123a8-istio-podinfo" pod "19acc006-2388-488d-bfff-6ce6bab123a8" (UID: "19acc006-2388-488d-bfff-6ce6bab123a8"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 01:00:05.474516 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:05.474415    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/19acc006-2388-488d-bfff-6ce6bab123a8-istio-token" pod "19acc006-2388-488d-bfff-6ce6bab123a8" (UID: "19acc006-2388-488d-bfff-6ce6bab123a8"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:00:05.474685 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:05.474662    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/19acc006-2388-488d-bfff-6ce6bab123a8-kube-api-access-jrs5w" pod "19acc006-2388-488d-bfff-6ce6bab123a8" (UID: "19acc006-2388-488d-bfff-6ce6bab123a8"). InnerVolumeSpecName "kube-api-access-jrs5w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:00:05.572841 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:05.572758    4284 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/19acc006-2388-488d-bfff-6ce6bab123a8-istio-envoy\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:00:05.572841 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:05.572787    4284 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jrs5w\" (UniqueName: \"kubernetes.io/projected/19acc006-2388-488d-bfff-6ce6bab123a8-kube-api-access-jrs5w\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:00:05.572841 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:05.572800    4284 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/19acc006-2388-488d-bfff-6ce6bab123a8-istio-token\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:00:05.572841 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:05.572811    4284 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/19acc006-2388-488d-bfff-6ce6bab123a8-istio-podinfo\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:00:05.704919 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:05.704882    4284 generic.go:356] "Generic (PLEG): container finished" podID="19acc006-2388-488d-bfff-6ce6bab123a8" containerID="cc637542466f434173e4554b90b9e78e20e03f3c87b39f900985c90e5e524c3a" exitCode=0
Oct 31 01:00:05.705462 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:05.704939    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-29ece5-openshift-default-6fcb86dbc-hl96f"
Oct 31 01:00:05.705462 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:05.704970    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-29ece5-openshift-default-6fcb86dbc-hl96f" event={"ID":"19acc006-2388-488d-bfff-6ce6bab123a8","Type":"ContainerDied","Data":"cc637542466f434173e4554b90b9e78e20e03f3c87b39f900985c90e5e524c3a"}
Oct 31 01:00:05.705462 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:05.705036    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-29ece5-openshift-default-6fcb86dbc-hl96f" event={"ID":"19acc006-2388-488d-bfff-6ce6bab123a8","Type":"ContainerDied","Data":"511cc46aa456ba4633a13d1258deabe17a03dcd191811a49effce1d4838f38de"}
Oct 31 01:00:05.705462 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:05.705074    4284 scope.go:122] "RemoveContainer" containerID="cc637542466f434173e4554b90b9e78e20e03f3c87b39f900985c90e5e524c3a"
Oct 31 01:00:05.718540 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:05.718519    4284 scope.go:122] "RemoveContainer" containerID="cc637542466f434173e4554b90b9e78e20e03f3c87b39f900985c90e5e524c3a"
Oct 31 01:00:05.718908 ip-10-0-132-173 kubenswrapper[4284]: E1031 01:00:05.718889    4284 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cc637542466f434173e4554b90b9e78e20e03f3c87b39f900985c90e5e524c3a\": container with ID starting with cc637542466f434173e4554b90b9e78e20e03f3c87b39f900985c90e5e524c3a not found: ID does not exist" containerID="cc637542466f434173e4554b90b9e78e20e03f3c87b39f900985c90e5e524c3a"
Oct 31 01:00:05.718964 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:05.718919    4284 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cc637542466f434173e4554b90b9e78e20e03f3c87b39f900985c90e5e524c3a"} err="failed to get container status \"cc637542466f434173e4554b90b9e78e20e03f3c87b39f900985c90e5e524c3a\": rpc error: code = NotFound desc = could not find container \"cc637542466f434173e4554b90b9e78e20e03f3c87b39f900985c90e5e524c3a\": container with ID starting with cc637542466f434173e4554b90b9e78e20e03f3c87b39f900985c90e5e524c3a not found: ID does not exist"
Oct 31 01:00:05.866957 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:05.866876    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-29ece5-openshift-default-6fcb86dbc-hl96f"]
Oct 31 01:00:05.874650 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:05.874622    4284 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w2-29ece5-openshift-default-6fcb86dbc-hl96f"]
Oct 31 01:00:05.963517 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:05.963485    4284 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="19acc006-2388-488d-bfff-6ce6bab123a8" path="/var/lib/kubelet/pods/19acc006-2388-488d-bfff-6ce6bab123a8/volumes"
Oct 31 01:00:05.963971 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:05.963958    4284 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1e029b42-c66c-4ef0-b6e4-0a5ef5670758" path="/var/lib/kubelet/pods/1e029b42-c66c-4ef0-b6e4-0a5ef5670758/volumes"
Oct 31 01:00:06.904927 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:06.904888    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:00:07.410077 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:07.410038    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:00:08.493390 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:08.493354    4284 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-b647c3-799446d767-7g2mp"]
Oct 31 01:00:08.493950 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:08.493937    4284 cpu_manager.go:414] "containerMap: removing container" podUID="1e029b42-c66c-4ef0-b6e4-0a5ef5670758" containerName="istio-proxy"
Oct 31 01:00:08.493990 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:08.493951    4284 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1e029b42-c66c-4ef0-b6e4-0a5ef5670758" containerName="istio-proxy"
Oct 31 01:00:08.493990 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:08.493974    4284 cpu_manager.go:414] "containerMap: removing container" podUID="1e029b42-c66c-4ef0-b6e4-0a5ef5670758" containerName="istio-proxy"
Oct 31 01:00:08.493990 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:08.493979    4284 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1e029b42-c66c-4ef0-b6e4-0a5ef5670758" containerName="istio-proxy"
Oct 31 01:00:08.493990 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:08.493987    4284 cpu_manager.go:414] "containerMap: removing container" podUID="19acc006-2388-488d-bfff-6ce6bab123a8" containerName="istio-proxy"
Oct 31 01:00:08.494129 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:08.493991    4284 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="19acc006-2388-488d-bfff-6ce6bab123a8" containerName="istio-proxy"
Oct 31 01:00:08.494129 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:08.494077    4284 memory_manager.go:365] "RemoveStaleState removing state" podUID="19acc006-2388-488d-bfff-6ce6bab123a8" containerName="istio-proxy"
Oct 31 01:00:08.494129 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:08.494089    4284 memory_manager.go:365] "RemoveStaleState removing state" podUID="1e029b42-c66c-4ef0-b6e4-0a5ef5670758" containerName="istio-proxy"
Oct 31 01:00:08.494129 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:08.494098    4284 memory_manager.go:365] "RemoveStaleState removing state" podUID="1e029b42-c66c-4ef0-b6e4-0a5ef5670758" containerName="istio-proxy"
Oct 31 01:00:08.499403 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:08.499383    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-b647c3-799446d767-7g2mp"
Oct 31 01:00:08.552868 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:08.552835    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-b647c3-799446d767-7g2mp"]
Oct 31 01:00:08.595961 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:08.595929    4284 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-a-b647c3\""
Oct 31 01:00:08.596126 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:08.596086    4284 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-a-b647c3-dockercfg-vqjq8\""
Oct 31 01:00:08.617917 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:08.617879    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-drr5m\" (UniqueName: \"kubernetes.io/projected/73ee0248-f0a6-4d16-9e30-5301a63fa113-kube-api-access-drr5m\") pod \"payload-pre-processing-e2e-ipp-a-b647c3-799446d767-7g2mp\" (UID: \"73ee0248-f0a6-4d16-9e30-5301a63fa113\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-b647c3-799446d767-7g2mp"
Oct 31 01:00:08.618093 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:08.618076    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/73ee0248-f0a6-4d16-9e30-5301a63fa113-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-b647c3-799446d767-7g2mp\" (UID: \"73ee0248-f0a6-4d16-9e30-5301a63fa113\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-b647c3-799446d767-7g2mp"
Oct 31 01:00:08.719893 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:08.719856    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-drr5m\" (UniqueName: \"kubernetes.io/projected/73ee0248-f0a6-4d16-9e30-5301a63fa113-kube-api-access-drr5m\") pod \"payload-pre-processing-e2e-ipp-a-b647c3-799446d767-7g2mp\" (UID: \"73ee0248-f0a6-4d16-9e30-5301a63fa113\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-b647c3-799446d767-7g2mp"
Oct 31 01:00:08.720076 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:08.720059    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/73ee0248-f0a6-4d16-9e30-5301a63fa113-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-b647c3-799446d767-7g2mp\" (UID: \"73ee0248-f0a6-4d16-9e30-5301a63fa113\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-b647c3-799446d767-7g2mp"
Oct 31 01:00:08.720788 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:08.720761    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/73ee0248-f0a6-4d16-9e30-5301a63fa113-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-b647c3-799446d767-7g2mp\" (UID: \"73ee0248-f0a6-4d16-9e30-5301a63fa113\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-b647c3-799446d767-7g2mp"
Oct 31 01:00:08.778748 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:08.778677    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-drr5m\" (UniqueName: \"kubernetes.io/projected/73ee0248-f0a6-4d16-9e30-5301a63fa113-kube-api-access-drr5m\") pod \"payload-pre-processing-e2e-ipp-a-b647c3-799446d767-7g2mp\" (UID: \"73ee0248-f0a6-4d16-9e30-5301a63fa113\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-b647c3-799446d767-7g2mp"
Oct 31 01:00:08.812323 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:08.812286    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-b647c3-799446d767-7g2mp"
Oct 31 01:00:09.185450 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:09.185423    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-b647c3-799446d767-7g2mp"]
Oct 31 01:00:09.185622 ip-10-0-132-173 kubenswrapper[4284]: W1031 01:00:09.185596    4284 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod73ee0248_f0a6_4d16_9e30_5301a63fa113.slice/crio-ffe1ff649692cba73f29f1e0d624c69be86c3d728b56d7ecf0e1bfd792131235 WatchSource:0}: Error finding container ffe1ff649692cba73f29f1e0d624c69be86c3d728b56d7ecf0e1bfd792131235: Status 404 returned error can't find the container with id ffe1ff649692cba73f29f1e0d624c69be86c3d728b56d7ecf0e1bfd792131235
Oct 31 01:00:09.296667 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:09.296632    4284 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-b647c3-5b7c7c4c44-srzhm"]
Oct 31 01:00:09.301076 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:09.301051    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-b647c3-5b7c7c4c44-srzhm"
Oct 31 01:00:09.357104 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:09.357071    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-b647c3-5b7c7c4c44-srzhm"]
Oct 31 01:00:09.428539 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:09.428504    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-62sbv\" (UniqueName: \"kubernetes.io/projected/1bfc3a17-7cea-4195-a6ab-0c82338451af-kube-api-access-62sbv\") pod \"payload-processing-e2e-ipp-a-b647c3-5b7c7c4c44-srzhm\" (UID: \"1bfc3a17-7cea-4195-a6ab-0c82338451af\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-b647c3-5b7c7c4c44-srzhm"
Oct 31 01:00:09.428744 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:09.428572    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1bfc3a17-7cea-4195-a6ab-0c82338451af-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-b647c3-5b7c7c4c44-srzhm\" (UID: \"1bfc3a17-7cea-4195-a6ab-0c82338451af\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-b647c3-5b7c7c4c44-srzhm"
Oct 31 01:00:09.530428 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:09.530345    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-62sbv\" (UniqueName: \"kubernetes.io/projected/1bfc3a17-7cea-4195-a6ab-0c82338451af-kube-api-access-62sbv\") pod \"payload-processing-e2e-ipp-a-b647c3-5b7c7c4c44-srzhm\" (UID: \"1bfc3a17-7cea-4195-a6ab-0c82338451af\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-b647c3-5b7c7c4c44-srzhm"
Oct 31 01:00:09.530428 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:09.530416    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1bfc3a17-7cea-4195-a6ab-0c82338451af-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-b647c3-5b7c7c4c44-srzhm\" (UID: \"1bfc3a17-7cea-4195-a6ab-0c82338451af\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-b647c3-5b7c7c4c44-srzhm"
Oct 31 01:00:09.531091 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:09.531070    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1bfc3a17-7cea-4195-a6ab-0c82338451af-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-b647c3-5b7c7c4c44-srzhm\" (UID: \"1bfc3a17-7cea-4195-a6ab-0c82338451af\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-b647c3-5b7c7c4c44-srzhm"
Oct 31 01:00:09.589578 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:09.589550    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-62sbv\" (UniqueName: \"kubernetes.io/projected/1bfc3a17-7cea-4195-a6ab-0c82338451af-kube-api-access-62sbv\") pod \"payload-processing-e2e-ipp-a-b647c3-5b7c7c4c44-srzhm\" (UID: \"1bfc3a17-7cea-4195-a6ab-0c82338451af\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-b647c3-5b7c7c4c44-srzhm"
Oct 31 01:00:09.616484 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:09.616435    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-b647c3-5b7c7c4c44-srzhm"
Oct 31 01:00:09.733836 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:09.733792    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-b647c3-799446d767-7g2mp" event={"ID":"73ee0248-f0a6-4d16-9e30-5301a63fa113","Type":"ContainerStarted","Data":"12e86cc48025aafdfb02221872e23243a50d37d8db54b2c05e250a3219d1b631"}
Oct 31 01:00:09.734046 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:09.733885    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-b647c3-799446d767-7g2mp" event={"ID":"73ee0248-f0a6-4d16-9e30-5301a63fa113","Type":"ContainerStarted","Data":"ffe1ff649692cba73f29f1e0d624c69be86c3d728b56d7ecf0e1bfd792131235"}
Oct 31 01:00:09.734046 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:09.733938    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-b647c3-799446d767-7g2mp"
Oct 31 01:00:09.848077 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:09.847977    4284 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-b647c3-799446d767-7g2mp" podStartSLOduration=1.8479645869999999 podStartE2EDuration="1.847964587s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 01:00:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 01:00:09.845589149 +0000 UTC m=+1923.990596066" watchObservedRunningTime="2026-10-31 01:00:09.847964587 +0000 UTC m=+1923.992971542"
Oct 31 01:00:10.004980 ip-10-0-132-173 kubenswrapper[4284]: W1031 01:00:10.004918    4284 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1bfc3a17_7cea_4195_a6ab_0c82338451af.slice/crio-bf6ed398aa85addbbcb3d9f0b1ad0069788f4bca484cacef7c99d429f75c1077 WatchSource:0}: Error finding container bf6ed398aa85addbbcb3d9f0b1ad0069788f4bca484cacef7c99d429f75c1077: Status 404 returned error can't find the container with id bf6ed398aa85addbbcb3d9f0b1ad0069788f4bca484cacef7c99d429f75c1077
Oct 31 01:00:10.005603 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:10.005574    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-b647c3-5b7c7c4c44-srzhm"]
Oct 31 01:00:10.350737 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:10.350701    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:00:10.744014 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:10.743951    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-b647c3-5b7c7c4c44-srzhm" event={"ID":"1bfc3a17-7cea-4195-a6ab-0c82338451af","Type":"ContainerStarted","Data":"5f1ef4ad48c15f4355c0e409487b49956acdf37d3f9d3b459f1b553461316781"}
Oct 31 01:00:10.744542 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:10.744295    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-b647c3-5b7c7c4c44-srzhm"
Oct 31 01:00:10.744542 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:10.744338    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-b647c3-5b7c7c4c44-srzhm" event={"ID":"1bfc3a17-7cea-4195-a6ab-0c82338451af","Type":"ContainerStarted","Data":"bf6ed398aa85addbbcb3d9f0b1ad0069788f4bca484cacef7c99d429f75c1077"}
Oct 31 01:00:10.865538 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:10.865490    4284 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-a-b647c3-5b7c7c4c44-srzhm" podStartSLOduration=1.865477767 podStartE2EDuration="1.865477767s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 01:00:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 01:00:10.862917821 +0000 UTC m=+1925.007924738" watchObservedRunningTime="2026-10-31 01:00:10.865477767 +0000 UTC m=+1925.010484682"
Oct 31 01:00:13.689418 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:13.689388    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-f6979f-7754cc869f-gmkpn"
Oct 31 01:00:14.700137 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:14.700109    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-f6979f-78bd886898-gf67r"
Oct 31 01:00:15.725139 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:15.725099    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:00:15.948989 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:15.948950    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:00:15.959833 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:15.959807    4284 scope.go:122] "RemoveContainer" containerID="f72d4aa46c91f9740971bd2799b03ca34835810acfe00ae590cb399ff5a4b6d6"
Oct 31 01:00:16.226186 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:16.226156    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:00:16.685164 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:16.685127    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:00:16.796926 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:16.796886    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-b63c24-29890140-jxtzl" event={"ID":"6af0f6fa-a965-4083-af26-8338eeb6e796","Type":"ContainerStarted","Data":"97621d2dd2099bbb614070d640ec7c5cfcea842000317bd848fb4f810956679e"}
Oct 31 01:00:17.018116 ip-10-0-132-173 kubenswrapper[4284]: E1031 01:00:17.018023    4284 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fd8sp" containerName="extractor"
Oct 31 01:00:17.806652 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:17.806613    4284 generic.go:356] "Generic (PLEG): container finished" podID="6af0f6fa-a965-4083-af26-8338eeb6e796" containerID="97621d2dd2099bbb614070d640ec7c5cfcea842000317bd848fb4f810956679e" exitCode=60
Oct 31 01:00:17.807160 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:17.806702    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-b63c24-29890140-jxtzl" event={"ID":"6af0f6fa-a965-4083-af26-8338eeb6e796","Type":"ContainerDied","Data":"97621d2dd2099bbb614070d640ec7c5cfcea842000317bd848fb4f810956679e"}
Oct 31 01:00:17.807160 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:17.806780    4284 scope.go:122] "RemoveContainer" containerID="f72d4aa46c91f9740971bd2799b03ca34835810acfe00ae590cb399ff5a4b6d6"
Oct 31 01:00:17.807323 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:17.807206    4284 scope.go:122] "RemoveContainer" containerID="97621d2dd2099bbb614070d640ec7c5cfcea842000317bd848fb4f810956679e"
Oct 31 01:00:17.807633 ip-10-0-132-173 kubenswrapper[4284]: E1031 01:00:17.807612    4284 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w1-b63c24-29890140-jxtzl_odh-ai-gateway-infra(6af0f6fa-a965-4083-af26-8338eeb6e796)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-b63c24-29890140-jxtzl" podUID="6af0f6fa-a965-4083-af26-8338eeb6e796"
Oct 31 01:00:17.918906 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:17.918875    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-b63c24-29890140-jxtzl"]
Oct 31 01:00:18.968006 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:18.967974    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-b63c24-29890140-jxtzl"
Oct 31 01:00:19.059581 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:19.059545    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6af0f6fa-a965-4083-af26-8338eeb6e796-kube-api-access-4r5kq\" (UniqueName: \"kubernetes.io/projected/6af0f6fa-a965-4083-af26-8338eeb6e796-kube-api-access-4r5kq\") pod \"6af0f6fa-a965-4083-af26-8338eeb6e796\" (UID: \"6af0f6fa-a965-4083-af26-8338eeb6e796\") "
Oct 31 01:00:19.059752 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:19.059609    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/6af0f6fa-a965-4083-af26-8338eeb6e796-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6af0f6fa-a965-4083-af26-8338eeb6e796-maas-api-service-ca\") pod \"6af0f6fa-a965-4083-af26-8338eeb6e796\" (UID: \"6af0f6fa-a965-4083-af26-8338eeb6e796\") "
Oct 31 01:00:19.062239 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:19.060503    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6af0f6fa-a965-4083-af26-8338eeb6e796-maas-api-service-ca" pod "6af0f6fa-a965-4083-af26-8338eeb6e796" (UID: "6af0f6fa-a965-4083-af26-8338eeb6e796"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 01:00:19.062239 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:19.061129    4284 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6af0f6fa-a965-4083-af26-8338eeb6e796-maas-api-service-ca\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:00:19.066269 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:19.063054    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6af0f6fa-a965-4083-af26-8338eeb6e796-kube-api-access-4r5kq" pod "6af0f6fa-a965-4083-af26-8338eeb6e796" (UID: "6af0f6fa-a965-4083-af26-8338eeb6e796"). InnerVolumeSpecName "kube-api-access-4r5kq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:00:19.162849 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:19.162813    4284 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4r5kq\" (UniqueName: \"kubernetes.io/projected/6af0f6fa-a965-4083-af26-8338eeb6e796-kube-api-access-4r5kq\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:00:19.822300 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:19.822269    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-b63c24-29890140-jxtzl"
Oct 31 01:00:19.822501 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:19.822310    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-b63c24-29890140-jxtzl" event={"ID":"6af0f6fa-a965-4083-af26-8338eeb6e796","Type":"ContainerDied","Data":"67070e7da659e916532086284b172ef58c69d6f972430b5ce11101d1bb04a7a1"}
Oct 31 01:00:19.822501 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:19.822389    4284 scope.go:122] "RemoveContainer" containerID="97621d2dd2099bbb614070d640ec7c5cfcea842000317bd848fb4f810956679e"
Oct 31 01:00:19.979788 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:19.979760    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-b63c24-29890140-jxtzl"]
Oct 31 01:00:19.983832 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:19.983809    4284 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-b63c24-29890140-jxtzl"]
Oct 31 01:00:20.746029 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:20.745993    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-b647c3-799446d767-7g2mp"
Oct 31 01:00:21.111718 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:21.111632    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:00:21.753070 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:21.753039    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-b647c3-5b7c7c4c44-srzhm"
Oct 31 01:00:21.963571 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:21.963542    4284 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6af0f6fa-a965-4083-af26-8338eeb6e796" path="/var/lib/kubelet/pods/6af0f6fa-a965-4083-af26-8338eeb6e796/volumes"
Oct 31 01:00:27.358717 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:27.358679    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:00:27.511065 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:27.511031    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:00:27.660814 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:27.660780    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:00:32.294418 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:32.294384    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:00:37.102382 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:37.102342    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:00:42.167834 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:42.167799    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:00:47.066819 ip-10-0-132-173 kubenswrapper[4284]: E1031 01:00:47.066786    4284 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fd8sp" containerName="extractor"
Oct 31 01:00:47.212239 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:47.202907    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:00:48.060981 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:48.060951    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:00:48.508238 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:48.507605    4284 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-31ae40-openshift-default-66bb6b84bc-8wrm6"]
Oct 31 01:00:48.516777 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:48.516744    4284 cpu_manager.go:414] "containerMap: removing container" podUID="6af0f6fa-a965-4083-af26-8338eeb6e796" containerName="cleanup"
Oct 31 01:00:48.517006 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:48.516988    4284 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6af0f6fa-a965-4083-af26-8338eeb6e796" containerName="cleanup"
Oct 31 01:00:48.517183 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:48.517170    4284 cpu_manager.go:414] "containerMap: removing container" podUID="6af0f6fa-a965-4083-af26-8338eeb6e796" containerName="cleanup"
Oct 31 01:00:48.517309 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:48.517296    4284 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6af0f6fa-a965-4083-af26-8338eeb6e796" containerName="cleanup"
Oct 31 01:00:48.517568 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:48.517554    4284 memory_manager.go:365] "RemoveStaleState removing state" podUID="6af0f6fa-a965-4083-af26-8338eeb6e796" containerName="cleanup"
Oct 31 01:00:48.517684 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:48.517673    4284 memory_manager.go:365] "RemoveStaleState removing state" podUID="6af0f6fa-a965-4083-af26-8338eeb6e796" containerName="cleanup"
Oct 31 01:00:48.517774 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:48.517764    4284 memory_manager.go:365] "RemoveStaleState removing state" podUID="6af0f6fa-a965-4083-af26-8338eeb6e796" containerName="cleanup"
Oct 31 01:00:48.523457 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:48.523430    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-31ae40-openshift-default-66bb6b84bc-8wrm6"
Oct 31 01:00:48.569677 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:48.568953    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5005c539-d22a-4c99-9235-e08fe509575f-istio-envoy\") pod \"e2e-api-b-31ae40-openshift-default-66bb6b84bc-8wrm6\" (UID: \"5005c539-d22a-4c99-9235-e08fe509575f\") " pod="openshift-ingress/e2e-api-b-31ae40-openshift-default-66bb6b84bc-8wrm6"
Oct 31 01:00:48.569677 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:48.569010    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5005c539-d22a-4c99-9235-e08fe509575f-istio-data\") pod \"e2e-api-b-31ae40-openshift-default-66bb6b84bc-8wrm6\" (UID: \"5005c539-d22a-4c99-9235-e08fe509575f\") " pod="openshift-ingress/e2e-api-b-31ae40-openshift-default-66bb6b84bc-8wrm6"
Oct 31 01:00:48.569677 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:48.569041    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5005c539-d22a-4c99-9235-e08fe509575f-workload-certs\") pod \"e2e-api-b-31ae40-openshift-default-66bb6b84bc-8wrm6\" (UID: \"5005c539-d22a-4c99-9235-e08fe509575f\") " pod="openshift-ingress/e2e-api-b-31ae40-openshift-default-66bb6b84bc-8wrm6"
Oct 31 01:00:48.569677 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:48.569104    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5005c539-d22a-4c99-9235-e08fe509575f-istio-podinfo\") pod \"e2e-api-b-31ae40-openshift-default-66bb6b84bc-8wrm6\" (UID: \"5005c539-d22a-4c99-9235-e08fe509575f\") " pod="openshift-ingress/e2e-api-b-31ae40-openshift-default-66bb6b84bc-8wrm6"
Oct 31 01:00:48.569677 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:48.569133    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5005c539-d22a-4c99-9235-e08fe509575f-workload-socket\") pod \"e2e-api-b-31ae40-openshift-default-66bb6b84bc-8wrm6\" (UID: \"5005c539-d22a-4c99-9235-e08fe509575f\") " pod="openshift-ingress/e2e-api-b-31ae40-openshift-default-66bb6b84bc-8wrm6"
Oct 31 01:00:48.569677 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:48.569161    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9tm4x\" (UniqueName: \"kubernetes.io/projected/5005c539-d22a-4c99-9235-e08fe509575f-kube-api-access-9tm4x\") pod \"e2e-api-b-31ae40-openshift-default-66bb6b84bc-8wrm6\" (UID: \"5005c539-d22a-4c99-9235-e08fe509575f\") " pod="openshift-ingress/e2e-api-b-31ae40-openshift-default-66bb6b84bc-8wrm6"
Oct 31 01:00:48.569677 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:48.569240    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5005c539-d22a-4c99-9235-e08fe509575f-credential-socket\") pod \"e2e-api-b-31ae40-openshift-default-66bb6b84bc-8wrm6\" (UID: \"5005c539-d22a-4c99-9235-e08fe509575f\") " pod="openshift-ingress/e2e-api-b-31ae40-openshift-default-66bb6b84bc-8wrm6"
Oct 31 01:00:48.569677 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:48.569300    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5005c539-d22a-4c99-9235-e08fe509575f-istiod-ca-cert\") pod \"e2e-api-b-31ae40-openshift-default-66bb6b84bc-8wrm6\" (UID: \"5005c539-d22a-4c99-9235-e08fe509575f\") " pod="openshift-ingress/e2e-api-b-31ae40-openshift-default-66bb6b84bc-8wrm6"
Oct 31 01:00:48.569677 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:48.569432    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5005c539-d22a-4c99-9235-e08fe509575f-istio-token\") pod \"e2e-api-b-31ae40-openshift-default-66bb6b84bc-8wrm6\" (UID: \"5005c539-d22a-4c99-9235-e08fe509575f\") " pod="openshift-ingress/e2e-api-b-31ae40-openshift-default-66bb6b84bc-8wrm6"
Oct 31 01:00:48.581489 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:48.581443    4284 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-31ae40-openshift-default-dockercfg-vj4t9\""
Oct 31 01:00:48.679248 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:48.679038    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5005c539-d22a-4c99-9235-e08fe509575f-istio-envoy\") pod \"e2e-api-b-31ae40-openshift-default-66bb6b84bc-8wrm6\" (UID: \"5005c539-d22a-4c99-9235-e08fe509575f\") " pod="openshift-ingress/e2e-api-b-31ae40-openshift-default-66bb6b84bc-8wrm6"
Oct 31 01:00:48.679248 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:48.679127    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5005c539-d22a-4c99-9235-e08fe509575f-istio-data\") pod \"e2e-api-b-31ae40-openshift-default-66bb6b84bc-8wrm6\" (UID: \"5005c539-d22a-4c99-9235-e08fe509575f\") " pod="openshift-ingress/e2e-api-b-31ae40-openshift-default-66bb6b84bc-8wrm6"
Oct 31 01:00:48.679248 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:48.679159    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5005c539-d22a-4c99-9235-e08fe509575f-workload-certs\") pod \"e2e-api-b-31ae40-openshift-default-66bb6b84bc-8wrm6\" (UID: \"5005c539-d22a-4c99-9235-e08fe509575f\") " pod="openshift-ingress/e2e-api-b-31ae40-openshift-default-66bb6b84bc-8wrm6"
Oct 31 01:00:48.679248 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:48.679248    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5005c539-d22a-4c99-9235-e08fe509575f-istio-podinfo\") pod \"e2e-api-b-31ae40-openshift-default-66bb6b84bc-8wrm6\" (UID: \"5005c539-d22a-4c99-9235-e08fe509575f\") " pod="openshift-ingress/e2e-api-b-31ae40-openshift-default-66bb6b84bc-8wrm6"
Oct 31 01:00:48.679652 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:48.679282    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5005c539-d22a-4c99-9235-e08fe509575f-workload-socket\") pod \"e2e-api-b-31ae40-openshift-default-66bb6b84bc-8wrm6\" (UID: \"5005c539-d22a-4c99-9235-e08fe509575f\") " pod="openshift-ingress/e2e-api-b-31ae40-openshift-default-66bb6b84bc-8wrm6"
Oct 31 01:00:48.679652 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:48.679309    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9tm4x\" (UniqueName: \"kubernetes.io/projected/5005c539-d22a-4c99-9235-e08fe509575f-kube-api-access-9tm4x\") pod \"e2e-api-b-31ae40-openshift-default-66bb6b84bc-8wrm6\" (UID: \"5005c539-d22a-4c99-9235-e08fe509575f\") " pod="openshift-ingress/e2e-api-b-31ae40-openshift-default-66bb6b84bc-8wrm6"
Oct 31 01:00:48.679652 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:48.679373    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5005c539-d22a-4c99-9235-e08fe509575f-credential-socket\") pod \"e2e-api-b-31ae40-openshift-default-66bb6b84bc-8wrm6\" (UID: \"5005c539-d22a-4c99-9235-e08fe509575f\") " pod="openshift-ingress/e2e-api-b-31ae40-openshift-default-66bb6b84bc-8wrm6"
Oct 31 01:00:48.679652 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:48.679438    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5005c539-d22a-4c99-9235-e08fe509575f-istiod-ca-cert\") pod \"e2e-api-b-31ae40-openshift-default-66bb6b84bc-8wrm6\" (UID: \"5005c539-d22a-4c99-9235-e08fe509575f\") " pod="openshift-ingress/e2e-api-b-31ae40-openshift-default-66bb6b84bc-8wrm6"
Oct 31 01:00:48.679652 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:48.679585    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5005c539-d22a-4c99-9235-e08fe509575f-istio-token\") pod \"e2e-api-b-31ae40-openshift-default-66bb6b84bc-8wrm6\" (UID: \"5005c539-d22a-4c99-9235-e08fe509575f\") " pod="openshift-ingress/e2e-api-b-31ae40-openshift-default-66bb6b84bc-8wrm6"
Oct 31 01:00:48.730260 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:48.715764    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5005c539-d22a-4c99-9235-e08fe509575f-workload-socket\") pod \"e2e-api-b-31ae40-openshift-default-66bb6b84bc-8wrm6\" (UID: \"5005c539-d22a-4c99-9235-e08fe509575f\") " pod="openshift-ingress/e2e-api-b-31ae40-openshift-default-66bb6b84bc-8wrm6"
Oct 31 01:00:48.730260 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:48.716024    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5005c539-d22a-4c99-9235-e08fe509575f-credential-socket\") pod \"e2e-api-b-31ae40-openshift-default-66bb6b84bc-8wrm6\" (UID: \"5005c539-d22a-4c99-9235-e08fe509575f\") " pod="openshift-ingress/e2e-api-b-31ae40-openshift-default-66bb6b84bc-8wrm6"
Oct 31 01:00:48.730260 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:48.724908    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5005c539-d22a-4c99-9235-e08fe509575f-istio-data\") pod \"e2e-api-b-31ae40-openshift-default-66bb6b84bc-8wrm6\" (UID: \"5005c539-d22a-4c99-9235-e08fe509575f\") " pod="openshift-ingress/e2e-api-b-31ae40-openshift-default-66bb6b84bc-8wrm6"
Oct 31 01:00:48.730260 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:48.725676    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5005c539-d22a-4c99-9235-e08fe509575f-istiod-ca-cert\") pod \"e2e-api-b-31ae40-openshift-default-66bb6b84bc-8wrm6\" (UID: \"5005c539-d22a-4c99-9235-e08fe509575f\") " pod="openshift-ingress/e2e-api-b-31ae40-openshift-default-66bb6b84bc-8wrm6"
Oct 31 01:00:48.730260 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:48.725919    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5005c539-d22a-4c99-9235-e08fe509575f-workload-certs\") pod \"e2e-api-b-31ae40-openshift-default-66bb6b84bc-8wrm6\" (UID: \"5005c539-d22a-4c99-9235-e08fe509575f\") " pod="openshift-ingress/e2e-api-b-31ae40-openshift-default-66bb6b84bc-8wrm6"
Oct 31 01:00:48.735275 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:48.731187    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:00:48.852411 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:48.803332    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-31ae40-openshift-default-66bb6b84bc-8wrm6"]
Oct 31 01:00:48.881397 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:48.881338    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5005c539-d22a-4c99-9235-e08fe509575f-istio-envoy\") pod \"e2e-api-b-31ae40-openshift-default-66bb6b84bc-8wrm6\" (UID: \"5005c539-d22a-4c99-9235-e08fe509575f\") " pod="openshift-ingress/e2e-api-b-31ae40-openshift-default-66bb6b84bc-8wrm6"
Oct 31 01:00:48.881895 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:48.881866    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9tm4x\" (UniqueName: \"kubernetes.io/projected/5005c539-d22a-4c99-9235-e08fe509575f-kube-api-access-9tm4x\") pod \"e2e-api-b-31ae40-openshift-default-66bb6b84bc-8wrm6\" (UID: \"5005c539-d22a-4c99-9235-e08fe509575f\") " pod="openshift-ingress/e2e-api-b-31ae40-openshift-default-66bb6b84bc-8wrm6"
Oct 31 01:00:48.882190 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:48.882139    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5005c539-d22a-4c99-9235-e08fe509575f-istio-podinfo\") pod \"e2e-api-b-31ae40-openshift-default-66bb6b84bc-8wrm6\" (UID: \"5005c539-d22a-4c99-9235-e08fe509575f\") " pod="openshift-ingress/e2e-api-b-31ae40-openshift-default-66bb6b84bc-8wrm6"
Oct 31 01:00:48.923244 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:48.916697    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5005c539-d22a-4c99-9235-e08fe509575f-istio-token\") pod \"e2e-api-b-31ae40-openshift-default-66bb6b84bc-8wrm6\" (UID: \"5005c539-d22a-4c99-9235-e08fe509575f\") " pod="openshift-ingress/e2e-api-b-31ae40-openshift-default-66bb6b84bc-8wrm6"
Oct 31 01:00:49.013938 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:49.013837    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:00:49.170605 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:49.170572    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:00:49.205238 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:49.194781    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-31ae40-openshift-default-66bb6b84bc-8wrm6"
Oct 31 01:00:49.740960 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:49.740786    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-31ae40-openshift-default-66bb6b84bc-8wrm6"]
Oct 31 01:00:49.748521 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:49.748489    4284 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 01:00:49.748613 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:49.748556    4284 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 01:00:49.748613 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:49.748584    4284 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 01:00:50.070276 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:50.070177    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-31ae40-openshift-default-66bb6b84bc-8wrm6" event={"ID":"5005c539-d22a-4c99-9235-e08fe509575f","Type":"ContainerStarted","Data":"838a3937661be6950efee60ba7a12b02ce531b5b69885538b4a9d49fb3b4edc6"}
Oct 31 01:00:50.070276 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:50.070243    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-31ae40-openshift-default-66bb6b84bc-8wrm6" event={"ID":"5005c539-d22a-4c99-9235-e08fe509575f","Type":"ContainerStarted","Data":"55942112b1f7c6d42f731a5b145fc6aed978a5943401048a459a4169b9359601"}
Oct 31 01:00:50.215642 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:50.215609    4284 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-31ae40-openshift-default-66bb6b84bc-8wrm6"
Oct 31 01:00:50.217597 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:50.217561    4284 patch_prober.go:29] interesting pod/e2e-api-b-31ae40-openshift-default-66bb6b84bc-8wrm6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.128:15021/healthz/ready\": dial tcp 10.134.0.128:15021: connect: connection refused" start-of-body=
Oct 31 01:00:50.217742 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:50.217624    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-31ae40-openshift-default-66bb6b84bc-8wrm6" podUID="5005c539-d22a-4c99-9235-e08fe509575f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.128:15021/healthz/ready\": dial tcp 10.134.0.128:15021: connect: connection refused"
Oct 31 01:00:50.219123 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:50.219070    4284 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-31ae40-openshift-default-66bb6b84bc-8wrm6" podStartSLOduration=2.219050689 podStartE2EDuration="2.219050689s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 01:00:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 01:00:50.215579723 +0000 UTC m=+1964.360586639" watchObservedRunningTime="2026-10-31 01:00:50.219050689 +0000 UTC m=+1964.364057606"
Oct 31 01:00:51.229138 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:51.229105    4284 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-31ae40-openshift-default-66bb6b84bc-8wrm6"
Oct 31 01:00:52.084550 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:52.084460    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-31ae40-openshift-default-66bb6b84bc-8wrm6"
Oct 31 01:00:52.085675 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:52.085655    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-31ae40-openshift-default-66bb6b84bc-8wrm6"
Oct 31 01:00:53.178399 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:53.178349    4284 patch_prober.go:29] interesting pod/e2e-praxis-a-8c08e7-openshift-default-5d56d76c65-qnh6x container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.119:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 31 01:00:53.178912 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:53.178418    4284 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-praxis-a-8c08e7-openshift-default-5d56d76c65-qnh6x" podUID="16131784-3cb6-4d01-a27f-6ce599437f3c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.119:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 31 01:00:53.185412 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:53.185371    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:00:53.371591 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:53.371443    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:00:53.527966 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:53.527882    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:00:54.203623 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:54.202399    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:00:54.792537 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:54.792512    4284 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-55fbbbf9dc-j9vgx_5fa890d5-a2f9-495a-972b-06b8db658c88/payload-processing/0.log"
Oct 31 01:00:55.050122 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:55.050046    4284 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-55fbbbf9dc-j9vgx_5fa890d5-a2f9-495a-972b-06b8db658c88/payload-processing/0.log"
Oct 31 01:00:56.114191 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:56.114155    4284 generic.go:356] "Generic (PLEG): container finished" podID="16131784-3cb6-4d01-a27f-6ce599437f3c" containerID="bae8086e21fa68bb91f5084ddbb3811c359f00373ecd02892c41bad22b4625dc" exitCode=0
Oct 31 01:00:56.114728 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:56.114241    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-a-8c08e7-openshift-default-5d56d76c65-qnh6x" event={"ID":"16131784-3cb6-4d01-a27f-6ce599437f3c","Type":"ContainerDied","Data":"bae8086e21fa68bb91f5084ddbb3811c359f00373ecd02892c41bad22b4625dc"}
Oct 31 01:00:56.114728 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:56.114652    4284 scope.go:122] "RemoveContainer" containerID="bae8086e21fa68bb91f5084ddbb3811c359f00373ecd02892c41bad22b4625dc"
Oct 31 01:00:56.116130 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:56.116107    4284 generic.go:356] "Generic (PLEG): container finished" podID="2728e644-cebe-42fe-88ef-3d17a8a5817e" containerID="7d2e3989067ab9ebc481325739a6a6843c952e478c9b181b3e522b9af1458faf" exitCode=0
Oct 31 01:00:56.116245 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:56.116160    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-eb41d3-openshift-default-5bdbd7758c-lqfzl" event={"ID":"2728e644-cebe-42fe-88ef-3d17a8a5817e","Type":"ContainerDied","Data":"7d2e3989067ab9ebc481325739a6a6843c952e478c9b181b3e522b9af1458faf"}
Oct 31 01:00:56.116407 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:56.116382    4284 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 01:00:56.116459 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:56.116447    4284 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 01:00:56.116493 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:56.116478    4284 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 01:00:56.116529 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:56.116490    4284 scope.go:122] "RemoveContainer" containerID="7d2e3989067ab9ebc481325739a6a6843c952e478c9b181b3e522b9af1458faf"
Oct 31 01:00:56.117716 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:56.117689    4284 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 01:00:56.117817 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:56.117764    4284 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 01:00:56.117885 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:56.117813    4284 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 01:00:56.472543 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:56.472483    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:00:56.682743 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:56.682712    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:00:57.225250 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:57.224371    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-a-8c08e7-openshift-default-5d56d76c65-qnh6x" event={"ID":"16131784-3cb6-4d01-a27f-6ce599437f3c","Type":"ContainerStarted","Data":"94ac5923a1bf6d888ab9c36887d11735840e2818943bb261763441c8eac4b4db"}
Oct 31 01:00:57.250261 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:57.228339    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-eb41d3-openshift-default-5bdbd7758c-lqfzl" event={"ID":"2728e644-cebe-42fe-88ef-3d17a8a5817e","Type":"ContainerStarted","Data":"cc61d765850e8845baa04fa2e46fbed9ffd8a7dcb42302d0233f86713dcd2156"}
Oct 31 01:00:57.554656 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:57.554551    4284 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-eb41d3-openshift-default-5bdbd7758c-lqfzl"
Oct 31 01:00:57.556154 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:57.556126    4284 patch_prober.go:29] interesting pod/e2e-ipp-b-eb41d3-openshift-default-5bdbd7758c-lqfzl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.120:15021/healthz/ready\": dial tcp 10.134.0.120:15021: connect: connection refused" start-of-body=
Oct 31 01:00:57.556325 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:57.556177    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-eb41d3-openshift-default-5bdbd7758c-lqfzl" podUID="2728e644-cebe-42fe-88ef-3d17a8a5817e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.120:15021/healthz/ready\": dial tcp 10.134.0.120:15021: connect: connection refused"
Oct 31 01:00:57.652680 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:57.652643    4284 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-praxis-a-8c08e7-openshift-default-5d56d76c65-qnh6x"
Oct 31 01:00:57.653030 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:57.653008    4284 patch_prober.go:29] interesting pod/e2e-praxis-a-8c08e7-openshift-default-5d56d76c65-qnh6x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.119:15021/healthz/ready\": dial tcp 10.134.0.119:15021: connect: connection refused" start-of-body=
Oct 31 01:00:57.653079 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:57.653057    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-a-8c08e7-openshift-default-5d56d76c65-qnh6x" podUID="16131784-3cb6-4d01-a27f-6ce599437f3c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.119:15021/healthz/ready\": dial tcp 10.134.0.119:15021: connect: connection refused"
Oct 31 01:00:58.554384 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:58.554343    4284 patch_prober.go:29] interesting pod/e2e-ipp-b-eb41d3-openshift-default-5bdbd7758c-lqfzl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.120:15021/healthz/ready\": dial tcp 10.134.0.120:15021: connect: connection refused" start-of-body=
Oct 31 01:00:58.554907 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:58.554413    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-eb41d3-openshift-default-5bdbd7758c-lqfzl" podUID="2728e644-cebe-42fe-88ef-3d17a8a5817e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.120:15021/healthz/ready\": dial tcp 10.134.0.120:15021: connect: connection refused"
Oct 31 01:00:58.652437 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:58.652369    4284 patch_prober.go:29] interesting pod/e2e-praxis-a-8c08e7-openshift-default-5d56d76c65-qnh6x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.119:15021/healthz/ready\": dial tcp 10.134.0.119:15021: connect: connection refused" start-of-body=
Oct 31 01:00:58.652437 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:58.652432    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-a-8c08e7-openshift-default-5d56d76c65-qnh6x" podUID="16131784-3cb6-4d01-a27f-6ce599437f3c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.119:15021/healthz/ready\": dial tcp 10.134.0.119:15021: connect: connection refused"
Oct 31 01:00:59.554270 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:59.554199    4284 patch_prober.go:29] interesting pod/e2e-ipp-b-eb41d3-openshift-default-5bdbd7758c-lqfzl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.120:15021/healthz/ready\": dial tcp 10.134.0.120:15021: connect: connection refused" start-of-body=
Oct 31 01:00:59.554467 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:59.554284    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-eb41d3-openshift-default-5bdbd7758c-lqfzl" podUID="2728e644-cebe-42fe-88ef-3d17a8a5817e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.120:15021/healthz/ready\": dial tcp 10.134.0.120:15021: connect: connection refused"
Oct 31 01:00:59.652313 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:59.652278    4284 patch_prober.go:29] interesting pod/e2e-praxis-a-8c08e7-openshift-default-5d56d76c65-qnh6x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.119:15021/healthz/ready\": dial tcp 10.134.0.119:15021: connect: connection refused" start-of-body=
Oct 31 01:00:59.652490 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:59.652331    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-a-8c08e7-openshift-default-5d56d76c65-qnh6x" podUID="16131784-3cb6-4d01-a27f-6ce599437f3c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.119:15021/healthz/ready\": dial tcp 10.134.0.119:15021: connect: connection refused"
Oct 31 01:00:59.734519 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:59.734072    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:00:59.933234 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:00:59.932934    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:01:00.307292 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:01:00.306681    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-eb41d3-openshift-default-5bdbd7758c-lqfzl"
Oct 31 01:01:00.554460 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:01:00.554425    4284 patch_prober.go:29] interesting pod/e2e-ipp-b-eb41d3-openshift-default-5bdbd7758c-lqfzl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.120:15021/healthz/ready\": dial tcp 10.134.0.120:15021: connect: connection refused" start-of-body=
Oct 31 01:01:00.554871 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:01:00.554475    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-eb41d3-openshift-default-5bdbd7758c-lqfzl" podUID="2728e644-cebe-42fe-88ef-3d17a8a5817e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.120:15021/healthz/ready\": dial tcp 10.134.0.120:15021: connect: connection refused"
Oct 31 01:01:00.652446 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:01:00.652367    4284 patch_prober.go:29] interesting pod/e2e-praxis-a-8c08e7-openshift-default-5d56d76c65-qnh6x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.119:15021/healthz/ready\": dial tcp 10.134.0.119:15021: connect: connection refused" start-of-body=
Oct 31 01:01:00.652446 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:01:00.652422    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-a-8c08e7-openshift-default-5d56d76c65-qnh6x" podUID="16131784-3cb6-4d01-a27f-6ce599437f3c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.119:15021/healthz/ready\": dial tcp 10.134.0.119:15021: connect: connection refused"
Oct 31 01:01:01.554247 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:01:01.554167    4284 patch_prober.go:29] interesting pod/e2e-ipp-b-eb41d3-openshift-default-5bdbd7758c-lqfzl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.120:15021/healthz/ready\": dial tcp 10.134.0.120:15021: connect: connection refused" start-of-body=
Oct 31 01:01:01.554439 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:01:01.554257    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-eb41d3-openshift-default-5bdbd7758c-lqfzl" podUID="2728e644-cebe-42fe-88ef-3d17a8a5817e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.120:15021/healthz/ready\": dial tcp 10.134.0.120:15021: connect: connection refused"
Oct 31 01:01:01.653028 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:01:01.652978    4284 patch_prober.go:29] interesting pod/e2e-praxis-a-8c08e7-openshift-default-5d56d76c65-qnh6x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.119:15021/healthz/ready\": dial tcp 10.134.0.119:15021: connect: connection refused" start-of-body=
Oct 31 01:01:01.653641 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:01:01.653030    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-a-8c08e7-openshift-default-5d56d76c65-qnh6x" podUID="16131784-3cb6-4d01-a27f-6ce599437f3c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.119:15021/healthz/ready\": dial tcp 10.134.0.119:15021: connect: connection refused"
Oct 31 01:01:02.554647 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:01:02.554609    4284 patch_prober.go:29] interesting pod/e2e-ipp-b-eb41d3-openshift-default-5bdbd7758c-lqfzl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.120:15021/healthz/ready\": dial tcp 10.134.0.120:15021: connect: connection refused" start-of-body=
Oct 31 01:01:02.554877 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:01:02.554668    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-eb41d3-openshift-default-5bdbd7758c-lqfzl" podUID="2728e644-cebe-42fe-88ef-3d17a8a5817e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.120:15021/healthz/ready\": dial tcp 10.134.0.120:15021: connect: connection refused"
Oct 31 01:01:02.652640 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:01:02.652596    4284 patch_prober.go:29] interesting pod/e2e-praxis-a-8c08e7-openshift-default-5d56d76c65-qnh6x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.119:15021/healthz/ready\": dial tcp 10.134.0.119:15021: connect: connection refused" start-of-body=
Oct 31 01:01:02.652795 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:01:02.652675    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-a-8c08e7-openshift-default-5d56d76c65-qnh6x" podUID="16131784-3cb6-4d01-a27f-6ce599437f3c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.119:15021/healthz/ready\": dial tcp 10.134.0.119:15021: connect: connection refused"
Oct 31 01:01:03.558416 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:01:03.558387    4284 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-eb41d3-openshift-default-5bdbd7758c-lqfzl"
Oct 31 01:01:03.559490 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:01:03.559461    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-eb41d3-openshift-default-5bdbd7758c-lqfzl"
Oct 31 01:01:03.657136 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:01:03.657107    4284 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-praxis-a-8c08e7-openshift-default-5d56d76c65-qnh6x"
Oct 31 01:01:03.657546 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:01:03.657527    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-praxis-a-8c08e7-openshift-default-5d56d76c65-qnh6x"
Oct 31 01:01:03.658285 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:01:03.658269    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-praxis-a-8c08e7-openshift-default-5d56d76c65-qnh6x"
Oct 31 01:01:05.033938 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:01:05.033899    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:01:09.279544 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:01:09.279513    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:01:09.592005 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:01:09.591912    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-f6979f-78bd886898-gf67r"]
Oct 31 01:01:09.592242 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:01:09.592169    4284 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-b-f6979f-78bd886898-gf67r" podUID="8a1ca904-c09a-4b92-ab80-72241680c116" containerName="payload-processing" containerID="cri-o://e84b1f8636bc89199d26c5a57a57975dd4929c263b690963b9a4fa74f6fb8e31" gracePeriod=30
Oct 31 01:01:09.638817 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:01:09.638784    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-f6979f-7754cc869f-gmkpn"]
Oct 31 01:01:09.639048 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:01:09.639007    4284 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-f6979f-7754cc869f-gmkpn" podUID="b5d4b079-fbfb-4b9a-9027-47ab2b92aae5" containerName="payload-pre-processing" containerID="cri-o://f53da27474169195c494765752239be6973b48d29961d0f418ac32f2e34cbd89" gracePeriod=30
Oct 31 01:01:09.757183 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:01:09.757118    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:01:09.920645 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:01:09.920618    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-f6979f-78bd886898-gf67r"
Oct 31 01:01:09.940759 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:01:09.940724    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-f6979f-7754cc869f-gmkpn"
Oct 31 01:01:09.959594 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:01:09.959567    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:01:09.970838 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:01:09.970814    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/8a1ca904-c09a-4b92-ab80-72241680c116-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8a1ca904-c09a-4b92-ab80-72241680c116-plugins-config-volume\") pod \"8a1ca904-c09a-4b92-ab80-72241680c116\" (UID: \"8a1ca904-c09a-4b92-ab80-72241680c116\") "
Oct 31 01:01:09.971005 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:01:09.970913    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8a1ca904-c09a-4b92-ab80-72241680c116-kube-api-access-t85hx\" (UniqueName: \"kubernetes.io/projected/8a1ca904-c09a-4b92-ab80-72241680c116-kube-api-access-t85hx\") pod \"8a1ca904-c09a-4b92-ab80-72241680c116\" (UID: \"8a1ca904-c09a-4b92-ab80-72241680c116\") "
Oct 31 01:01:09.971288 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:01:09.971262    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8a1ca904-c09a-4b92-ab80-72241680c116-plugins-config-volume" pod "8a1ca904-c09a-4b92-ab80-72241680c116" (UID: "8a1ca904-c09a-4b92-ab80-72241680c116"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 01:01:09.971615 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:01:09.971596    4284 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8a1ca904-c09a-4b92-ab80-72241680c116-plugins-config-volume\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:01:09.973087 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:01:09.973070    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8a1ca904-c09a-4b92-ab80-72241680c116-kube-api-access-t85hx" pod "8a1ca904-c09a-4b92-ab80-72241680c116" (UID: "8a1ca904-c09a-4b92-ab80-72241680c116"). InnerVolumeSpecName "kube-api-access-t85hx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:01:10.090381 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:01:10.090318    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b5d4b079-fbfb-4b9a-9027-47ab2b92aae5-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b5d4b079-fbfb-4b9a-9027-47ab2b92aae5-plugins-config-volume\") pod \"b5d4b079-fbfb-4b9a-9027-47ab2b92aae5\" (UID: \"b5d4b079-fbfb-4b9a-9027-47ab2b92aae5\") "
Oct 31 01:01:10.126651 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:01:10.126618    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b5d4b079-fbfb-4b9a-9027-47ab2b92aae5-kube-api-access-zv57q\" (UniqueName: \"kubernetes.io/projected/b5d4b079-fbfb-4b9a-9027-47ab2b92aae5-kube-api-access-zv57q\") pod \"b5d4b079-fbfb-4b9a-9027-47ab2b92aae5\" (UID: \"b5d4b079-fbfb-4b9a-9027-47ab2b92aae5\") "
Oct 31 01:01:10.128192 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:01:10.128165    4284 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t85hx\" (UniqueName: \"kubernetes.io/projected/8a1ca904-c09a-4b92-ab80-72241680c116-kube-api-access-t85hx\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:01:10.145951 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:01:10.102331    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b5d4b079-fbfb-4b9a-9027-47ab2b92aae5-plugins-config-volume" pod "b5d4b079-fbfb-4b9a-9027-47ab2b92aae5" (UID: "b5d4b079-fbfb-4b9a-9027-47ab2b92aae5"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 01:01:10.181444 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:01:10.181409    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b5d4b079-fbfb-4b9a-9027-47ab2b92aae5-kube-api-access-zv57q" pod "b5d4b079-fbfb-4b9a-9027-47ab2b92aae5" (UID: "b5d4b079-fbfb-4b9a-9027-47ab2b92aae5"). InnerVolumeSpecName "kube-api-access-zv57q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:01:10.247663 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:01:10.247626    4284 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b5d4b079-fbfb-4b9a-9027-47ab2b92aae5-plugins-config-volume\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:01:10.247663 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:01:10.247662    4284 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zv57q\" (UniqueName: \"kubernetes.io/projected/b5d4b079-fbfb-4b9a-9027-47ab2b92aae5-kube-api-access-zv57q\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:01:10.320690 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:01:10.320658    4284 generic.go:356] "Generic (PLEG): container finished" podID="8a1ca904-c09a-4b92-ab80-72241680c116" containerID="e84b1f8636bc89199d26c5a57a57975dd4929c263b690963b9a4fa74f6fb8e31" exitCode=0
Oct 31 01:01:10.321182 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:01:10.320714    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-f6979f-78bd886898-gf67r"
Oct 31 01:01:10.321182 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:01:10.320742    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-f6979f-78bd886898-gf67r" event={"ID":"8a1ca904-c09a-4b92-ab80-72241680c116","Type":"ContainerDied","Data":"e84b1f8636bc89199d26c5a57a57975dd4929c263b690963b9a4fa74f6fb8e31"}
Oct 31 01:01:10.321182 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:01:10.320798    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-f6979f-78bd886898-gf67r" event={"ID":"8a1ca904-c09a-4b92-ab80-72241680c116","Type":"ContainerDied","Data":"cbfa6505e86269d313240ae65311054164c39a424391d7af138b5b20e98d05b5"}
Oct 31 01:01:10.321182 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:01:10.320830    4284 scope.go:122] "RemoveContainer" containerID="e84b1f8636bc89199d26c5a57a57975dd4929c263b690963b9a4fa74f6fb8e31"
Oct 31 01:01:10.322442 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:01:10.322404    4284 generic.go:356] "Generic (PLEG): container finished" podID="b5d4b079-fbfb-4b9a-9027-47ab2b92aae5" containerID="f53da27474169195c494765752239be6973b48d29961d0f418ac32f2e34cbd89" exitCode=0
Oct 31 01:01:10.322575 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:01:10.322463    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-f6979f-7754cc869f-gmkpn"
Oct 31 01:01:10.322575 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:01:10.322497    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-f6979f-7754cc869f-gmkpn" event={"ID":"b5d4b079-fbfb-4b9a-9027-47ab2b92aae5","Type":"ContainerDied","Data":"f53da27474169195c494765752239be6973b48d29961d0f418ac32f2e34cbd89"}
Oct 31 01:01:10.322575 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:01:10.322539    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-f6979f-7754cc869f-gmkpn" event={"ID":"b5d4b079-fbfb-4b9a-9027-47ab2b92aae5","Type":"ContainerDied","Data":"5afd021614af06d83f5525bd66bc186c91a27057722d4ad56de59ba440ac83f0"}
Oct 31 01:01:10.331375 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:01:10.331359    4284 scope.go:122] "RemoveContainer" containerID="e84b1f8636bc89199d26c5a57a57975dd4929c263b690963b9a4fa74f6fb8e31"
Oct 31 01:01:10.331668 ip-10-0-132-173 kubenswrapper[4284]: E1031 01:01:10.331648    4284 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e84b1f8636bc89199d26c5a57a57975dd4929c263b690963b9a4fa74f6fb8e31\": container with ID starting with e84b1f8636bc89199d26c5a57a57975dd4929c263b690963b9a4fa74f6fb8e31 not found: ID does not exist" containerID="e84b1f8636bc89199d26c5a57a57975dd4929c263b690963b9a4fa74f6fb8e31"
Oct 31 01:01:10.331754 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:01:10.331681    4284 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e84b1f8636bc89199d26c5a57a57975dd4929c263b690963b9a4fa74f6fb8e31"} err="failed to get container status \"e84b1f8636bc89199d26c5a57a57975dd4929c263b690963b9a4fa74f6fb8e31\": rpc error: code = NotFound desc = could not find container \"e84b1f8636bc89199d26c5a57a57975dd4929c263b690963b9a4fa74f6fb8e31\": container with ID starting with e84b1f8636bc89199d26c5a57a57975dd4929c263b690963b9a4fa74f6fb8e31 not found: ID does not exist"
Oct 31 01:01:10.331754 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:01:10.331704    4284 scope.go:122] "RemoveContainer" containerID="f53da27474169195c494765752239be6973b48d29961d0f418ac32f2e34cbd89"
Oct 31 01:01:10.340761 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:01:10.340728    4284 scope.go:122] "RemoveContainer" containerID="f53da27474169195c494765752239be6973b48d29961d0f418ac32f2e34cbd89"
Oct 31 01:01:10.341026 ip-10-0-132-173 kubenswrapper[4284]: E1031 01:01:10.341007    4284 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f53da27474169195c494765752239be6973b48d29961d0f418ac32f2e34cbd89\": container with ID starting with f53da27474169195c494765752239be6973b48d29961d0f418ac32f2e34cbd89 not found: ID does not exist" containerID="f53da27474169195c494765752239be6973b48d29961d0f418ac32f2e34cbd89"
Oct 31 01:01:10.341089 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:01:10.341033    4284 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f53da27474169195c494765752239be6973b48d29961d0f418ac32f2e34cbd89"} err="failed to get container status \"f53da27474169195c494765752239be6973b48d29961d0f418ac32f2e34cbd89\": rpc error: code = NotFound desc = could not find container \"f53da27474169195c494765752239be6973b48d29961d0f418ac32f2e34cbd89\": container with ID starting with f53da27474169195c494765752239be6973b48d29961d0f418ac32f2e34cbd89 not found: ID does not exist"
Oct 31 01:01:10.481371 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:01:10.481339    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-f6979f-78bd886898-gf67r"]
Oct 31 01:01:10.484534 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:01:10.484513    4284 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-f6979f-78bd886898-gf67r"]
Oct 31 01:01:10.588750 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:01:10.588711    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-f6979f-7754cc869f-gmkpn"]
Oct 31 01:01:10.592525 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:01:10.592500    4284 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-f6979f-7754cc869f-gmkpn"]
Oct 31 01:01:10.756632 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:01:10.756554    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:01:11.036761 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:01:11.036676    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:01:11.238108 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:01:11.238073    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:01:11.716868 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:01:11.716835    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:01:11.879097 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:01:11.879060    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:01:11.964232 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:01:11.964188    4284 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="8a1ca904-c09a-4b92-ab80-72241680c116" path="/var/lib/kubelet/pods/8a1ca904-c09a-4b92-ab80-72241680c116/volumes"
Oct 31 01:01:11.964686 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:01:11.964672    4284 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b5d4b079-fbfb-4b9a-9027-47ab2b92aae5" path="/var/lib/kubelet/pods/b5d4b079-fbfb-4b9a-9027-47ab2b92aae5/volumes"
Oct 31 01:01:17.014363 ip-10-0-132-173 kubenswrapper[4284]: E1031 01:01:17.014328    4284 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fd8sp" containerName="extractor"
Oct 31 01:01:17.759006 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:01:17.758958    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:01:18.098277 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:01:18.098179    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:01:18.318864 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:01:18.318823    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:01:18.442537 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:01:18.442508    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-f6979f-openshift-default-569f6f8dff-4k4gk"]
Oct 31 01:01:18.442740 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:01:18.442720    4284 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-b-f6979f-openshift-default-569f6f8dff-4k4gk" podUID="c0f54831-726b-41bd-adfb-445f8b9a39f9" containerName="istio-proxy" containerID="cri-o://055dc4b196bef410ceec47f2f5edeeb694a88f89b4bc5c445db16c0dc805c6b3" gracePeriod=30
Oct 31 01:01:18.561247 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:01:18.561133    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:01:20.085679 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:01:20.085642    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:01:20.713946 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:01:20.713912    4284 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-praxis-a-7ea4b0-openshift-default-65c5776677-nw4cq"]
Oct 31 01:01:20.714501 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:01:20.714487    4284 cpu_manager.go:414] "containerMap: removing container" podUID="6af0f6fa-a965-4083-af26-8338eeb6e796" containerName="cleanup"
Oct 31 01:01:20.714556 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:01:20.714503    4284 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6af0f6fa-a965-4083-af26-8338eeb6e796" containerName="cleanup"
Oct 31 01:01:20.714556 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:01:20.714532    4284 cpu_manager.go:414] "containerMap: removing container" podUID="8a1ca904-c09a-4b92-ab80-72241680c116" containerName="payload-processing"
Oct 31 01:01:20.714556 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:01:20.714537    4284 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8a1ca904-c09a-4b92-ab80-72241680c116" containerName="payload-processing"
Oct 31 01:01:20.714556 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:01:20.714550    4284 cpu_manager.go:414] "containerMap: removing container" podUID="b5d4b079-fbfb-4b9a-9027-47ab2b92aae5" containerName="payload-pre-processing"
Oct 31 01:01:20.714556 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:01:20.714556    4284 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b5d4b079-fbfb-4b9a-9027-47ab2b92aae5" containerName="payload-pre-processing"
Oct 31 01:01:20.714733 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:01:20.714649    4284 memory_manager.go:365] "RemoveStaleState removing state" podUID="b5d4b079-fbfb-4b9a-9027-47ab2b92aae5" containerName="payload-pre-processing"
Oct 31 01:01:20.714733 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:01:20.714656    4284 memory_manager.go:365] "RemoveStaleState removing state" podUID="8a1ca904-c09a-4b92-ab80-72241680c116" containerName="payload-processing"
Oct 31 01:01:20.718498 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:01:20.718478    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-praxis-a-7ea4b0-openshift-default-65c5776677-nw4cq"
Oct 31 01:01:20.768835 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:01:20.768800    4284 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-praxis-a-7ea4b0-openshift-default-dockercfg-8mdln\""
Oct 31 01:01:20.799771 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:01:20.799731    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:01:20.819741 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:01:20.819709    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-praxis-a-7ea4b0-openshift-default-65c5776677-nw4cq"]
Oct 31 01:01:20.819974 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:01:20.819953    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4b7ffc18-b208-499f-afed-cb71c96259e4-workload-certs\") pod \"e2e-praxis-a-7ea4b0-openshift-default-65c5776677-nw4cq\" (UID: \"4b7ffc18-b208-499f-afed-cb71c96259e4\") " pod="openshift-ingress/e2e-praxis-a-7ea4b0-openshift-default-65c5776677-nw4cq"
Oct 31 01:01:20.820054 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:01:20.819986    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4b7ffc18-b208-499f-afed-cb71c96259e4-istio-envoy\") pod \"e2e-praxis-a-7ea4b0-openshift-default-65c5776677-nw4cq\" (UID: \"4b7ffc18-b208-499f-afed-cb71c96259e4\") " pod="openshift-ingress/e2e-praxis-a-7ea4b0-openshift-default-65c5776677-nw4cq"
Oct 31 01:01:20.820054 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:01:20.820030    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4b7ffc18-b208-499f-afed-cb71c96259e4-istio-token\") pod \"e2e-praxis-a-7ea4b0-openshift-default-65c5776677-nw4cq\" (UID: \"4b7ffc18-b208-499f-afed-cb71c96259e4\") " pod="openshift-ingress/e2e-praxis-a-7ea4b0-openshift-default-65c5776677-nw4cq"
Oct 31 01:01:20.820201 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:01:20.820186    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6vkpb\" (UniqueName: \"kubernetes.io/projected/4b7ffc18-b208-499f-afed-cb71c96259e4-kube-api-access-6vkpb\") pod \"e2e-praxis-a-7ea4b0-openshift-default-65c5776677-nw4cq\" (UID: \"4b7ffc18-b208-499f-afed-cb71c96259e4\") " pod="openshift-ingress/e2e-praxis-a-7ea4b0-openshift-default-65c5776677-nw4cq"
Oct 31 01:01:20.820299 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:01:20.820240    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4b7ffc18-b208-499f-afed-cb71c96259e4-credential-socket\") pod \"e2e-praxis-a-7ea4b0-openshift-default-65c5776677-nw4cq\" (UID: \"4b7ffc18-b208-499f-afed-cb71c96259e4\") " pod="openshift-ingress/e2e-praxis-a-7ea4b0-openshift-default-65c5776677-nw4cq"
Oct 31 01:01:20.820299 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:01:20.820278    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4b7ffc18-b208-499f-afed-cb71c96259e4-istio-podinfo\") pod \"e2e-praxis-a-7ea4b0-openshift-default-65c5776677-nw4cq\" (UID: \"4b7ffc18-b208-499f-afed-cb71c96259e4\") " pod="openshift-ingress/e2e-praxis-a-7ea4b0-openshift-default-65c5776677-nw4cq"
Oct 31 01:01:20.820408 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:01:20.820321    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4b7ffc18-b208-499f-afed-cb71c96259e4-istio-data\") pod \"e2e-praxis-a-7ea4b0-openshift-default-65c5776677-nw4cq\" (UID: \"4b7ffc18-b208-499f-afed-cb71c96259e4\") " pod="openshift-ingress/e2e-praxis-a-7ea4b0-openshift-default-65c5776677-nw4cq"
Oct 31 01:01:20.820408 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:01:20.820370    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4b7ffc18-b208-499f-afed-cb71c96259e4-istiod-ca-cert\") pod \"e2e-praxis-a-7ea4b0-openshift-default-65c5776677-nw4cq\" (UID: \"4b7ffc18-b208-499f-afed-cb71c96259e4\") " pod="openshift-ingress/e2e-praxis-a-7ea4b0-openshift-default-65c5776677-nw4cq"
Oct 31 01:01:20.820519 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:01:20.820414    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4b7ffc18-b208-499f-afed-cb71c96259e4-workload-socket\") pod \"e2e-praxis-a-7ea4b0-openshift-default-65c5776677-nw4cq\" (UID: \"4b7ffc18-b208-499f-afed-cb71c96259e4\") " pod="openshift-ingress/e2e-praxis-a-7ea4b0-openshift-default-65c5776677-nw4cq"
Oct 31 01:01:20.957255 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:01:20.957167    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6vkpb\" (UniqueName: \"kubernetes.io/projected/4b7ffc18-b208-499f-afed-cb71c96259e4-kube-api-access-6vkpb\") pod \"e2e-praxis-a-7ea4b0-openshift-default-65c5776677-nw4cq\" (UID: \"4b7ffc18-b208-499f-afed-cb71c96259e4\") " pod="openshift-ingress/e2e-praxis-a-7ea4b0-openshift-default-65c5776677-nw4cq"
Oct 31 01:01:20.957255 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:01:20.957261    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4b7ffc18-b208-499f-afed-cb71c96259e4-credential-socket\") pod \"e2e-praxis-a-7ea4b0-openshift-default-65c5776677-nw4cq\" (UID: \"4b7ffc18-b208-499f-afed-cb71c96259e4\") " pod="openshift-ingress/e2e-praxis-a-7ea4b0-openshift-default-65c5776677-nw4cq"
Oct 31 01:01:20.957519 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:01:20.957312    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4b7ffc18-b208-499f-afed-cb71c96259e4-istio-podinfo\") pod \"e2e-praxis-a-7ea4b0-openshift-default-65c5776677-nw4cq\" (UID: \"4b7ffc18-b208-499f-afed-cb71c96259e4\") " pod="openshift-ingress/e2e-praxis-a-7ea4b0-openshift-default-65c5776677-nw4cq"
Oct 31 01:01:20.957519 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:01:20.957378    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4b7ffc18-b208-499f-afed-cb71c96259e4-istio-data\") pod \"e2e-praxis-a-7ea4b0-openshift-default-65c5776677-nw4cq\" (UID: \"4b7ffc18-b208-499f-afed-cb71c96259e4\") " pod="openshift-ingress/e2e-praxis-a-7ea4b0-openshift-default-65c5776677-nw4cq"
Oct 31 01:01:20.957519 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:01:20.957446    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4b7ffc18-b208-499f-afed-cb71c96259e4-istiod-ca-cert\") pod \"e2e-praxis-a-7ea4b0-openshift-default-65c5776677-nw4cq\" (UID: \"4b7ffc18-b208-499f-afed-cb71c96259e4\") " pod="openshift-ingress/e2e-praxis-a-7ea4b0-openshift-default-65c5776677-nw4cq"
Oct 31 01:01:20.957519 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:01:20.957506    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4b7ffc18-b208-499f-afed-cb71c96259e4-workload-socket\") pod \"e2e-praxis-a-7ea4b0-openshift-default-65c5776677-nw4cq\" (UID: \"4b7ffc18-b208-499f-afed-cb71c96259e4\") " pod="openshift-ingress/e2e-praxis-a-7ea4b0-openshift-default-65c5776677-nw4cq"
Oct 31 01:01:20.957735 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:01:20.957578    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4b7ffc18-b208-499f-afed-cb71c96259e4-workload-certs\") pod \"e2e-praxis-a-7ea4b0-openshift-default-65c5776677-nw4cq\" (UID: \"4b7ffc18-b208-499f-afed-cb71c96259e4\") " pod="openshift-ingress/e2e-praxis-a-7ea4b0-openshift-default-65c5776677-nw4cq"
Oct 31 01:01:20.957735 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:01:20.957600    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4b7ffc18-b208-499f-afed-cb71c96259e4-istio-envoy\") pod \"e2e-praxis-a-7ea4b0-openshift-default-65c5776677-nw4cq\" (UID: \"4b7ffc18-b208-499f-afed-cb71c96259e4\") " pod="openshift-ingress/e2e-praxis-a-7ea4b0-openshift-default-65c5776677-nw4cq"
Oct 31 01:01:20.957735 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:01:20.957643    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4b7ffc18-b208-499f-afed-cb71c96259e4-istio-token\") pod \"e2e-praxis-a-7ea4b0-openshift-default-65c5776677-nw4cq\" (UID: \"4b7ffc18-b208-499f-afed-cb71c96259e4\") " pod="openshift-ingress/e2e-praxis-a-7ea4b0-openshift-default-65c5776677-nw4cq"
Oct 31 01:01:20.962027 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:01:20.960870    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4b7ffc18-b208-499f-afed-cb71c96259e4-istiod-ca-cert\") pod \"e2e-praxis-a-7ea4b0-openshift-default-65c5776677-nw4cq\" (UID: \"4b7ffc18-b208-499f-afed-cb71c96259e4\") " pod="openshift-ingress/e2e-praxis-a-7ea4b0-openshift-default-65c5776677-nw4cq"
Oct 31 01:01:20.962027 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:01:20.961625    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4b7ffc18-b208-499f-afed-cb71c96259e4-credential-socket\") pod \"e2e-praxis-a-7ea4b0-openshift-default-65c5776677-nw4cq\" (UID: \"4b7ffc18-b208-499f-afed-cb71c96259e4\") " pod="openshift-ingress/e2e-praxis-a-7ea4b0-openshift-default-65c5776677-nw4cq"
Oct 31 01:01:20.962027 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:01:20.961882    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4b7ffc18-b208-499f-afed-cb71c96259e4-workload-socket\") pod \"e2e-praxis-a-7ea4b0-openshift-default-65c5776677-nw4cq\" (UID: \"4b7ffc18-b208-499f-afed-cb71c96259e4\") " pod="openshift-ingress/e2e-praxis-a-7ea4b0-openshift-default-65c5776677-nw4cq"
Oct 31 01:01:20.962381 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:01:20.962105    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4b7ffc18-b208-499f-afed-cb71c96259e4-workload-certs\") pod \"e2e-praxis-a-7ea4b0-openshift-default-65c5776677-nw4cq\" (UID: \"4b7ffc18-b208-499f-afed-cb71c96259e4\") " pod="openshift-ingress/e2e-praxis-a-7ea4b0-openshift-default-65c5776677-nw4cq"
Oct 31 01:01:20.969071 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:01:20.965719    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4b7ffc18-b208-499f-afed-cb71c96259e4-istio-data\") pod \"e2e-praxis-a-7ea4b0-openshift-default-65c5776677-nw4cq\" (UID: \"4b7ffc18-b208-499f-afed-cb71c96259e4\") " pod="openshift-ingress/e2e-praxis-a-7ea4b0-openshift-default-65c5776677-nw4cq"
Oct 31 01:01:20.969071 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:01:20.967694    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4b7ffc18-b208-499f-afed-cb71c96259e4-istio-envoy\") pod \"e2e-praxis-a-7ea4b0-openshift-default-65c5776677-nw4cq\" (UID: \"4b7ffc18-b208-499f-afed-cb71c96259e4\") " pod="openshift-ingress/e2e-praxis-a-7ea4b0-openshift-default-65c5776677-nw4cq"
Oct 31 01:01:20.970187 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:01:20.970143    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4b7ffc18-b208-499f-afed-cb71c96259e4-istio-podinfo\") pod \"e2e-praxis-a-7ea4b0-openshift-default-65c5776677-nw4cq\" (UID: \"4b7ffc18-b208-499f-afed-cb71c96259e4\") " pod="openshift-ingress/e2e-praxis-a-7ea4b0-openshift-default-65c5776677-nw4cq"
Oct 31 01:01:21.016883 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:01:21.016852    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4b7ffc18-b208-499f-afed-cb71c96259e4-istio-token\") pod \"e2e-praxis-a-7ea4b0-openshift-default-65c5776677-nw4cq\" (UID: \"4b7ffc18-b208-499f-afed-cb71c96259e4\") " pod="openshift-ingress/e2e-praxis-a-7ea4b0-openshift-default-65c5776677-nw4cq"
Oct 31 01:01:21.017177 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:01:21.017152    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6vkpb\" (UniqueName: \"kubernetes.io/projected/4b7ffc18-b208-499f-afed-cb71c96259e4-kube-api-access-6vkpb\") pod \"e2e-praxis-a-7ea4b0-openshift-default-65c5776677-nw4cq\" (UID: \"4b7ffc18-b208-499f-afed-cb71c96259e4\") " pod="openshift-ingress/e2e-praxis-a-7ea4b0-openshift-default-65c5776677-nw4cq"
Oct 31 01:01:21.031669 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:01:21.031644    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-praxis-a-7ea4b0-openshift-default-65c5776677-nw4cq"
Oct 31 01:01:21.300920 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:01:21.300787    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:01:21.464062 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:01:21.464035    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-praxis-a-7ea4b0-openshift-default-65c5776677-nw4cq"]
Oct 31 01:01:21.465194 ip-10-0-132-173 kubenswrapper[4284]: W1031 01:01:21.465154    4284 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4b7ffc18_b208_499f_afed_cb71c96259e4.slice/crio-6065afcc90988b94cd819ccf9e0bf6c32830a041445b5ca782c08be38cb99e95 WatchSource:0}: Error finding container 6065afcc90988b94cd819ccf9e0bf6c32830a041445b5ca782c08be38cb99e95: Status 404 returned error can't find the container with id 6065afcc90988b94cd819ccf9e0bf6c32830a041445b5ca782c08be38cb99e95
Oct 31 01:01:21.467898 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:01:21.467866    4284 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 01:01:21.467980 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:01:21.467931    4284 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 01:01:21.467980 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:01:21.467961    4284 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 01:01:22.448651 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:01:22.448612    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-a-7ea4b0-openshift-default-65c5776677-nw4cq" event={"ID":"4b7ffc18-b208-499f-afed-cb71c96259e4","Type":"ContainerStarted","Data":"1acc08efa1ca567c1520e2bce6f783ec39e030c89612915e86fa5ddc750b6eea"}
Oct 31 01:01:22.449070 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:01:22.448674    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-a-7ea4b0-openshift-default-65c5776677-nw4cq" event={"ID":"4b7ffc18-b208-499f-afed-cb71c96259e4","Type":"ContainerStarted","Data":"6065afcc90988b94cd819ccf9e0bf6c32830a041445b5ca782c08be38cb99e95"}
Oct 31 01:01:22.570579 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:01:22.570436    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:01:23.032255 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:01:23.032206    4284 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-praxis-a-7ea4b0-openshift-default-65c5776677-nw4cq"
Oct 31 01:01:23.037610 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:01:23.037585    4284 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-praxis-a-7ea4b0-openshift-default-65c5776677-nw4cq"
Oct 31 01:01:23.155003 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:01:23.154954    4284 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-praxis-a-7ea4b0-openshift-default-65c5776677-nw4cq" podStartSLOduration=3.154939323 podStartE2EDuration="3.154939323s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 01:01:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 01:01:22.592471437 +0000 UTC m=+1996.737478394" watchObservedRunningTime="2026-10-31 01:01:23.154939323 +0000 UTC m=+1997.299946238"
Oct 31 01:01:23.454356 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:01:23.454331    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-praxis-a-7ea4b0-openshift-default-65c5776677-nw4cq"
Oct 31 01:01:23.455292 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:01:23.455274    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-praxis-a-7ea4b0-openshift-default-65c5776677-nw4cq"
Oct 31 01:01:23.725316 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:01:23.725292    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-f6979f-openshift-default-569f6f8dff-4k4gk"
Oct 31 01:01:23.734323 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:01:23.734303    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c0f54831-726b-41bd-adfb-445f8b9a39f9-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c0f54831-726b-41bd-adfb-445f8b9a39f9-credential-socket\") pod \"c0f54831-726b-41bd-adfb-445f8b9a39f9\" (UID: \"c0f54831-726b-41bd-adfb-445f8b9a39f9\") "
Oct 31 01:01:23.734444 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:01:23.734333    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c0f54831-726b-41bd-adfb-445f8b9a39f9-kube-api-access-q57zj\" (UniqueName: \"kubernetes.io/projected/c0f54831-726b-41bd-adfb-445f8b9a39f9-kube-api-access-q57zj\") pod \"c0f54831-726b-41bd-adfb-445f8b9a39f9\" (UID: \"c0f54831-726b-41bd-adfb-445f8b9a39f9\") "
Oct 31 01:01:23.734444 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:01:23.734421    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c0f54831-726b-41bd-adfb-445f8b9a39f9-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c0f54831-726b-41bd-adfb-445f8b9a39f9-istiod-ca-cert\") pod \"c0f54831-726b-41bd-adfb-445f8b9a39f9\" (UID: \"c0f54831-726b-41bd-adfb-445f8b9a39f9\") "
Oct 31 01:01:23.734557 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:01:23.734520    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c0f54831-726b-41bd-adfb-445f8b9a39f9-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c0f54831-726b-41bd-adfb-445f8b9a39f9-istio-data\") pod \"c0f54831-726b-41bd-adfb-445f8b9a39f9\" (UID: \"c0f54831-726b-41bd-adfb-445f8b9a39f9\") "
Oct 31 01:01:23.734609 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:01:23.734564    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c0f54831-726b-41bd-adfb-445f8b9a39f9-istio-token\" (UniqueName: \"kubernetes.io/projected/c0f54831-726b-41bd-adfb-445f8b9a39f9-istio-token\") pod \"c0f54831-726b-41bd-adfb-445f8b9a39f9\" (UID: \"c0f54831-726b-41bd-adfb-445f8b9a39f9\") "
Oct 31 01:01:23.734661 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:01:23.734575    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c0f54831-726b-41bd-adfb-445f8b9a39f9-credential-socket" pod "c0f54831-726b-41bd-adfb-445f8b9a39f9" (UID: "c0f54831-726b-41bd-adfb-445f8b9a39f9"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:01:23.734661 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:01:23.734636    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c0f54831-726b-41bd-adfb-445f8b9a39f9-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c0f54831-726b-41bd-adfb-445f8b9a39f9-workload-socket\") pod \"c0f54831-726b-41bd-adfb-445f8b9a39f9\" (UID: \"c0f54831-726b-41bd-adfb-445f8b9a39f9\") "
Oct 31 01:01:23.734770 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:01:23.734702    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/c0f54831-726b-41bd-adfb-445f8b9a39f9-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c0f54831-726b-41bd-adfb-445f8b9a39f9-istio-podinfo\") pod \"c0f54831-726b-41bd-adfb-445f8b9a39f9\" (UID: \"c0f54831-726b-41bd-adfb-445f8b9a39f9\") "
Oct 31 01:01:23.734770 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:01:23.734708    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c0f54831-726b-41bd-adfb-445f8b9a39f9-istiod-ca-cert" pod "c0f54831-726b-41bd-adfb-445f8b9a39f9" (UID: "c0f54831-726b-41bd-adfb-445f8b9a39f9"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 01:01:23.734770 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:01:23.734733    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c0f54831-726b-41bd-adfb-445f8b9a39f9-istio-data" pod "c0f54831-726b-41bd-adfb-445f8b9a39f9" (UID: "c0f54831-726b-41bd-adfb-445f8b9a39f9"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:01:23.734770 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:01:23.734763    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c0f54831-726b-41bd-adfb-445f8b9a39f9-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c0f54831-726b-41bd-adfb-445f8b9a39f9-workload-certs\") pod \"c0f54831-726b-41bd-adfb-445f8b9a39f9\" (UID: \"c0f54831-726b-41bd-adfb-445f8b9a39f9\") "
Oct 31 01:01:23.734951 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:01:23.734795    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c0f54831-726b-41bd-adfb-445f8b9a39f9-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c0f54831-726b-41bd-adfb-445f8b9a39f9-istio-envoy\") pod \"c0f54831-726b-41bd-adfb-445f8b9a39f9\" (UID: \"c0f54831-726b-41bd-adfb-445f8b9a39f9\") "
Oct 31 01:01:23.735011 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:01:23.734962    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c0f54831-726b-41bd-adfb-445f8b9a39f9-workload-socket" pod "c0f54831-726b-41bd-adfb-445f8b9a39f9" (UID: "c0f54831-726b-41bd-adfb-445f8b9a39f9"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:01:23.735011 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:01:23.734972    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c0f54831-726b-41bd-adfb-445f8b9a39f9-workload-certs" pod "c0f54831-726b-41bd-adfb-445f8b9a39f9" (UID: "c0f54831-726b-41bd-adfb-445f8b9a39f9"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:01:23.735294 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:01:23.735277    4284 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c0f54831-726b-41bd-adfb-445f8b9a39f9-credential-socket\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:01:23.735294 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:01:23.735294    4284 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c0f54831-726b-41bd-adfb-445f8b9a39f9-istiod-ca-cert\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:01:23.735623 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:01:23.735304    4284 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c0f54831-726b-41bd-adfb-445f8b9a39f9-istio-data\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:01:23.735623 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:01:23.735315    4284 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c0f54831-726b-41bd-adfb-445f8b9a39f9-workload-socket\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:01:23.735623 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:01:23.735324    4284 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c0f54831-726b-41bd-adfb-445f8b9a39f9-workload-certs\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:01:23.736787 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:01:23.736764    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c0f54831-726b-41bd-adfb-445f8b9a39f9-kube-api-access-q57zj" pod "c0f54831-726b-41bd-adfb-445f8b9a39f9" (UID: "c0f54831-726b-41bd-adfb-445f8b9a39f9"). InnerVolumeSpecName "kube-api-access-q57zj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:01:23.736879 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:01:23.736795    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c0f54831-726b-41bd-adfb-445f8b9a39f9-istio-token" pod "c0f54831-726b-41bd-adfb-445f8b9a39f9" (UID: "c0f54831-726b-41bd-adfb-445f8b9a39f9"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:01:23.737015 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:01:23.736991    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c0f54831-726b-41bd-adfb-445f8b9a39f9-istio-envoy" pod "c0f54831-726b-41bd-adfb-445f8b9a39f9" (UID: "c0f54831-726b-41bd-adfb-445f8b9a39f9"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:01:23.737057 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:01:23.737042    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c0f54831-726b-41bd-adfb-445f8b9a39f9-istio-podinfo" pod "c0f54831-726b-41bd-adfb-445f8b9a39f9" (UID: "c0f54831-726b-41bd-adfb-445f8b9a39f9"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 01:01:23.836405 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:01:23.836363    4284 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c0f54831-726b-41bd-adfb-445f8b9a39f9-istio-envoy\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:01:23.836405 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:01:23.836397    4284 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q57zj\" (UniqueName: \"kubernetes.io/projected/c0f54831-726b-41bd-adfb-445f8b9a39f9-kube-api-access-q57zj\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:01:23.836405 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:01:23.836413    4284 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c0f54831-726b-41bd-adfb-445f8b9a39f9-istio-token\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:01:23.836644 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:01:23.836425    4284 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c0f54831-726b-41bd-adfb-445f8b9a39f9-istio-podinfo\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:01:24.461824 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:01:24.461784    4284 generic.go:356] "Generic (PLEG): container finished" podID="c0f54831-726b-41bd-adfb-445f8b9a39f9" containerID="055dc4b196bef410ceec47f2f5edeeb694a88f89b4bc5c445db16c0dc805c6b3" exitCode=0
Oct 31 01:01:24.462364 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:01:24.461841    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-f6979f-openshift-default-569f6f8dff-4k4gk"
Oct 31 01:01:24.462364 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:01:24.461870    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-f6979f-openshift-default-569f6f8dff-4k4gk" event={"ID":"c0f54831-726b-41bd-adfb-445f8b9a39f9","Type":"ContainerDied","Data":"055dc4b196bef410ceec47f2f5edeeb694a88f89b4bc5c445db16c0dc805c6b3"}
Oct 31 01:01:24.462364 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:01:24.461935    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-f6979f-openshift-default-569f6f8dff-4k4gk" event={"ID":"c0f54831-726b-41bd-adfb-445f8b9a39f9","Type":"ContainerDied","Data":"57aed69861500ab272cabd7fafdd4d08e5acda41a95cf2aa07ddb1b2a0a82ca0"}
Oct 31 01:01:24.462364 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:01:24.461968    4284 scope.go:122] "RemoveContainer" containerID="055dc4b196bef410ceec47f2f5edeeb694a88f89b4bc5c445db16c0dc805c6b3"
Oct 31 01:01:24.471658 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:01:24.471635    4284 scope.go:122] "RemoveContainer" containerID="fbcb1a47b41849c5e466e4e7fbbe5368ac5730a6bad7c113a530a849b67c6762"
Oct 31 01:01:24.481108 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:01:24.481086    4284 scope.go:122] "RemoveContainer" containerID="055dc4b196bef410ceec47f2f5edeeb694a88f89b4bc5c445db16c0dc805c6b3"
Oct 31 01:01:24.481465 ip-10-0-132-173 kubenswrapper[4284]: E1031 01:01:24.481443    4284 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"055dc4b196bef410ceec47f2f5edeeb694a88f89b4bc5c445db16c0dc805c6b3\": container with ID starting with 055dc4b196bef410ceec47f2f5edeeb694a88f89b4bc5c445db16c0dc805c6b3 not found: ID does not exist" containerID="055dc4b196bef410ceec47f2f5edeeb694a88f89b4bc5c445db16c0dc805c6b3"
Oct 31 01:01:24.481563 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:01:24.481474    4284 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"055dc4b196bef410ceec47f2f5edeeb694a88f89b4bc5c445db16c0dc805c6b3"} err="failed to get container status \"055dc4b196bef410ceec47f2f5edeeb694a88f89b4bc5c445db16c0dc805c6b3\": rpc error: code = NotFound desc = could not find container \"055dc4b196bef410ceec47f2f5edeeb694a88f89b4bc5c445db16c0dc805c6b3\": container with ID starting with 055dc4b196bef410ceec47f2f5edeeb694a88f89b4bc5c445db16c0dc805c6b3 not found: ID does not exist"
Oct 31 01:01:24.481563 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:01:24.481500    4284 scope.go:122] "RemoveContainer" containerID="fbcb1a47b41849c5e466e4e7fbbe5368ac5730a6bad7c113a530a849b67c6762"
Oct 31 01:01:24.481784 ip-10-0-132-173 kubenswrapper[4284]: E1031 01:01:24.481756    4284 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fbcb1a47b41849c5e466e4e7fbbe5368ac5730a6bad7c113a530a849b67c6762\": container with ID starting with fbcb1a47b41849c5e466e4e7fbbe5368ac5730a6bad7c113a530a849b67c6762 not found: ID does not exist" containerID="fbcb1a47b41849c5e466e4e7fbbe5368ac5730a6bad7c113a530a849b67c6762"
Oct 31 01:01:24.481841 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:01:24.481790    4284 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fbcb1a47b41849c5e466e4e7fbbe5368ac5730a6bad7c113a530a849b67c6762"} err="failed to get container status \"fbcb1a47b41849c5e466e4e7fbbe5368ac5730a6bad7c113a530a849b67c6762\": rpc error: code = NotFound desc = could not find container \"fbcb1a47b41849c5e466e4e7fbbe5368ac5730a6bad7c113a530a849b67c6762\": container with ID starting with fbcb1a47b41849c5e466e4e7fbbe5368ac5730a6bad7c113a530a849b67c6762 not found: ID does not exist"
Oct 31 01:01:24.621596 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:01:24.621561    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-f6979f-openshift-default-569f6f8dff-4k4gk"]
Oct 31 01:01:24.669148 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:01:24.669118    4284 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-b-f6979f-openshift-default-569f6f8dff-4k4gk"]
Oct 31 01:01:25.556513 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:01:25.556478    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:01:25.750093 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:01:25.750046    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:01:25.963601 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:01:25.963573    4284 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c0f54831-726b-41bd-adfb-445f8b9a39f9" path="/var/lib/kubelet/pods/c0f54831-726b-41bd-adfb-445f8b9a39f9/volumes"
Oct 31 01:01:27.780563 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:01:27.780516    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:01:32.614081 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:01:32.614041    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:01:32.841669 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:01:32.841630    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:01:33.008431 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:01:33.008393    4284 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-praxis-a-7ea4b0-68cb74f9-rlhnf"]
Oct 31 01:01:33.009033 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:01:33.009018    4284 cpu_manager.go:414] "containerMap: removing container" podUID="c0f54831-726b-41bd-adfb-445f8b9a39f9" containerName="istio-proxy"
Oct 31 01:01:33.009092 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:01:33.009035    4284 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c0f54831-726b-41bd-adfb-445f8b9a39f9" containerName="istio-proxy"
Oct 31 01:01:33.009171 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:01:33.009161    4284 memory_manager.go:365] "RemoveStaleState removing state" podUID="c0f54831-726b-41bd-adfb-445f8b9a39f9" containerName="istio-proxy"
Oct 31 01:01:33.012059 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:01:33.012043    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-7ea4b0-68cb74f9-rlhnf"
Oct 31 01:01:33.063457 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:01:33.063417    4284 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-praxis-a-7ea4b0\""
Oct 31 01:01:33.067419 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:01:33.067392    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-praxis-a-7ea4b0-68cb74f9-rlhnf"]
Oct 31 01:01:33.119246 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:01:33.117563    4284 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-praxis-a-7ea4b0-dockercfg-p88fd\""
Oct 31 01:01:33.136462 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:01:33.119625    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x9hzm\" (UniqueName: \"kubernetes.io/projected/869411ac-8712-4f5c-8634-7b4cc2ab37f9-kube-api-access-x9hzm\") pod \"payload-pre-processing-e2e-praxis-a-7ea4b0-68cb74f9-rlhnf\" (UID: \"869411ac-8712-4f5c-8634-7b4cc2ab37f9\") " pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-7ea4b0-68cb74f9-rlhnf"
Oct 31 01:01:33.144976 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:01:33.139175    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/869411ac-8712-4f5c-8634-7b4cc2ab37f9-plugins-config-volume\") pod \"payload-pre-processing-e2e-praxis-a-7ea4b0-68cb74f9-rlhnf\" (UID: \"869411ac-8712-4f5c-8634-7b4cc2ab37f9\") " pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-7ea4b0-68cb74f9-rlhnf"
Oct 31 01:01:33.238708 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:01:33.238559    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:01:33.257005 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:01:33.250976    4284 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-praxis-a-7ea4b0-75f8f8786f-6cljj"]
Oct 31 01:01:33.260601 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:01:33.253376    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/869411ac-8712-4f5c-8634-7b4cc2ab37f9-plugins-config-volume\") pod \"payload-pre-processing-e2e-praxis-a-7ea4b0-68cb74f9-rlhnf\" (UID: \"869411ac-8712-4f5c-8634-7b4cc2ab37f9\") " pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-7ea4b0-68cb74f9-rlhnf"
Oct 31 01:01:33.261083 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:01:33.261043    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x9hzm\" (UniqueName: \"kubernetes.io/projected/869411ac-8712-4f5c-8634-7b4cc2ab37f9-kube-api-access-x9hzm\") pod \"payload-pre-processing-e2e-praxis-a-7ea4b0-68cb74f9-rlhnf\" (UID: \"869411ac-8712-4f5c-8634-7b4cc2ab37f9\") " pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-7ea4b0-68cb74f9-rlhnf"
Oct 31 01:01:33.262267 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:01:33.254195    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/869411ac-8712-4f5c-8634-7b4cc2ab37f9-plugins-config-volume\") pod \"payload-pre-processing-e2e-praxis-a-7ea4b0-68cb74f9-rlhnf\" (UID: \"869411ac-8712-4f5c-8634-7b4cc2ab37f9\") " pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-7ea4b0-68cb74f9-rlhnf"
Oct 31 01:01:33.263255 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:01:33.263235    4284 cpu_manager.go:414] "containerMap: removing container" podUID="c0f54831-726b-41bd-adfb-445f8b9a39f9" containerName="istio-proxy"
Oct 31 01:01:33.263407 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:01:33.263392    4284 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c0f54831-726b-41bd-adfb-445f8b9a39f9" containerName="istio-proxy"
Oct 31 01:01:33.263755 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:01:33.263739    4284 memory_manager.go:365] "RemoveStaleState removing state" podUID="c0f54831-726b-41bd-adfb-445f8b9a39f9" containerName="istio-proxy"
Oct 31 01:01:33.289814 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:01:33.289783    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-praxis-a-7ea4b0-75f8f8786f-6cljj"
Oct 31 01:01:33.310414 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:01:33.310363    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-praxis-a-7ea4b0-75f8f8786f-6cljj"]
Oct 31 01:01:33.326476 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:01:33.326436    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x9hzm\" (UniqueName: \"kubernetes.io/projected/869411ac-8712-4f5c-8634-7b4cc2ab37f9-kube-api-access-x9hzm\") pod \"payload-pre-processing-e2e-praxis-a-7ea4b0-68cb74f9-rlhnf\" (UID: \"869411ac-8712-4f5c-8634-7b4cc2ab37f9\") " pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-7ea4b0-68cb74f9-rlhnf"
Oct 31 01:01:33.362953 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:01:33.362911    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/462d3195-7fa1-4a0c-a8aa-9d32f0e87b6e-plugins-config-volume\") pod \"payload-processing-e2e-praxis-a-7ea4b0-75f8f8786f-6cljj\" (UID: \"462d3195-7fa1-4a0c-a8aa-9d32f0e87b6e\") " pod="openshift-ingress/payload-processing-e2e-praxis-a-7ea4b0-75f8f8786f-6cljj"
Oct 31 01:01:33.363124 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:01:33.362986    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8xl2f\" (UniqueName: \"kubernetes.io/projected/462d3195-7fa1-4a0c-a8aa-9d32f0e87b6e-kube-api-access-8xl2f\") pod \"payload-processing-e2e-praxis-a-7ea4b0-75f8f8786f-6cljj\" (UID: \"462d3195-7fa1-4a0c-a8aa-9d32f0e87b6e\") " pod="openshift-ingress/payload-processing-e2e-praxis-a-7ea4b0-75f8f8786f-6cljj"
Oct 31 01:01:33.464365 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:01:33.464326    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/462d3195-7fa1-4a0c-a8aa-9d32f0e87b6e-plugins-config-volume\") pod \"payload-processing-e2e-praxis-a-7ea4b0-75f8f8786f-6cljj\" (UID: \"462d3195-7fa1-4a0c-a8aa-9d32f0e87b6e\") " pod="openshift-ingress/payload-processing-e2e-praxis-a-7ea4b0-75f8f8786f-6cljj"
Oct 31 01:01:33.464632 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:01:33.464399    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8xl2f\" (UniqueName: \"kubernetes.io/projected/462d3195-7fa1-4a0c-a8aa-9d32f0e87b6e-kube-api-access-8xl2f\") pod \"payload-processing-e2e-praxis-a-7ea4b0-75f8f8786f-6cljj\" (UID: \"462d3195-7fa1-4a0c-a8aa-9d32f0e87b6e\") " pod="openshift-ingress/payload-processing-e2e-praxis-a-7ea4b0-75f8f8786f-6cljj"
Oct 31 01:01:33.465079 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:01:33.465059    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/462d3195-7fa1-4a0c-a8aa-9d32f0e87b6e-plugins-config-volume\") pod \"payload-processing-e2e-praxis-a-7ea4b0-75f8f8786f-6cljj\" (UID: \"462d3195-7fa1-4a0c-a8aa-9d32f0e87b6e\") " pod="openshift-ingress/payload-processing-e2e-praxis-a-7ea4b0-75f8f8786f-6cljj"
Oct 31 01:01:33.524568 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:01:33.524495    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8xl2f\" (UniqueName: \"kubernetes.io/projected/462d3195-7fa1-4a0c-a8aa-9d32f0e87b6e-kube-api-access-8xl2f\") pod \"payload-processing-e2e-praxis-a-7ea4b0-75f8f8786f-6cljj\" (UID: \"462d3195-7fa1-4a0c-a8aa-9d32f0e87b6e\") " pod="openshift-ingress/payload-processing-e2e-praxis-a-7ea4b0-75f8f8786f-6cljj"
Oct 31 01:01:33.621273 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:01:33.621203    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-praxis-a-7ea4b0-75f8f8786f-6cljj"
Oct 31 01:01:33.626105 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:01:33.626081    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-7ea4b0-68cb74f9-rlhnf"
Oct 31 01:01:33.910752 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:01:33.910718    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:01:33.999321 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:01:33.999291    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-praxis-a-7ea4b0-75f8f8786f-6cljj"]
Oct 31 01:01:33.999967 ip-10-0-132-173 kubenswrapper[4284]: W1031 01:01:33.999932    4284 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod462d3195_7fa1_4a0c_a8aa_9d32f0e87b6e.slice/crio-e4d625020ec2a65d0405c16c4f5e985b357da8f4c4a720cbf5f9e59429e154b9 WatchSource:0}: Error finding container e4d625020ec2a65d0405c16c4f5e985b357da8f4c4a720cbf5f9e59429e154b9: Status 404 returned error can't find the container with id e4d625020ec2a65d0405c16c4f5e985b357da8f4c4a720cbf5f9e59429e154b9
Oct 31 01:01:34.048356 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:01:34.048327    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-praxis-a-7ea4b0-68cb74f9-rlhnf"]
Oct 31 01:01:34.050245 ip-10-0-132-173 kubenswrapper[4284]: W1031 01:01:34.050196    4284 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod869411ac_8712_4f5c_8634_7b4cc2ab37f9.slice/crio-837a72bee81e8b9300e9b905cd4873b2062164ef8da240b337d47aeae23e4ffe WatchSource:0}: Error finding container 837a72bee81e8b9300e9b905cd4873b2062164ef8da240b337d47aeae23e4ffe: Status 404 returned error can't find the container with id 837a72bee81e8b9300e9b905cd4873b2062164ef8da240b337d47aeae23e4ffe
Oct 31 01:01:34.622124 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:01:34.622086    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:01:34.648167 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:01:34.642784    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-7ea4b0-68cb74f9-rlhnf" event={"ID":"869411ac-8712-4f5c-8634-7b4cc2ab37f9","Type":"ContainerStarted","Data":"4c38cd91eea1ccbaf125481d2ed16e9c17cea3573ad412c147ff82a8f6f4b325"}
Oct 31 01:01:34.648167 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:01:34.642873    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-7ea4b0-68cb74f9-rlhnf" event={"ID":"869411ac-8712-4f5c-8634-7b4cc2ab37f9","Type":"ContainerStarted","Data":"837a72bee81e8b9300e9b905cd4873b2062164ef8da240b337d47aeae23e4ffe"}
Oct 31 01:01:34.648167 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:01:34.647805    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-7ea4b0-68cb74f9-rlhnf"
Oct 31 01:01:34.655709 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:01:34.649633    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-praxis-a-7ea4b0-75f8f8786f-6cljj" event={"ID":"462d3195-7fa1-4a0c-a8aa-9d32f0e87b6e","Type":"ContainerStarted","Data":"53ed149ce6c083eedfcfb6b1c8fb638e0200559f4abf3311a4503f17f5ab8db9"}
Oct 31 01:01:34.655709 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:01:34.649689    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-praxis-a-7ea4b0-75f8f8786f-6cljj" event={"ID":"462d3195-7fa1-4a0c-a8aa-9d32f0e87b6e","Type":"ContainerStarted","Data":"e4d625020ec2a65d0405c16c4f5e985b357da8f4c4a720cbf5f9e59429e154b9"}
Oct 31 01:01:34.655709 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:01:34.650035    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-praxis-a-7ea4b0-75f8f8786f-6cljj"
Oct 31 01:01:34.771082 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:01:34.771033    4284 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-7ea4b0-68cb74f9-rlhnf" podStartSLOduration=2.771018709 podStartE2EDuration="2.771018709s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 01:01:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 01:01:34.770678558 +0000 UTC m=+2008.915685474" watchObservedRunningTime="2026-10-31 01:01:34.771018709 +0000 UTC m=+2008.916025625"
Oct 31 01:01:34.865044 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:01:34.865006    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:01:35.191819 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:01:35.191779    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:01:38.286795 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:01:38.286753    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:01:38.528881 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:01:38.528844    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:01:38.674361 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:01:38.674306    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:01:38.836087 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:01:38.836052    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:01:38.989917 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:01:38.989828    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:01:39.135783 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:01:39.135746    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:01:40.985941 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:01:40.985909    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:01:41.150769 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:01:41.150730    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:01:41.983096 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:01:41.983061    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:01:42.176741 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:01:42.175967    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:01:42.380007 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:01:42.379896    4284 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-praxis-a-7ea4b0-75f8f8786f-6cljj" podStartSLOduration=9.379882009 podStartE2EDuration="9.379882009s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 01:01:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 01:01:34.894822871 +0000 UTC m=+2009.039829788" watchObservedRunningTime="2026-10-31 01:01:42.379882009 +0000 UTC m=+2016.524888925"
Oct 31 01:01:42.382379 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:01:42.382359    4284 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-praxis-b-9e4801-748ccff66c-r2x25"]
Oct 31 01:01:42.386023 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:01:42.386006    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-9e4801-748ccff66c-r2x25"
Oct 31 01:01:42.401559 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:01:42.401527    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/aec30731-ea23-4b49-9d93-36243434cf3d-plugins-config-volume\") pod \"payload-pre-processing-e2e-praxis-b-9e4801-748ccff66c-r2x25\" (UID: \"aec30731-ea23-4b49-9d93-36243434cf3d\") " pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-9e4801-748ccff66c-r2x25"
Oct 31 01:01:42.401671 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:01:42.401628    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lf5p2\" (UniqueName: \"kubernetes.io/projected/aec30731-ea23-4b49-9d93-36243434cf3d-kube-api-access-lf5p2\") pod \"payload-pre-processing-e2e-praxis-b-9e4801-748ccff66c-r2x25\" (UID: \"aec30731-ea23-4b49-9d93-36243434cf3d\") " pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-9e4801-748ccff66c-r2x25"
Oct 31 01:01:42.505243 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:01:42.441940    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-praxis-b-9e4801-748ccff66c-r2x25"]
Oct 31 01:01:42.505243 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:01:42.484650    4284 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-praxis-b-9e4801\""
Oct 31 01:01:42.505243 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:01:42.484804    4284 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-praxis-b-9e4801-dockercfg-lcbd7\""
Oct 31 01:01:42.581285 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:01:42.577058    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/aec30731-ea23-4b49-9d93-36243434cf3d-plugins-config-volume\") pod \"payload-pre-processing-e2e-praxis-b-9e4801-748ccff66c-r2x25\" (UID: \"aec30731-ea23-4b49-9d93-36243434cf3d\") " pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-9e4801-748ccff66c-r2x25"
Oct 31 01:01:42.581285 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:01:42.577145    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lf5p2\" (UniqueName: \"kubernetes.io/projected/aec30731-ea23-4b49-9d93-36243434cf3d-kube-api-access-lf5p2\") pod \"payload-pre-processing-e2e-praxis-b-9e4801-748ccff66c-r2x25\" (UID: \"aec30731-ea23-4b49-9d93-36243434cf3d\") " pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-9e4801-748ccff66c-r2x25"
Oct 31 01:01:42.581285 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:01:42.578197    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/aec30731-ea23-4b49-9d93-36243434cf3d-plugins-config-volume\") pod \"payload-pre-processing-e2e-praxis-b-9e4801-748ccff66c-r2x25\" (UID: \"aec30731-ea23-4b49-9d93-36243434cf3d\") " pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-9e4801-748ccff66c-r2x25"
Oct 31 01:01:42.636303 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:01:42.636192    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lf5p2\" (UniqueName: \"kubernetes.io/projected/aec30731-ea23-4b49-9d93-36243434cf3d-kube-api-access-lf5p2\") pod \"payload-pre-processing-e2e-praxis-b-9e4801-748ccff66c-r2x25\" (UID: \"aec30731-ea23-4b49-9d93-36243434cf3d\") " pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-9e4801-748ccff66c-r2x25"
Oct 31 01:01:42.702429 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:01:42.702395    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-9e4801-748ccff66c-r2x25"
Oct 31 01:01:43.069949 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:01:43.069919    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-praxis-b-9e4801-748ccff66c-r2x25"]
Oct 31 01:01:43.070174 ip-10-0-132-173 kubenswrapper[4284]: W1031 01:01:43.070154    4284 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaec30731_ea23_4b49_9d93_36243434cf3d.slice/crio-68aa9cdfed49f705236c9584394138599f70fef8ce3174f4c046ffb0cd288ff8 WatchSource:0}: Error finding container 68aa9cdfed49f705236c9584394138599f70fef8ce3174f4c046ffb0cd288ff8: Status 404 returned error can't find the container with id 68aa9cdfed49f705236c9584394138599f70fef8ce3174f4c046ffb0cd288ff8
Oct 31 01:01:43.523842 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:01:43.523805    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:01:43.728618 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:01:43.728583    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-9e4801-748ccff66c-r2x25" event={"ID":"aec30731-ea23-4b49-9d93-36243434cf3d","Type":"ContainerStarted","Data":"8adf85581319abb3ae3103136d9ad913b1c8b482204e96193bf3007cfc4eae4f"}
Oct 31 01:01:43.728785 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:01:43.728642    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-9e4801-748ccff66c-r2x25" event={"ID":"aec30731-ea23-4b49-9d93-36243434cf3d","Type":"ContainerStarted","Data":"68aa9cdfed49f705236c9584394138599f70fef8ce3174f4c046ffb0cd288ff8"}
Oct 31 01:01:43.728785 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:01:43.728672    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-9e4801-748ccff66c-r2x25"
Oct 31 01:01:43.842376 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:01:43.842272    4284 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-9e4801-748ccff66c-r2x25" podStartSLOduration=1.842258981 podStartE2EDuration="1.842258981s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 01:01:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 01:01:43.839687671 +0000 UTC m=+2017.984694589" watchObservedRunningTime="2026-10-31 01:01:43.842258981 +0000 UTC m=+2017.987265908"
Oct 31 01:01:45.663385 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:01:45.663357    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-praxis-a-7ea4b0-75f8f8786f-6cljj"
Oct 31 01:01:46.668480 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:01:46.668392    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-7ea4b0-68cb74f9-rlhnf"
Oct 31 01:01:47.018378 ip-10-0-132-173 kubenswrapper[4284]: E1031 01:01:47.018292    4284 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fd8sp" containerName="extractor"
Oct 31 01:01:49.786488 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:01:49.786442    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:01:54.446289 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:01:54.446188    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:01:54.597398 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:01:54.597348    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:01:54.739427 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:01:54.739344    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-9e4801-748ccff66c-r2x25"
Oct 31 01:01:54.779001 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:01:54.778968    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:01:59.403292 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:01:59.403251    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:02:00.292800 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:00.292712    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:02:04.194146 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:04.194103    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:02:08.890415 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:08.890380    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:02:12.919334 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:12.919306    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-b647c3-6bc8996947-bd4gb"]
Oct 31 01:02:12.919906 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:12.919536    4284 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-b647c3-6bc8996947-bd4gb" podUID="990f55f0-10cc-4930-8f0a-62989ef4aded" containerName="maas-api" containerID="cri-o://b61f38ac5d15cee202f2a732243fc0393386888c1ace9a6d1f392650dab66fbb" gracePeriod=30
Oct 31 01:02:13.020777 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:13.020743    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:02:13.240563 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:13.240538    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-b647c3-6bc8996947-bd4gb"
Oct 31 01:02:13.360407 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:13.360375    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/990f55f0-10cc-4930-8f0a-62989ef4aded-metrics-certs\" (UniqueName: \"kubernetes.io/secret/990f55f0-10cc-4930-8f0a-62989ef4aded-metrics-certs\") pod \"990f55f0-10cc-4930-8f0a-62989ef4aded\" (UID: \"990f55f0-10cc-4930-8f0a-62989ef4aded\") "
Oct 31 01:02:13.360751 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:13.360659    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/990f55f0-10cc-4930-8f0a-62989ef4aded-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/990f55f0-10cc-4930-8f0a-62989ef4aded-openshift-service-ca\") pod \"990f55f0-10cc-4930-8f0a-62989ef4aded\" (UID: \"990f55f0-10cc-4930-8f0a-62989ef4aded\") "
Oct 31 01:02:13.360904 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:13.360889    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/990f55f0-10cc-4930-8f0a-62989ef4aded-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/990f55f0-10cc-4930-8f0a-62989ef4aded-maas-api-tls\") pod \"990f55f0-10cc-4930-8f0a-62989ef4aded\" (UID: \"990f55f0-10cc-4930-8f0a-62989ef4aded\") "
Oct 31 01:02:13.361155 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:13.361128    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/990f55f0-10cc-4930-8f0a-62989ef4aded-kube-api-access-kkblb\" (UniqueName: \"kubernetes.io/projected/990f55f0-10cc-4930-8f0a-62989ef4aded-kube-api-access-kkblb\") pod \"990f55f0-10cc-4930-8f0a-62989ef4aded\" (UID: \"990f55f0-10cc-4930-8f0a-62989ef4aded\") "
Oct 31 01:02:13.367875 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:13.367468    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/990f55f0-10cc-4930-8f0a-62989ef4aded-openshift-service-ca" pod "990f55f0-10cc-4930-8f0a-62989ef4aded" (UID: "990f55f0-10cc-4930-8f0a-62989ef4aded"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 01:02:13.373590 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:13.372477    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/990f55f0-10cc-4930-8f0a-62989ef4aded-metrics-certs" pod "990f55f0-10cc-4930-8f0a-62989ef4aded" (UID: "990f55f0-10cc-4930-8f0a-62989ef4aded"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 01:02:13.380444 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:13.380413    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/990f55f0-10cc-4930-8f0a-62989ef4aded-kube-api-access-kkblb" pod "990f55f0-10cc-4930-8f0a-62989ef4aded" (UID: "990f55f0-10cc-4930-8f0a-62989ef4aded"). InnerVolumeSpecName "kube-api-access-kkblb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:02:13.380672 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:13.380637    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/990f55f0-10cc-4930-8f0a-62989ef4aded-maas-api-tls" pod "990f55f0-10cc-4930-8f0a-62989ef4aded" (UID: "990f55f0-10cc-4930-8f0a-62989ef4aded"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 01:02:13.392414 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:13.392388    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-b647c3-5b7c7c4c44-srzhm"]
Oct 31 01:02:13.392782 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:13.392742    4284 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-a-b647c3-5b7c7c4c44-srzhm" podUID="1bfc3a17-7cea-4195-a6ab-0c82338451af" containerName="payload-processing" containerID="cri-o://5f1ef4ad48c15f4355c0e409487b49956acdf37d3f9d3b459f1b553461316781" gracePeriod=30
Oct 31 01:02:13.445521 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:13.445491    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-b647c3-799446d767-7g2mp"]
Oct 31 01:02:13.445694 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:13.445674    4284 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-b647c3-799446d767-7g2mp" podUID="73ee0248-f0a6-4d16-9e30-5301a63fa113" containerName="payload-pre-processing" containerID="cri-o://12e86cc48025aafdfb02221872e23243a50d37d8db54b2c05e250a3219d1b631" gracePeriod=30
Oct 31 01:02:13.462783 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:13.462729    4284 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/990f55f0-10cc-4930-8f0a-62989ef4aded-metrics-certs\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:02:13.462783 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:13.462755    4284 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/990f55f0-10cc-4930-8f0a-62989ef4aded-openshift-service-ca\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:02:13.462783 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:13.462766    4284 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/990f55f0-10cc-4930-8f0a-62989ef4aded-maas-api-tls\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:02:13.462783 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:13.462775    4284 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kkblb\" (UniqueName: \"kubernetes.io/projected/990f55f0-10cc-4930-8f0a-62989ef4aded-kube-api-access-kkblb\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:02:13.560107 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:13.560070    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:02:13.701908 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:13.701879    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-b647c3-5b7c7c4c44-srzhm"
Oct 31 01:02:13.735011 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:13.734984    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-b647c3-799446d767-7g2mp"
Oct 31 01:02:13.748565 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:13.748519    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:02:13.766185 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:13.766153    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1bfc3a17-7cea-4195-a6ab-0c82338451af-kube-api-access-62sbv\" (UniqueName: \"kubernetes.io/projected/1bfc3a17-7cea-4195-a6ab-0c82338451af-kube-api-access-62sbv\") pod \"1bfc3a17-7cea-4195-a6ab-0c82338451af\" (UID: \"1bfc3a17-7cea-4195-a6ab-0c82338451af\") "
Oct 31 01:02:13.766343 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:13.766206    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1bfc3a17-7cea-4195-a6ab-0c82338451af-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1bfc3a17-7cea-4195-a6ab-0c82338451af-plugins-config-volume\") pod \"1bfc3a17-7cea-4195-a6ab-0c82338451af\" (UID: \"1bfc3a17-7cea-4195-a6ab-0c82338451af\") "
Oct 31 01:02:13.766629 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:13.766609    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1bfc3a17-7cea-4195-a6ab-0c82338451af-plugins-config-volume" pod "1bfc3a17-7cea-4195-a6ab-0c82338451af" (UID: "1bfc3a17-7cea-4195-a6ab-0c82338451af"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 01:02:13.768450 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:13.768426    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1bfc3a17-7cea-4195-a6ab-0c82338451af-kube-api-access-62sbv" pod "1bfc3a17-7cea-4195-a6ab-0c82338451af" (UID: "1bfc3a17-7cea-4195-a6ab-0c82338451af"). InnerVolumeSpecName "kube-api-access-62sbv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:02:13.874259 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:13.867408    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/73ee0248-f0a6-4d16-9e30-5301a63fa113-kube-api-access-drr5m\" (UniqueName: \"kubernetes.io/projected/73ee0248-f0a6-4d16-9e30-5301a63fa113-kube-api-access-drr5m\") pod \"73ee0248-f0a6-4d16-9e30-5301a63fa113\" (UID: \"73ee0248-f0a6-4d16-9e30-5301a63fa113\") "
Oct 31 01:02:13.874259 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:13.867980    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/73ee0248-f0a6-4d16-9e30-5301a63fa113-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/73ee0248-f0a6-4d16-9e30-5301a63fa113-plugins-config-volume\") pod \"73ee0248-f0a6-4d16-9e30-5301a63fa113\" (UID: \"73ee0248-f0a6-4d16-9e30-5301a63fa113\") "
Oct 31 01:02:13.874259 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:13.868808    4284 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-62sbv\" (UniqueName: \"kubernetes.io/projected/1bfc3a17-7cea-4195-a6ab-0c82338451af-kube-api-access-62sbv\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:02:13.874259 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:13.868833    4284 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1bfc3a17-7cea-4195-a6ab-0c82338451af-plugins-config-volume\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:02:13.874259 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:13.870589    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/73ee0248-f0a6-4d16-9e30-5301a63fa113-plugins-config-volume" pod "73ee0248-f0a6-4d16-9e30-5301a63fa113" (UID: "73ee0248-f0a6-4d16-9e30-5301a63fa113"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 01:02:13.874259 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:13.873293    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/73ee0248-f0a6-4d16-9e30-5301a63fa113-kube-api-access-drr5m" pod "73ee0248-f0a6-4d16-9e30-5301a63fa113" (UID: "73ee0248-f0a6-4d16-9e30-5301a63fa113"). InnerVolumeSpecName "kube-api-access-drr5m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:02:13.982532 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:13.982003    4284 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/73ee0248-f0a6-4d16-9e30-5301a63fa113-plugins-config-volume\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:02:13.982532 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:13.982039    4284 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-drr5m\" (UniqueName: \"kubernetes.io/projected/73ee0248-f0a6-4d16-9e30-5301a63fa113-kube-api-access-drr5m\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:02:14.042372 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:14.042345    4284 generic.go:356] "Generic (PLEG): container finished" podID="1bfc3a17-7cea-4195-a6ab-0c82338451af" containerID="5f1ef4ad48c15f4355c0e409487b49956acdf37d3f9d3b459f1b553461316781" exitCode=0
Oct 31 01:02:14.042539 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:14.042405    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-b647c3-5b7c7c4c44-srzhm"
Oct 31 01:02:14.042539 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:14.042493    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-b647c3-5b7c7c4c44-srzhm" event={"ID":"1bfc3a17-7cea-4195-a6ab-0c82338451af","Type":"ContainerDied","Data":"5f1ef4ad48c15f4355c0e409487b49956acdf37d3f9d3b459f1b553461316781"}
Oct 31 01:02:14.042539 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:14.042528    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-b647c3-5b7c7c4c44-srzhm" event={"ID":"1bfc3a17-7cea-4195-a6ab-0c82338451af","Type":"ContainerDied","Data":"bf6ed398aa85addbbcb3d9f0b1ad0069788f4bca484cacef7c99d429f75c1077"}
Oct 31 01:02:14.042696 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:14.042549    4284 scope.go:122] "RemoveContainer" containerID="5f1ef4ad48c15f4355c0e409487b49956acdf37d3f9d3b459f1b553461316781"
Oct 31 01:02:14.044353 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:14.044331    4284 generic.go:356] "Generic (PLEG): container finished" podID="73ee0248-f0a6-4d16-9e30-5301a63fa113" containerID="12e86cc48025aafdfb02221872e23243a50d37d8db54b2c05e250a3219d1b631" exitCode=0
Oct 31 01:02:14.044618 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:14.044465    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-b647c3-799446d767-7g2mp"
Oct 31 01:02:14.045036 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:14.045012    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-b647c3-799446d767-7g2mp" event={"ID":"73ee0248-f0a6-4d16-9e30-5301a63fa113","Type":"ContainerDied","Data":"12e86cc48025aafdfb02221872e23243a50d37d8db54b2c05e250a3219d1b631"}
Oct 31 01:02:14.045126 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:14.045067    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-b647c3-799446d767-7g2mp" event={"ID":"73ee0248-f0a6-4d16-9e30-5301a63fa113","Type":"ContainerDied","Data":"ffe1ff649692cba73f29f1e0d624c69be86c3d728b56d7ecf0e1bfd792131235"}
Oct 31 01:02:14.046424 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:14.046391    4284 generic.go:356] "Generic (PLEG): container finished" podID="990f55f0-10cc-4930-8f0a-62989ef4aded" containerID="b61f38ac5d15cee202f2a732243fc0393386888c1ace9a6d1f392650dab66fbb" exitCode=0
Oct 31 01:02:14.046541 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:14.046438    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-b647c3-6bc8996947-bd4gb" event={"ID":"990f55f0-10cc-4930-8f0a-62989ef4aded","Type":"ContainerDied","Data":"b61f38ac5d15cee202f2a732243fc0393386888c1ace9a6d1f392650dab66fbb"}
Oct 31 01:02:14.046541 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:14.046464    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-b647c3-6bc8996947-bd4gb" event={"ID":"990f55f0-10cc-4930-8f0a-62989ef4aded","Type":"ContainerDied","Data":"07815d151c6405e6dc4960fdc727673549698ab8a966ca66ee7492379a4ef103"}
Oct 31 01:02:14.046541 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:14.046469    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-b647c3-6bc8996947-bd4gb"
Oct 31 01:02:14.054520 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:14.054502    4284 scope.go:122] "RemoveContainer" containerID="5f1ef4ad48c15f4355c0e409487b49956acdf37d3f9d3b459f1b553461316781"
Oct 31 01:02:14.054811 ip-10-0-132-173 kubenswrapper[4284]: E1031 01:02:14.054794    4284 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5f1ef4ad48c15f4355c0e409487b49956acdf37d3f9d3b459f1b553461316781\": container with ID starting with 5f1ef4ad48c15f4355c0e409487b49956acdf37d3f9d3b459f1b553461316781 not found: ID does not exist" containerID="5f1ef4ad48c15f4355c0e409487b49956acdf37d3f9d3b459f1b553461316781"
Oct 31 01:02:14.054864 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:14.054842    4284 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5f1ef4ad48c15f4355c0e409487b49956acdf37d3f9d3b459f1b553461316781"} err="failed to get container status \"5f1ef4ad48c15f4355c0e409487b49956acdf37d3f9d3b459f1b553461316781\": rpc error: code = NotFound desc = could not find container \"5f1ef4ad48c15f4355c0e409487b49956acdf37d3f9d3b459f1b553461316781\": container with ID starting with 5f1ef4ad48c15f4355c0e409487b49956acdf37d3f9d3b459f1b553461316781 not found: ID does not exist"
Oct 31 01:02:14.054864 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:14.054860    4284 scope.go:122] "RemoveContainer" containerID="12e86cc48025aafdfb02221872e23243a50d37d8db54b2c05e250a3219d1b631"
Oct 31 01:02:14.064760 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:14.064744    4284 scope.go:122] "RemoveContainer" containerID="12e86cc48025aafdfb02221872e23243a50d37d8db54b2c05e250a3219d1b631"
Oct 31 01:02:14.065033 ip-10-0-132-173 kubenswrapper[4284]: E1031 01:02:14.065011    4284 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"12e86cc48025aafdfb02221872e23243a50d37d8db54b2c05e250a3219d1b631\": container with ID starting with 12e86cc48025aafdfb02221872e23243a50d37d8db54b2c05e250a3219d1b631 not found: ID does not exist" containerID="12e86cc48025aafdfb02221872e23243a50d37d8db54b2c05e250a3219d1b631"
Oct 31 01:02:14.065123 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:14.065037    4284 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"12e86cc48025aafdfb02221872e23243a50d37d8db54b2c05e250a3219d1b631"} err="failed to get container status \"12e86cc48025aafdfb02221872e23243a50d37d8db54b2c05e250a3219d1b631\": rpc error: code = NotFound desc = could not find container \"12e86cc48025aafdfb02221872e23243a50d37d8db54b2c05e250a3219d1b631\": container with ID starting with 12e86cc48025aafdfb02221872e23243a50d37d8db54b2c05e250a3219d1b631 not found: ID does not exist"
Oct 31 01:02:14.065123 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:14.065053    4284 scope.go:122] "RemoveContainer" containerID="b61f38ac5d15cee202f2a732243fc0393386888c1ace9a6d1f392650dab66fbb"
Oct 31 01:02:14.073354 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:14.073340    4284 scope.go:122] "RemoveContainer" containerID="b61f38ac5d15cee202f2a732243fc0393386888c1ace9a6d1f392650dab66fbb"
Oct 31 01:02:14.073619 ip-10-0-132-173 kubenswrapper[4284]: E1031 01:02:14.073602    4284 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b61f38ac5d15cee202f2a732243fc0393386888c1ace9a6d1f392650dab66fbb\": container with ID starting with b61f38ac5d15cee202f2a732243fc0393386888c1ace9a6d1f392650dab66fbb not found: ID does not exist" containerID="b61f38ac5d15cee202f2a732243fc0393386888c1ace9a6d1f392650dab66fbb"
Oct 31 01:02:14.073672 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:14.073622    4284 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b61f38ac5d15cee202f2a732243fc0393386888c1ace9a6d1f392650dab66fbb"} err="failed to get container status \"b61f38ac5d15cee202f2a732243fc0393386888c1ace9a6d1f392650dab66fbb\": rpc error: code = NotFound desc = could not find container \"b61f38ac5d15cee202f2a732243fc0393386888c1ace9a6d1f392650dab66fbb\": container with ID starting with b61f38ac5d15cee202f2a732243fc0393386888c1ace9a6d1f392650dab66fbb not found: ID does not exist"
Oct 31 01:02:14.201882 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:14.201850    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-b647c3-5b7c7c4c44-srzhm"]
Oct 31 01:02:14.208040 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:14.208015    4284 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-b647c3-5b7c7c4c44-srzhm"]
Oct 31 01:02:14.315023 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:14.314945    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-b647c3-6bc8996947-bd4gb"]
Oct 31 01:02:14.321671 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:14.321640    4284 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-b647c3-6bc8996947-bd4gb"]
Oct 31 01:02:14.426733 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:14.426666    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-b647c3-799446d767-7g2mp"]
Oct 31 01:02:14.430311 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:14.430287    4284 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-b647c3-799446d767-7g2mp"]
Oct 31 01:02:14.565093 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:14.565013    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:02:14.768804 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:14.768754    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:02:14.920720 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:14.920691    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:02:15.553323 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:15.553288    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:02:16.008235 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:16.004797    4284 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1bfc3a17-7cea-4195-a6ab-0c82338451af" path="/var/lib/kubelet/pods/1bfc3a17-7cea-4195-a6ab-0c82338451af/volumes"
Oct 31 01:02:16.008235 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:16.005432    4284 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="73ee0248-f0a6-4d16-9e30-5301a63fa113" path="/var/lib/kubelet/pods/73ee0248-f0a6-4d16-9e30-5301a63fa113/volumes"
Oct 31 01:02:16.008235 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:16.006169    4284 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="990f55f0-10cc-4930-8f0a-62989ef4aded" path="/var/lib/kubelet/pods/990f55f0-10cc-4930-8f0a-62989ef4aded/volumes"
Oct 31 01:02:17.038926 ip-10-0-132-173 kubenswrapper[4284]: E1031 01:02:17.038892    4284 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fd8sp" containerName="extractor"
Oct 31 01:02:19.910780 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:19.910743    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:02:22.199999 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:22.199953    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:02:22.393414 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:22.393378    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:02:22.540114 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:22.540028    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-b647c3-openshift-default-674fcc8445-tvzlw"]
Oct 31 01:02:22.540323 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:22.540270    4284 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-a-b647c3-openshift-default-674fcc8445-tvzlw" podUID="caf73a48-2172-46df-9cbc-fd85b49552c1" containerName="istio-proxy" containerID="cri-o://867fd738ff0fb9c788a891250912b0306b7069ff4157488172a234aa175e56eb" gracePeriod=30
Oct 31 01:02:22.641190 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:22.641029    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:02:23.125459 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:23.125417    4284 patch_prober.go:29] interesting pod/e2e-api-b-31ae40-openshift-default-66bb6b84bc-8wrm6 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.128:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 31 01:02:23.125642 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:23.125526    4284 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-31ae40-openshift-default-66bb6b84bc-8wrm6" podUID="5005c539-d22a-4c99-9235-e08fe509575f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.128:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 31 01:02:24.586507 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:24.586476    4284 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-a-7ea4b0-55986d8fcc-tc4zl"]
Oct 31 01:02:24.587034 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:24.587021    4284 cpu_manager.go:414] "containerMap: removing container" podUID="990f55f0-10cc-4930-8f0a-62989ef4aded" containerName="maas-api"
Oct 31 01:02:24.587079 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:24.587035    4284 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="990f55f0-10cc-4930-8f0a-62989ef4aded" containerName="maas-api"
Oct 31 01:02:24.587079 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:24.587051    4284 cpu_manager.go:414] "containerMap: removing container" podUID="1bfc3a17-7cea-4195-a6ab-0c82338451af" containerName="payload-processing"
Oct 31 01:02:24.587079 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:24.587056    4284 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1bfc3a17-7cea-4195-a6ab-0c82338451af" containerName="payload-processing"
Oct 31 01:02:24.587079 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:24.587073    4284 cpu_manager.go:414] "containerMap: removing container" podUID="73ee0248-f0a6-4d16-9e30-5301a63fa113" containerName="payload-pre-processing"
Oct 31 01:02:24.587079 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:24.587079    4284 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="73ee0248-f0a6-4d16-9e30-5301a63fa113" containerName="payload-pre-processing"
Oct 31 01:02:24.587257 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:24.587179    4284 memory_manager.go:365] "RemoveStaleState removing state" podUID="990f55f0-10cc-4930-8f0a-62989ef4aded" containerName="maas-api"
Oct 31 01:02:24.587257 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:24.587187    4284 memory_manager.go:365] "RemoveStaleState removing state" podUID="1bfc3a17-7cea-4195-a6ab-0c82338451af" containerName="payload-processing"
Oct 31 01:02:24.587257 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:24.587196    4284 memory_manager.go:365] "RemoveStaleState removing state" podUID="73ee0248-f0a6-4d16-9e30-5301a63fa113" containerName="payload-pre-processing"
Oct 31 01:02:24.592309 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:24.592290    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-7ea4b0-55986d8fcc-tc4zl"
Oct 31 01:02:24.695167 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:24.695131    4284 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-praxis-a-7ea4b0-serving-cert\""
Oct 31 01:02:24.698354 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:24.698329    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-a-7ea4b0-55986d8fcc-tc4zl"]
Oct 31 01:02:24.771194 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:24.771157    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9eb52777-7315-4581-972c-544042cc9c17-metrics-certs\") pod \"maas-api-e2e-praxis-a-7ea4b0-55986d8fcc-tc4zl\" (UID: \"9eb52777-7315-4581-972c-544042cc9c17\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-7ea4b0-55986d8fcc-tc4zl"
Oct 31 01:02:24.771535 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:24.771509    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9eb52777-7315-4581-972c-544042cc9c17-maas-api-tls\") pod \"maas-api-e2e-praxis-a-7ea4b0-55986d8fcc-tc4zl\" (UID: \"9eb52777-7315-4581-972c-544042cc9c17\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-7ea4b0-55986d8fcc-tc4zl"
Oct 31 01:02:24.771732 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:24.771710    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9eb52777-7315-4581-972c-544042cc9c17-openshift-service-ca\") pod \"maas-api-e2e-praxis-a-7ea4b0-55986d8fcc-tc4zl\" (UID: \"9eb52777-7315-4581-972c-544042cc9c17\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-7ea4b0-55986d8fcc-tc4zl"
Oct 31 01:02:24.771846 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:24.771825    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-56cf7\" (UniqueName: \"kubernetes.io/projected/9eb52777-7315-4581-972c-544042cc9c17-kube-api-access-56cf7\") pod \"maas-api-e2e-praxis-a-7ea4b0-55986d8fcc-tc4zl\" (UID: \"9eb52777-7315-4581-972c-544042cc9c17\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-7ea4b0-55986d8fcc-tc4zl"
Oct 31 01:02:24.799833 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:24.799794    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:02:24.880572 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:24.877541    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9eb52777-7315-4581-972c-544042cc9c17-metrics-certs\") pod \"maas-api-e2e-praxis-a-7ea4b0-55986d8fcc-tc4zl\" (UID: \"9eb52777-7315-4581-972c-544042cc9c17\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-7ea4b0-55986d8fcc-tc4zl"
Oct 31 01:02:24.880572 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:24.877592    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9eb52777-7315-4581-972c-544042cc9c17-maas-api-tls\") pod \"maas-api-e2e-praxis-a-7ea4b0-55986d8fcc-tc4zl\" (UID: \"9eb52777-7315-4581-972c-544042cc9c17\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-7ea4b0-55986d8fcc-tc4zl"
Oct 31 01:02:24.880572 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:24.877706    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9eb52777-7315-4581-972c-544042cc9c17-openshift-service-ca\") pod \"maas-api-e2e-praxis-a-7ea4b0-55986d8fcc-tc4zl\" (UID: \"9eb52777-7315-4581-972c-544042cc9c17\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-7ea4b0-55986d8fcc-tc4zl"
Oct 31 01:02:24.880572 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:24.877779    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-56cf7\" (UniqueName: \"kubernetes.io/projected/9eb52777-7315-4581-972c-544042cc9c17-kube-api-access-56cf7\") pod \"maas-api-e2e-praxis-a-7ea4b0-55986d8fcc-tc4zl\" (UID: \"9eb52777-7315-4581-972c-544042cc9c17\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-7ea4b0-55986d8fcc-tc4zl"
Oct 31 01:02:24.882921 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:24.882885    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9eb52777-7315-4581-972c-544042cc9c17-openshift-service-ca\") pod \"maas-api-e2e-praxis-a-7ea4b0-55986d8fcc-tc4zl\" (UID: \"9eb52777-7315-4581-972c-544042cc9c17\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-7ea4b0-55986d8fcc-tc4zl"
Oct 31 01:02:24.885020 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:24.884992    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9eb52777-7315-4581-972c-544042cc9c17-metrics-certs\") pod \"maas-api-e2e-praxis-a-7ea4b0-55986d8fcc-tc4zl\" (UID: \"9eb52777-7315-4581-972c-544042cc9c17\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-7ea4b0-55986d8fcc-tc4zl"
Oct 31 01:02:24.887055 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:24.887032    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9eb52777-7315-4581-972c-544042cc9c17-maas-api-tls\") pod \"maas-api-e2e-praxis-a-7ea4b0-55986d8fcc-tc4zl\" (UID: \"9eb52777-7315-4581-972c-544042cc9c17\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-7ea4b0-55986d8fcc-tc4zl"
Oct 31 01:02:24.937107 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:24.937070    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-56cf7\" (UniqueName: \"kubernetes.io/projected/9eb52777-7315-4581-972c-544042cc9c17-kube-api-access-56cf7\") pod \"maas-api-e2e-praxis-a-7ea4b0-55986d8fcc-tc4zl\" (UID: \"9eb52777-7315-4581-972c-544042cc9c17\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-7ea4b0-55986d8fcc-tc4zl"
Oct 31 01:02:24.974281 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:24.973573    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:02:25.132751 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:25.132657    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:02:25.207163 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:25.207133    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-7ea4b0-55986d8fcc-tc4zl"
Oct 31 01:02:25.592772 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:25.592746    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-a-7ea4b0-55986d8fcc-tc4zl"]
Oct 31 01:02:25.593726 ip-10-0-132-173 kubenswrapper[4284]: W1031 01:02:25.593695    4284 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9eb52777_7315_4581_972c_544042cc9c17.slice/crio-0664daf1e211ee878603d786c77e686dba5984c57bb8fd16c77a43d2fb58ddd3 WatchSource:0}: Error finding container 0664daf1e211ee878603d786c77e686dba5984c57bb8fd16c77a43d2fb58ddd3: Status 404 returned error can't find the container with id 0664daf1e211ee878603d786c77e686dba5984c57bb8fd16c77a43d2fb58ddd3
Oct 31 01:02:25.598729 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:25.598709    4284 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 31 01:02:25.759312 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:25.759275    4284 patch_prober.go:29] interesting pod/e2e-ipp-a-b647c3-openshift-default-674fcc8445-tvzlw container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 31 01:02:25.759491 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:25.759357    4284 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-a-b647c3-openshift-default-674fcc8445-tvzlw" podUID="caf73a48-2172-46df-9cbc-fd85b49552c1" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 31 01:02:26.158608 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:26.158572    4284 generic.go:356] "Generic (PLEG): container finished" podID="5005c539-d22a-4c99-9235-e08fe509575f" containerID="838a3937661be6950efee60ba7a12b02ce531b5b69885538b4a9d49fb3b4edc6" exitCode=0
Oct 31 01:02:26.158791 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:26.158648    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-31ae40-openshift-default-66bb6b84bc-8wrm6" event={"ID":"5005c539-d22a-4c99-9235-e08fe509575f","Type":"ContainerDied","Data":"838a3937661be6950efee60ba7a12b02ce531b5b69885538b4a9d49fb3b4edc6"}
Oct 31 01:02:26.159161 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:26.159142    4284 scope.go:122] "RemoveContainer" containerID="838a3937661be6950efee60ba7a12b02ce531b5b69885538b4a9d49fb3b4edc6"
Oct 31 01:02:26.160108 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:26.159996    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-7ea4b0-55986d8fcc-tc4zl" event={"ID":"9eb52777-7315-4581-972c-544042cc9c17","Type":"ContainerStarted","Data":"0664daf1e211ee878603d786c77e686dba5984c57bb8fd16c77a43d2fb58ddd3"}
Oct 31 01:02:26.160782 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:26.160758    4284 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 01:02:26.160840 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:26.160828    4284 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 01:02:26.160880 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:26.160863    4284 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 01:02:27.168934 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:27.168904    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-31ae40-openshift-default-66bb6b84bc-8wrm6" event={"ID":"5005c539-d22a-4c99-9235-e08fe509575f","Type":"ContainerStarted","Data":"7bba6e781c33e10af0892869e6be00e88ffe3edcd856926ea3ec2cdc005a41a9"}
Oct 31 01:02:27.171052 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:27.171030    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-7ea4b0-55986d8fcc-tc4zl" event={"ID":"9eb52777-7315-4581-972c-544042cc9c17","Type":"ContainerStarted","Data":"d07b2b560b5a379f6f946516295e7def971e2045bebc6ac9bfd8042e08ddeb76"}
Oct 31 01:02:27.171173 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:27.171075    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-7ea4b0-55986d8fcc-tc4zl"
Oct 31 01:02:27.205238 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:27.198708    4284 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-31ae40-openshift-default-66bb6b84bc-8wrm6"
Oct 31 01:02:27.205238 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:27.198844    4284 patch_prober.go:29] interesting pod/e2e-api-b-31ae40-openshift-default-66bb6b84bc-8wrm6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.128:15021/healthz/ready\": dial tcp 10.134.0.128:15021: connect: connection refused" start-of-body=
Oct 31 01:02:27.205238 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:27.198890    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-31ae40-openshift-default-66bb6b84bc-8wrm6" podUID="5005c539-d22a-4c99-9235-e08fe509575f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.128:15021/healthz/ready\": dial tcp 10.134.0.128:15021: connect: connection refused"
Oct 31 01:02:27.417498 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:27.417413    4284 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-7ea4b0-55986d8fcc-tc4zl" podStartSLOduration=2.436788013 podStartE2EDuration="3.41739839s" totalImagesPullingTime="980.610377ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 01:02:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 01:02:27.414183937 +0000 UTC m=+2061.559190856" watchObservedRunningTime="2026-10-31 01:02:27.41739839 +0000 UTC m=+2061.562405307"
Oct 31 01:02:28.018631 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:28.018600    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-b647c3-openshift-default-674fcc8445-tvzlw"
Oct 31 01:02:28.143810 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:28.143773    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/caf73a48-2172-46df-9cbc-fd85b49552c1-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/caf73a48-2172-46df-9cbc-fd85b49552c1-workload-socket\") pod \"caf73a48-2172-46df-9cbc-fd85b49552c1\" (UID: \"caf73a48-2172-46df-9cbc-fd85b49552c1\") "
Oct 31 01:02:28.143810 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:28.143815    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/caf73a48-2172-46df-9cbc-fd85b49552c1-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/caf73a48-2172-46df-9cbc-fd85b49552c1-credential-socket\") pod \"caf73a48-2172-46df-9cbc-fd85b49552c1\" (UID: \"caf73a48-2172-46df-9cbc-fd85b49552c1\") "
Oct 31 01:02:28.144075 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:28.143852    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/caf73a48-2172-46df-9cbc-fd85b49552c1-kube-api-access-xkn66\" (UniqueName: \"kubernetes.io/projected/caf73a48-2172-46df-9cbc-fd85b49552c1-kube-api-access-xkn66\") pod \"caf73a48-2172-46df-9cbc-fd85b49552c1\" (UID: \"caf73a48-2172-46df-9cbc-fd85b49552c1\") "
Oct 31 01:02:28.144075 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:28.143923    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/caf73a48-2172-46df-9cbc-fd85b49552c1-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/caf73a48-2172-46df-9cbc-fd85b49552c1-workload-certs\") pod \"caf73a48-2172-46df-9cbc-fd85b49552c1\" (UID: \"caf73a48-2172-46df-9cbc-fd85b49552c1\") "
Oct 31 01:02:28.144075 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:28.143999    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/caf73a48-2172-46df-9cbc-fd85b49552c1-istio-token\" (UniqueName: \"kubernetes.io/projected/caf73a48-2172-46df-9cbc-fd85b49552c1-istio-token\") pod \"caf73a48-2172-46df-9cbc-fd85b49552c1\" (UID: \"caf73a48-2172-46df-9cbc-fd85b49552c1\") "
Oct 31 01:02:28.144286 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:28.144082    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/caf73a48-2172-46df-9cbc-fd85b49552c1-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/caf73a48-2172-46df-9cbc-fd85b49552c1-istio-envoy\") pod \"caf73a48-2172-46df-9cbc-fd85b49552c1\" (UID: \"caf73a48-2172-46df-9cbc-fd85b49552c1\") "
Oct 31 01:02:28.144286 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:28.144093    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/caf73a48-2172-46df-9cbc-fd85b49552c1-workload-socket" pod "caf73a48-2172-46df-9cbc-fd85b49552c1" (UID: "caf73a48-2172-46df-9cbc-fd85b49552c1"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:02:28.144286 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:28.144165    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/caf73a48-2172-46df-9cbc-fd85b49552c1-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/caf73a48-2172-46df-9cbc-fd85b49552c1-istiod-ca-cert\") pod \"caf73a48-2172-46df-9cbc-fd85b49552c1\" (UID: \"caf73a48-2172-46df-9cbc-fd85b49552c1\") "
Oct 31 01:02:28.144286 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:28.144190    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/caf73a48-2172-46df-9cbc-fd85b49552c1-credential-socket" pod "caf73a48-2172-46df-9cbc-fd85b49552c1" (UID: "caf73a48-2172-46df-9cbc-fd85b49552c1"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:02:28.144286 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:28.144262    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/caf73a48-2172-46df-9cbc-fd85b49552c1-workload-certs" pod "caf73a48-2172-46df-9cbc-fd85b49552c1" (UID: "caf73a48-2172-46df-9cbc-fd85b49552c1"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:02:28.144286 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:28.144250    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/caf73a48-2172-46df-9cbc-fd85b49552c1-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/caf73a48-2172-46df-9cbc-fd85b49552c1-istio-data\") pod \"caf73a48-2172-46df-9cbc-fd85b49552c1\" (UID: \"caf73a48-2172-46df-9cbc-fd85b49552c1\") "
Oct 31 01:02:28.144611 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:28.144316    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/caf73a48-2172-46df-9cbc-fd85b49552c1-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/caf73a48-2172-46df-9cbc-fd85b49552c1-istio-podinfo\") pod \"caf73a48-2172-46df-9cbc-fd85b49552c1\" (UID: \"caf73a48-2172-46df-9cbc-fd85b49552c1\") "
Oct 31 01:02:28.144611 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:28.144428    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/caf73a48-2172-46df-9cbc-fd85b49552c1-istio-data" pod "caf73a48-2172-46df-9cbc-fd85b49552c1" (UID: "caf73a48-2172-46df-9cbc-fd85b49552c1"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:02:28.144611 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:28.144529    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/caf73a48-2172-46df-9cbc-fd85b49552c1-istiod-ca-cert" pod "caf73a48-2172-46df-9cbc-fd85b49552c1" (UID: "caf73a48-2172-46df-9cbc-fd85b49552c1"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 01:02:28.145100 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:28.145076    4284 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/caf73a48-2172-46df-9cbc-fd85b49552c1-workload-socket\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:02:28.145174 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:28.145105    4284 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/caf73a48-2172-46df-9cbc-fd85b49552c1-credential-socket\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:02:28.145174 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:28.145121    4284 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/caf73a48-2172-46df-9cbc-fd85b49552c1-workload-certs\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:02:28.145174 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:28.145137    4284 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/caf73a48-2172-46df-9cbc-fd85b49552c1-istiod-ca-cert\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:02:28.145174 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:28.145151    4284 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/caf73a48-2172-46df-9cbc-fd85b49552c1-istio-data\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:02:28.146504 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:28.146482    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/caf73a48-2172-46df-9cbc-fd85b49552c1-istio-envoy" pod "caf73a48-2172-46df-9cbc-fd85b49552c1" (UID: "caf73a48-2172-46df-9cbc-fd85b49552c1"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:02:28.146504 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:28.146484    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/caf73a48-2172-46df-9cbc-fd85b49552c1-istio-token" pod "caf73a48-2172-46df-9cbc-fd85b49552c1" (UID: "caf73a48-2172-46df-9cbc-fd85b49552c1"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:02:28.146504 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:28.146496    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/caf73a48-2172-46df-9cbc-fd85b49552c1-istio-podinfo" pod "caf73a48-2172-46df-9cbc-fd85b49552c1" (UID: "caf73a48-2172-46df-9cbc-fd85b49552c1"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 01:02:28.146693 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:28.146485    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/caf73a48-2172-46df-9cbc-fd85b49552c1-kube-api-access-xkn66" pod "caf73a48-2172-46df-9cbc-fd85b49552c1" (UID: "caf73a48-2172-46df-9cbc-fd85b49552c1"). InnerVolumeSpecName "kube-api-access-xkn66". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:02:28.179254 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:28.179202    4284 generic.go:356] "Generic (PLEG): container finished" podID="caf73a48-2172-46df-9cbc-fd85b49552c1" containerID="867fd738ff0fb9c788a891250912b0306b7069ff4157488172a234aa175e56eb" exitCode=0
Oct 31 01:02:28.179705 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:28.179273    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-b647c3-openshift-default-674fcc8445-tvzlw"
Oct 31 01:02:28.179705 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:28.179266    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-b647c3-openshift-default-674fcc8445-tvzlw" event={"ID":"caf73a48-2172-46df-9cbc-fd85b49552c1","Type":"ContainerDied","Data":"867fd738ff0fb9c788a891250912b0306b7069ff4157488172a234aa175e56eb"}
Oct 31 01:02:28.179705 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:28.179408    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-b647c3-openshift-default-674fcc8445-tvzlw" event={"ID":"caf73a48-2172-46df-9cbc-fd85b49552c1","Type":"ContainerDied","Data":"41efd4be8621908e58ede650447224f6712b3dcb0c90423bee2a347e6c8b9ea7"}
Oct 31 01:02:28.179705 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:28.179441    4284 scope.go:122] "RemoveContainer" containerID="867fd738ff0fb9c788a891250912b0306b7069ff4157488172a234aa175e56eb"
Oct 31 01:02:28.224291 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:28.218445    4284 patch_prober.go:29] interesting pod/e2e-api-b-31ae40-openshift-default-66bb6b84bc-8wrm6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.128:15021/healthz/ready\": dial tcp 10.134.0.128:15021: connect: connection refused" start-of-body=
Oct 31 01:02:28.224291 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:28.218539    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-31ae40-openshift-default-66bb6b84bc-8wrm6" podUID="5005c539-d22a-4c99-9235-e08fe509575f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.128:15021/healthz/ready\": dial tcp 10.134.0.128:15021: connect: connection refused"
Oct 31 01:02:28.227202 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:28.227181    4284 scope.go:122] "RemoveContainer" containerID="200c2f3410b7b909a860d413e1b5e3459658ebce88cb879986f4357bffa51f2a"
Oct 31 01:02:28.238880 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:28.238852    4284 scope.go:122] "RemoveContainer" containerID="867fd738ff0fb9c788a891250912b0306b7069ff4157488172a234aa175e56eb"
Oct 31 01:02:28.239270 ip-10-0-132-173 kubenswrapper[4284]: E1031 01:02:28.239247    4284 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"867fd738ff0fb9c788a891250912b0306b7069ff4157488172a234aa175e56eb\": container with ID starting with 867fd738ff0fb9c788a891250912b0306b7069ff4157488172a234aa175e56eb not found: ID does not exist" containerID="867fd738ff0fb9c788a891250912b0306b7069ff4157488172a234aa175e56eb"
Oct 31 01:02:28.239362 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:28.239279    4284 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"867fd738ff0fb9c788a891250912b0306b7069ff4157488172a234aa175e56eb"} err="failed to get container status \"867fd738ff0fb9c788a891250912b0306b7069ff4157488172a234aa175e56eb\": rpc error: code = NotFound desc = could not find container \"867fd738ff0fb9c788a891250912b0306b7069ff4157488172a234aa175e56eb\": container with ID starting with 867fd738ff0fb9c788a891250912b0306b7069ff4157488172a234aa175e56eb not found: ID does not exist"
Oct 31 01:02:28.239362 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:28.239297    4284 scope.go:122] "RemoveContainer" containerID="200c2f3410b7b909a860d413e1b5e3459658ebce88cb879986f4357bffa51f2a"
Oct 31 01:02:28.239725 ip-10-0-132-173 kubenswrapper[4284]: E1031 01:02:28.239707    4284 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"200c2f3410b7b909a860d413e1b5e3459658ebce88cb879986f4357bffa51f2a\": container with ID starting with 200c2f3410b7b909a860d413e1b5e3459658ebce88cb879986f4357bffa51f2a not found: ID does not exist" containerID="200c2f3410b7b909a860d413e1b5e3459658ebce88cb879986f4357bffa51f2a"
Oct 31 01:02:28.239811 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:28.239729    4284 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"200c2f3410b7b909a860d413e1b5e3459658ebce88cb879986f4357bffa51f2a"} err="failed to get container status \"200c2f3410b7b909a860d413e1b5e3459658ebce88cb879986f4357bffa51f2a\": rpc error: code = NotFound desc = could not find container \"200c2f3410b7b909a860d413e1b5e3459658ebce88cb879986f4357bffa51f2a\": container with ID starting with 200c2f3410b7b909a860d413e1b5e3459658ebce88cb879986f4357bffa51f2a not found: ID does not exist"
Oct 31 01:02:28.246564 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:28.246532    4284 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xkn66\" (UniqueName: \"kubernetes.io/projected/caf73a48-2172-46df-9cbc-fd85b49552c1-kube-api-access-xkn66\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:02:28.246696 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:28.246567    4284 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/caf73a48-2172-46df-9cbc-fd85b49552c1-istio-token\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:02:28.246696 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:28.246583    4284 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/caf73a48-2172-46df-9cbc-fd85b49552c1-istio-envoy\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:02:28.246696 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:28.246601    4284 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/caf73a48-2172-46df-9cbc-fd85b49552c1-istio-podinfo\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:02:28.382980 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:28.382921    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-b647c3-openshift-default-674fcc8445-tvzlw"]
Oct 31 01:02:28.409680 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:28.400120    4284 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-a-b647c3-openshift-default-674fcc8445-tvzlw"]
Oct 31 01:02:29.202239 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:29.200331    4284 patch_prober.go:29] interesting pod/e2e-api-b-31ae40-openshift-default-66bb6b84bc-8wrm6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.128:15021/healthz/ready\": dial tcp 10.134.0.128:15021: connect: connection refused" start-of-body=
Oct 31 01:02:29.202239 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:29.200397    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-31ae40-openshift-default-66bb6b84bc-8wrm6" podUID="5005c539-d22a-4c99-9235-e08fe509575f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.128:15021/healthz/ready\": dial tcp 10.134.0.128:15021: connect: connection refused"
Oct 31 01:02:29.426455 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:29.426419    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-ac952a-7795444fc5-jxsgs"]
Oct 31 01:02:29.426660 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:29.426605    4284 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-ac952a-7795444fc5-jxsgs" podUID="221ea88d-8906-49c5-9e00-c94af9002ea8" containerName="maas-api" containerID="cri-o://6e80a29e13171240ad67f7ddd983535cf487e4f68fd4c289bf499aaa7c5523c0" gracePeriod=30
Oct 31 01:02:29.525289 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:29.525168    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:02:29.766227 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:29.766188    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-ac952a-7795444fc5-jxsgs"
Oct 31 01:02:29.837337 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:29.831740    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/221ea88d-8906-49c5-9e00-c94af9002ea8-metrics-certs\" (UniqueName: \"kubernetes.io/secret/221ea88d-8906-49c5-9e00-c94af9002ea8-metrics-certs\") pod \"221ea88d-8906-49c5-9e00-c94af9002ea8\" (UID: \"221ea88d-8906-49c5-9e00-c94af9002ea8\") "
Oct 31 01:02:29.837337 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:29.831865    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/221ea88d-8906-49c5-9e00-c94af9002ea8-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/221ea88d-8906-49c5-9e00-c94af9002ea8-maas-api-tls\") pod \"221ea88d-8906-49c5-9e00-c94af9002ea8\" (UID: \"221ea88d-8906-49c5-9e00-c94af9002ea8\") "
Oct 31 01:02:29.837337 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:29.832038    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/221ea88d-8906-49c5-9e00-c94af9002ea8-kube-api-access-4rbzz\" (UniqueName: \"kubernetes.io/projected/221ea88d-8906-49c5-9e00-c94af9002ea8-kube-api-access-4rbzz\") pod \"221ea88d-8906-49c5-9e00-c94af9002ea8\" (UID: \"221ea88d-8906-49c5-9e00-c94af9002ea8\") "
Oct 31 01:02:29.837337 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:29.832104    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/221ea88d-8906-49c5-9e00-c94af9002ea8-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/221ea88d-8906-49c5-9e00-c94af9002ea8-openshift-service-ca\") pod \"221ea88d-8906-49c5-9e00-c94af9002ea8\" (UID: \"221ea88d-8906-49c5-9e00-c94af9002ea8\") "
Oct 31 01:02:29.868239 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:29.863605    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/221ea88d-8906-49c5-9e00-c94af9002ea8-openshift-service-ca" pod "221ea88d-8906-49c5-9e00-c94af9002ea8" (UID: "221ea88d-8906-49c5-9e00-c94af9002ea8"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 01:02:29.868239 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:29.864649    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/221ea88d-8906-49c5-9e00-c94af9002ea8-kube-api-access-4rbzz" pod "221ea88d-8906-49c5-9e00-c94af9002ea8" (UID: "221ea88d-8906-49c5-9e00-c94af9002ea8"). InnerVolumeSpecName "kube-api-access-4rbzz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:02:29.876844 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:29.876790    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/221ea88d-8906-49c5-9e00-c94af9002ea8-maas-api-tls" pod "221ea88d-8906-49c5-9e00-c94af9002ea8" (UID: "221ea88d-8906-49c5-9e00-c94af9002ea8"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 01:02:29.913662 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:29.913597    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/221ea88d-8906-49c5-9e00-c94af9002ea8-metrics-certs" pod "221ea88d-8906-49c5-9e00-c94af9002ea8" (UID: "221ea88d-8906-49c5-9e00-c94af9002ea8"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 01:02:29.933405 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:29.933337    4284 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/221ea88d-8906-49c5-9e00-c94af9002ea8-metrics-certs\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:02:29.933405 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:29.933362    4284 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/221ea88d-8906-49c5-9e00-c94af9002ea8-maas-api-tls\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:02:29.933405 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:29.933375    4284 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4rbzz\" (UniqueName: \"kubernetes.io/projected/221ea88d-8906-49c5-9e00-c94af9002ea8-kube-api-access-4rbzz\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:02:29.933405 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:29.933385    4284 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/221ea88d-8906-49c5-9e00-c94af9002ea8-openshift-service-ca\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:02:29.956149 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:29.956116    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-ac952a-74bb985447-kbvfk"]
Oct 31 01:02:29.956476 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:29.956448    4284 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-ac952a-74bb985447-kbvfk" podUID="ab8bfde5-7ca3-40a0-b8ec-3a50139d0dd5" containerName="payload-pre-processing" containerID="cri-o://237cadc2421c9dce2ea5bc6b3928a67f404ef1d0ea2a3e72824968da3f009b59" gracePeriod=30
Oct 31 01:02:29.967693 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:29.967667    4284 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="caf73a48-2172-46df-9cbc-fd85b49552c1" path="/var/lib/kubelet/pods/caf73a48-2172-46df-9cbc-fd85b49552c1/volumes"
Oct 31 01:02:30.052845 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:30.052806    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:02:30.204879 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:30.204385    4284 patch_prober.go:29] interesting pod/e2e-api-b-31ae40-openshift-default-66bb6b84bc-8wrm6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.128:15021/healthz/ready\": dial tcp 10.134.0.128:15021: connect: connection refused" start-of-body=
Oct 31 01:02:30.204879 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:30.204445    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-31ae40-openshift-default-66bb6b84bc-8wrm6" podUID="5005c539-d22a-4c99-9235-e08fe509575f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.128:15021/healthz/ready\": dial tcp 10.134.0.128:15021: connect: connection refused"
Oct 31 01:02:30.222418 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:30.222233    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:02:30.238479 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:30.230913    4284 generic.go:356] "Generic (PLEG): container finished" podID="221ea88d-8906-49c5-9e00-c94af9002ea8" containerID="6e80a29e13171240ad67f7ddd983535cf487e4f68fd4c289bf499aaa7c5523c0" exitCode=0
Oct 31 01:02:30.238479 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:30.231106    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-ac952a-7795444fc5-jxsgs" event={"ID":"221ea88d-8906-49c5-9e00-c94af9002ea8","Type":"ContainerDied","Data":"6e80a29e13171240ad67f7ddd983535cf487e4f68fd4c289bf499aaa7c5523c0"}
Oct 31 01:02:30.238479 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:30.231160    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-ac952a-7795444fc5-jxsgs" event={"ID":"221ea88d-8906-49c5-9e00-c94af9002ea8","Type":"ContainerDied","Data":"63035fb902cf7d0f75568b28546abd2135a56515b1733aac89d9add249d76976"}
Oct 31 01:02:30.238479 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:30.231191    4284 scope.go:122] "RemoveContainer" containerID="6e80a29e13171240ad67f7ddd983535cf487e4f68fd4c289bf499aaa7c5523c0"
Oct 31 01:02:30.238479 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:30.231357    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-ac952a-7795444fc5-jxsgs"
Oct 31 01:02:30.238479 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:30.238430    4284 generic.go:356] "Generic (PLEG): container finished" podID="ab8bfde5-7ca3-40a0-b8ec-3a50139d0dd5" containerID="237cadc2421c9dce2ea5bc6b3928a67f404ef1d0ea2a3e72824968da3f009b59" exitCode=0
Oct 31 01:02:30.239200 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:30.238495    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-ac952a-74bb985447-kbvfk" event={"ID":"ab8bfde5-7ca3-40a0-b8ec-3a50139d0dd5","Type":"ContainerDied","Data":"237cadc2421c9dce2ea5bc6b3928a67f404ef1d0ea2a3e72824968da3f009b59"}
Oct 31 01:02:30.249830 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:30.249809    4284 scope.go:122] "RemoveContainer" containerID="6e80a29e13171240ad67f7ddd983535cf487e4f68fd4c289bf499aaa7c5523c0"
Oct 31 01:02:30.250142 ip-10-0-132-173 kubenswrapper[4284]: E1031 01:02:30.250123    4284 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6e80a29e13171240ad67f7ddd983535cf487e4f68fd4c289bf499aaa7c5523c0\": container with ID starting with 6e80a29e13171240ad67f7ddd983535cf487e4f68fd4c289bf499aaa7c5523c0 not found: ID does not exist" containerID="6e80a29e13171240ad67f7ddd983535cf487e4f68fd4c289bf499aaa7c5523c0"
Oct 31 01:02:30.250291 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:30.250150    4284 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6e80a29e13171240ad67f7ddd983535cf487e4f68fd4c289bf499aaa7c5523c0"} err="failed to get container status \"6e80a29e13171240ad67f7ddd983535cf487e4f68fd4c289bf499aaa7c5523c0\": rpc error: code = NotFound desc = could not find container \"6e80a29e13171240ad67f7ddd983535cf487e4f68fd4c289bf499aaa7c5523c0\": container with ID starting with 6e80a29e13171240ad67f7ddd983535cf487e4f68fd4c289bf499aaa7c5523c0 not found: ID does not exist"
Oct 31 01:02:30.299236 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:30.274068    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-ac952a-74bb985447-kbvfk"
Oct 31 01:02:30.338199 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:30.338170    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ab8bfde5-7ca3-40a0-b8ec-3a50139d0dd5-kube-api-access-zq4kw\" (UniqueName: \"kubernetes.io/projected/ab8bfde5-7ca3-40a0-b8ec-3a50139d0dd5-kube-api-access-zq4kw\") pod \"ab8bfde5-7ca3-40a0-b8ec-3a50139d0dd5\" (UID: \"ab8bfde5-7ca3-40a0-b8ec-3a50139d0dd5\") "
Oct 31 01:02:30.338441 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:30.338272    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ab8bfde5-7ca3-40a0-b8ec-3a50139d0dd5-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ab8bfde5-7ca3-40a0-b8ec-3a50139d0dd5-plugins-config-volume\") pod \"ab8bfde5-7ca3-40a0-b8ec-3a50139d0dd5\" (UID: \"ab8bfde5-7ca3-40a0-b8ec-3a50139d0dd5\") "
Oct 31 01:02:30.338919 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:30.338601    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ab8bfde5-7ca3-40a0-b8ec-3a50139d0dd5-plugins-config-volume" pod "ab8bfde5-7ca3-40a0-b8ec-3a50139d0dd5" (UID: "ab8bfde5-7ca3-40a0-b8ec-3a50139d0dd5"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 01:02:30.338919 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:30.338794    4284 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ab8bfde5-7ca3-40a0-b8ec-3a50139d0dd5-plugins-config-volume\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:02:30.340643 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:30.340616    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ab8bfde5-7ca3-40a0-b8ec-3a50139d0dd5-kube-api-access-zq4kw" pod "ab8bfde5-7ca3-40a0-b8ec-3a50139d0dd5" (UID: "ab8bfde5-7ca3-40a0-b8ec-3a50139d0dd5"). InnerVolumeSpecName "kube-api-access-zq4kw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:02:30.518238 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:30.516515    4284 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zq4kw\" (UniqueName: \"kubernetes.io/projected/ab8bfde5-7ca3-40a0-b8ec-3a50139d0dd5-kube-api-access-zq4kw\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:02:30.571271 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:30.570423    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-ac952a-7795444fc5-jxsgs"]
Oct 31 01:02:30.578959 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:30.578919    4284 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-ac952a-7795444fc5-jxsgs"]
Oct 31 01:02:31.208242 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:31.205656    4284 patch_prober.go:29] interesting pod/e2e-api-b-31ae40-openshift-default-66bb6b84bc-8wrm6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.128:15021/healthz/ready\": dial tcp 10.134.0.128:15021: connect: connection refused" start-of-body=
Oct 31 01:02:31.208242 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:31.205720    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-31ae40-openshift-default-66bb6b84bc-8wrm6" podUID="5005c539-d22a-4c99-9235-e08fe509575f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.128:15021/healthz/ready\": dial tcp 10.134.0.128:15021: connect: connection refused"
Oct 31 01:02:31.246877 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:31.246849    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-ac952a-74bb985447-kbvfk"
Oct 31 01:02:31.246877 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:31.246859    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-ac952a-74bb985447-kbvfk" event={"ID":"ab8bfde5-7ca3-40a0-b8ec-3a50139d0dd5","Type":"ContainerDied","Data":"717b12272849aa5599700115ebb35c8ab127d70574a38cbc44689e5f09c2a44e"}
Oct 31 01:02:31.247121 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:31.246911    4284 scope.go:122] "RemoveContainer" containerID="237cadc2421c9dce2ea5bc6b3928a67f404ef1d0ea2a3e72824968da3f009b59"
Oct 31 01:02:31.342042 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:31.342008    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:02:31.427254 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:31.427190    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-ac952a-74bb985447-kbvfk"]
Oct 31 01:02:31.429893 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:31.429863    4284 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-ac952a-74bb985447-kbvfk"]
Oct 31 01:02:31.540568 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:31.540480    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:02:31.702237 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:31.694682    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:02:31.964086 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:31.964047    4284 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="221ea88d-8906-49c5-9e00-c94af9002ea8" path="/var/lib/kubelet/pods/221ea88d-8906-49c5-9e00-c94af9002ea8/volumes"
Oct 31 01:02:31.964557 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:31.964536    4284 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ab8bfde5-7ca3-40a0-b8ec-3a50139d0dd5" path="/var/lib/kubelet/pods/ab8bfde5-7ca3-40a0-b8ec-3a50139d0dd5/volumes"
Oct 31 01:02:32.197476 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:32.197440    4284 patch_prober.go:29] interesting pod/e2e-api-b-31ae40-openshift-default-66bb6b84bc-8wrm6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.128:15021/healthz/ready\": dial tcp 10.134.0.128:15021: connect: connection refused" start-of-body=
Oct 31 01:02:32.197685 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:32.197497    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-31ae40-openshift-default-66bb6b84bc-8wrm6" podUID="5005c539-d22a-4c99-9235-e08fe509575f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.128:15021/healthz/ready\": dial tcp 10.134.0.128:15021: connect: connection refused"
Oct 31 01:02:32.336480 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:32.336385    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:02:33.185574 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:33.185545    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-7ea4b0-55986d8fcc-tc4zl"
Oct 31 01:02:33.201179 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:33.201150    4284 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-31ae40-openshift-default-66bb6b84bc-8wrm6"
Oct 31 01:02:33.201461 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:33.201437    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-31ae40-openshift-default-66bb6b84bc-8wrm6"
Oct 31 01:02:33.202131 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:33.202117    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-31ae40-openshift-default-66bb6b84bc-8wrm6"
Oct 31 01:02:35.226609 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:35.226569    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:02:35.684552 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:35.684511    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:02:36.801954 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:36.801922    4284 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-eb41d3-764494467f-4rfrk"]
Oct 31 01:02:36.802500 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:36.802485    4284 cpu_manager.go:414] "containerMap: removing container" podUID="caf73a48-2172-46df-9cbc-fd85b49552c1" containerName="istio-proxy"
Oct 31 01:02:36.802560 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:36.802502    4284 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="caf73a48-2172-46df-9cbc-fd85b49552c1" containerName="istio-proxy"
Oct 31 01:02:36.802560 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:36.802534    4284 cpu_manager.go:414] "containerMap: removing container" podUID="221ea88d-8906-49c5-9e00-c94af9002ea8" containerName="maas-api"
Oct 31 01:02:36.802560 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:36.802540    4284 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="221ea88d-8906-49c5-9e00-c94af9002ea8" containerName="maas-api"
Oct 31 01:02:36.802560 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:36.802552    4284 cpu_manager.go:414] "containerMap: removing container" podUID="ab8bfde5-7ca3-40a0-b8ec-3a50139d0dd5" containerName="payload-pre-processing"
Oct 31 01:02:36.802560 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:36.802558    4284 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ab8bfde5-7ca3-40a0-b8ec-3a50139d0dd5" containerName="payload-pre-processing"
Oct 31 01:02:36.802726 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:36.802640    4284 memory_manager.go:365] "RemoveStaleState removing state" podUID="caf73a48-2172-46df-9cbc-fd85b49552c1" containerName="istio-proxy"
Oct 31 01:02:36.802726 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:36.802649    4284 memory_manager.go:365] "RemoveStaleState removing state" podUID="caf73a48-2172-46df-9cbc-fd85b49552c1" containerName="istio-proxy"
Oct 31 01:02:36.802726 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:36.802655    4284 memory_manager.go:365] "RemoveStaleState removing state" podUID="221ea88d-8906-49c5-9e00-c94af9002ea8" containerName="maas-api"
Oct 31 01:02:36.802726 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:36.802662    4284 memory_manager.go:365] "RemoveStaleState removing state" podUID="ab8bfde5-7ca3-40a0-b8ec-3a50139d0dd5" containerName="payload-pre-processing"
Oct 31 01:02:36.806154 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:36.806135    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-eb41d3-764494467f-4rfrk"
Oct 31 01:02:36.906634 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:36.906600    4284 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-b-eb41d3-dockercfg-56p96\""
Oct 31 01:02:36.906634 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:36.906624    4284 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-b-eb41d3\""
Oct 31 01:02:36.909675 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:36.909652    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-eb41d3-764494467f-4rfrk"]
Oct 31 01:02:36.920835 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:36.920806    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/36405c36-d9a5-4627-b812-435b5632303b-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-eb41d3-764494467f-4rfrk\" (UID: \"36405c36-d9a5-4627-b812-435b5632303b\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-eb41d3-764494467f-4rfrk"
Oct 31 01:02:36.920975 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:36.920927    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rxwdk\" (UniqueName: \"kubernetes.io/projected/36405c36-d9a5-4627-b812-435b5632303b-kube-api-access-rxwdk\") pod \"payload-pre-processing-e2e-ipp-b-eb41d3-764494467f-4rfrk\" (UID: \"36405c36-d9a5-4627-b812-435b5632303b\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-eb41d3-764494467f-4rfrk"
Oct 31 01:02:37.022054 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:37.022014    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rxwdk\" (UniqueName: \"kubernetes.io/projected/36405c36-d9a5-4627-b812-435b5632303b-kube-api-access-rxwdk\") pod \"payload-pre-processing-e2e-ipp-b-eb41d3-764494467f-4rfrk\" (UID: \"36405c36-d9a5-4627-b812-435b5632303b\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-eb41d3-764494467f-4rfrk"
Oct 31 01:02:37.022362 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:37.022338    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/36405c36-d9a5-4627-b812-435b5632303b-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-eb41d3-764494467f-4rfrk\" (UID: \"36405c36-d9a5-4627-b812-435b5632303b\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-eb41d3-764494467f-4rfrk"
Oct 31 01:02:37.023254 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:37.023232    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/36405c36-d9a5-4627-b812-435b5632303b-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-eb41d3-764494467f-4rfrk\" (UID: \"36405c36-d9a5-4627-b812-435b5632303b\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-eb41d3-764494467f-4rfrk"
Oct 31 01:02:37.104802 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:37.104719    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rxwdk\" (UniqueName: \"kubernetes.io/projected/36405c36-d9a5-4627-b812-435b5632303b-kube-api-access-rxwdk\") pod \"payload-pre-processing-e2e-ipp-b-eb41d3-764494467f-4rfrk\" (UID: \"36405c36-d9a5-4627-b812-435b5632303b\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-eb41d3-764494467f-4rfrk"
Oct 31 01:02:37.119663 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:37.119633    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-eb41d3-764494467f-4rfrk"
Oct 31 01:02:37.298034 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:37.297504    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-ipp-a-ac952a/ipp-route-ac952a-kserve-75c45558d9-fxngk"]
Oct 31 01:02:37.455031 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:37.454997    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:02:37.558990 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:37.558949    4284 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-eb41d3-cbcb48f6b-tcknv"]
Oct 31 01:02:37.559608 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:37.559594    4284 cpu_manager.go:414] "containerMap: removing container" podUID="caf73a48-2172-46df-9cbc-fd85b49552c1" containerName="istio-proxy"
Oct 31 01:02:37.559664 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:37.559611    4284 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="caf73a48-2172-46df-9cbc-fd85b49552c1" containerName="istio-proxy"
Oct 31 01:02:37.562954 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:37.562935    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-eb41d3-cbcb48f6b-tcknv"
Oct 31 01:02:37.593763 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:37.593728    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-eb41d3-764494467f-4rfrk"]
Oct 31 01:02:37.594574 ip-10-0-132-173 kubenswrapper[4284]: W1031 01:02:37.594535    4284 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod36405c36_d9a5_4627_b812_435b5632303b.slice/crio-a3a20cc579d94b040d01b058db75a55c51f849385ba427faa653f87333cff243 WatchSource:0}: Error finding container a3a20cc579d94b040d01b058db75a55c51f849385ba427faa653f87333cff243: Status 404 returned error can't find the container with id a3a20cc579d94b040d01b058db75a55c51f849385ba427faa653f87333cff243
Oct 31 01:02:37.619892 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:37.619753    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-eb41d3-cbcb48f6b-tcknv"]
Oct 31 01:02:37.634983 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:37.634955    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bg5zx\" (UniqueName: \"kubernetes.io/projected/5ac12b4b-3d75-4f4d-866e-6dec6a7e5616-kube-api-access-bg5zx\") pod \"payload-processing-e2e-ipp-b-eb41d3-cbcb48f6b-tcknv\" (UID: \"5ac12b4b-3d75-4f4d-866e-6dec6a7e5616\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-eb41d3-cbcb48f6b-tcknv"
Oct 31 01:02:37.635130 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:37.635005    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5ac12b4b-3d75-4f4d-866e-6dec6a7e5616-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-eb41d3-cbcb48f6b-tcknv\" (UID: \"5ac12b4b-3d75-4f4d-866e-6dec6a7e5616\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-eb41d3-cbcb48f6b-tcknv"
Oct 31 01:02:37.736104 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:37.736020    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bg5zx\" (UniqueName: \"kubernetes.io/projected/5ac12b4b-3d75-4f4d-866e-6dec6a7e5616-kube-api-access-bg5zx\") pod \"payload-processing-e2e-ipp-b-eb41d3-cbcb48f6b-tcknv\" (UID: \"5ac12b4b-3d75-4f4d-866e-6dec6a7e5616\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-eb41d3-cbcb48f6b-tcknv"
Oct 31 01:02:37.736104 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:37.736064    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5ac12b4b-3d75-4f4d-866e-6dec6a7e5616-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-eb41d3-cbcb48f6b-tcknv\" (UID: \"5ac12b4b-3d75-4f4d-866e-6dec6a7e5616\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-eb41d3-cbcb48f6b-tcknv"
Oct 31 01:02:37.736797 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:37.736773    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5ac12b4b-3d75-4f4d-866e-6dec6a7e5616-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-eb41d3-cbcb48f6b-tcknv\" (UID: \"5ac12b4b-3d75-4f4d-866e-6dec6a7e5616\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-eb41d3-cbcb48f6b-tcknv"
Oct 31 01:02:37.796342 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:37.796303    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bg5zx\" (UniqueName: \"kubernetes.io/projected/5ac12b4b-3d75-4f4d-866e-6dec6a7e5616-kube-api-access-bg5zx\") pod \"payload-processing-e2e-ipp-b-eb41d3-cbcb48f6b-tcknv\" (UID: \"5ac12b4b-3d75-4f4d-866e-6dec6a7e5616\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-eb41d3-cbcb48f6b-tcknv"
Oct 31 01:02:37.876531 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:37.876497    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-eb41d3-cbcb48f6b-tcknv"
Oct 31 01:02:38.249163 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:38.249120    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-eb41d3-cbcb48f6b-tcknv"]
Oct 31 01:02:38.249400 ip-10-0-132-173 kubenswrapper[4284]: W1031 01:02:38.249379    4284 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5ac12b4b_3d75_4f4d_866e_6dec6a7e5616.slice/crio-637eee6cb49cb6cabcd713c06ec9080d0695398fdaec30d114a016937c8150b2 WatchSource:0}: Error finding container 637eee6cb49cb6cabcd713c06ec9080d0695398fdaec30d114a016937c8150b2: Status 404 returned error can't find the container with id 637eee6cb49cb6cabcd713c06ec9080d0695398fdaec30d114a016937c8150b2
Oct 31 01:02:38.331003 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:38.330964    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-eb41d3-cbcb48f6b-tcknv" event={"ID":"5ac12b4b-3d75-4f4d-866e-6dec6a7e5616","Type":"ContainerStarted","Data":"2be33e3187b5347719fa8ec5d2c0f1d8b42dfa0139af4dab46f70a5b65ff78e5"}
Oct 31 01:02:38.331158 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:38.331031    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-eb41d3-cbcb48f6b-tcknv" event={"ID":"5ac12b4b-3d75-4f4d-866e-6dec6a7e5616","Type":"ContainerStarted","Data":"637eee6cb49cb6cabcd713c06ec9080d0695398fdaec30d114a016937c8150b2"}
Oct 31 01:02:38.331288 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:38.331263    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-eb41d3-cbcb48f6b-tcknv"
Oct 31 01:02:38.334203 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:38.334175    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-eb41d3-764494467f-4rfrk" event={"ID":"36405c36-d9a5-4627-b812-435b5632303b","Type":"ContainerStarted","Data":"50a2c2423cbd0df04ece969b399bcfaddb8dfe69e88a34ee669b2f2052b51258"}
Oct 31 01:02:38.334370 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:38.334244    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-eb41d3-764494467f-4rfrk" event={"ID":"36405c36-d9a5-4627-b812-435b5632303b","Type":"ContainerStarted","Data":"a3a20cc579d94b040d01b058db75a55c51f849385ba427faa653f87333cff243"}
Oct 31 01:02:38.334370 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:38.334266    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-eb41d3-764494467f-4rfrk"
Oct 31 01:02:38.446436 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:38.446388    4284 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-b-eb41d3-cbcb48f6b-tcknv" podStartSLOduration=1.4463735739999999 podStartE2EDuration="1.446373574s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 01:02:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 01:02:38.446372094 +0000 UTC m=+2072.591379011" watchObservedRunningTime="2026-10-31 01:02:38.446373574 +0000 UTC m=+2072.591380489"
Oct 31 01:02:38.594332 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:38.588832    4284 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-eb41d3-764494467f-4rfrk" podStartSLOduration=2.588811343 podStartE2EDuration="2.588811343s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 01:02:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 01:02:38.569581465 +0000 UTC m=+2072.714588382" watchObservedRunningTime="2026-10-31 01:02:38.588811343 +0000 UTC m=+2072.733818261"
Oct 31 01:02:38.723598 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:38.723562    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:02:43.850239 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:43.850177    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:02:44.645059 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:44.645023    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:02:47.016936 ip-10-0-132-173 kubenswrapper[4284]: E1031 01:02:47.016894    4284 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fd8sp" containerName="extractor"
Oct 31 01:02:49.104943 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:49.104901    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:02:49.342288 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:49.342258    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-eb41d3-cbcb48f6b-tcknv"
Oct 31 01:02:49.342477 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:49.342326    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-eb41d3-764494467f-4rfrk"
Oct 31 01:02:52.333071 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:52.333034    4284 kuberuntime_container.go:898] "Killing container with a grace period" pod="ai-tenant-e2e-ipp-a-ac952a/ipp-route-ac952a-kserve-75c45558d9-fxngk" podUID="5e194b3e-7d44-437b-a50a-0b64a656d855" containerName="main" containerID="cri-o://b2d1d7e11c0337b0f222660ff04e257e1fc42c322f8986c7c84b7a9029463657" gracePeriod=45
Oct 31 01:02:52.586708 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:52.586651    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-ac952a/ipp-route-ac952a-kserve-75c45558d9-fxngk"
Oct 31 01:02:52.730459 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:52.730414    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5e194b3e-7d44-437b-a50a-0b64a656d855-dshm\" (UniqueName: \"kubernetes.io/empty-dir/5e194b3e-7d44-437b-a50a-0b64a656d855-dshm\") pod \"5e194b3e-7d44-437b-a50a-0b64a656d855\" (UID: \"5e194b3e-7d44-437b-a50a-0b64a656d855\") "
Oct 31 01:02:52.730655 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:52.730548    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5e194b3e-7d44-437b-a50a-0b64a656d855-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5e194b3e-7d44-437b-a50a-0b64a656d855-model-cache\") pod \"5e194b3e-7d44-437b-a50a-0b64a656d855\" (UID: \"5e194b3e-7d44-437b-a50a-0b64a656d855\") "
Oct 31 01:02:52.730655 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:52.730602    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5e194b3e-7d44-437b-a50a-0b64a656d855-kube-api-access-jjxvg\" (UniqueName: \"kubernetes.io/projected/5e194b3e-7d44-437b-a50a-0b64a656d855-kube-api-access-jjxvg\") pod \"5e194b3e-7d44-437b-a50a-0b64a656d855\" (UID: \"5e194b3e-7d44-437b-a50a-0b64a656d855\") "
Oct 31 01:02:52.730783 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:52.730706    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5e194b3e-7d44-437b-a50a-0b64a656d855-home\" (UniqueName: \"kubernetes.io/empty-dir/5e194b3e-7d44-437b-a50a-0b64a656d855-home\") pod \"5e194b3e-7d44-437b-a50a-0b64a656d855\" (UID: \"5e194b3e-7d44-437b-a50a-0b64a656d855\") "
Oct 31 01:02:52.730783 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:52.730776    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5e194b3e-7d44-437b-a50a-0b64a656d855-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5e194b3e-7d44-437b-a50a-0b64a656d855-tmp-dir\") pod \"5e194b3e-7d44-437b-a50a-0b64a656d855\" (UID: \"5e194b3e-7d44-437b-a50a-0b64a656d855\") "
Oct 31 01:02:52.730946 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:52.730925    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/5e194b3e-7d44-437b-a50a-0b64a656d855-tls-certs\" (UniqueName: \"kubernetes.io/secret/5e194b3e-7d44-437b-a50a-0b64a656d855-tls-certs\") pod \"5e194b3e-7d44-437b-a50a-0b64a656d855\" (UID: \"5e194b3e-7d44-437b-a50a-0b64a656d855\") "
Oct 31 01:02:52.731764 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:52.731083    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5e194b3e-7d44-437b-a50a-0b64a656d855-tmp-dir" pod "5e194b3e-7d44-437b-a50a-0b64a656d855" (UID: "5e194b3e-7d44-437b-a50a-0b64a656d855"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:02:52.731764 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:52.731396    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5e194b3e-7d44-437b-a50a-0b64a656d855-model-cache" pod "5e194b3e-7d44-437b-a50a-0b64a656d855" (UID: "5e194b3e-7d44-437b-a50a-0b64a656d855"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:02:52.731764 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:52.731660    4284 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5e194b3e-7d44-437b-a50a-0b64a656d855-tmp-dir\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:02:52.731764 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:52.731735    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5e194b3e-7d44-437b-a50a-0b64a656d855-home" pod "5e194b3e-7d44-437b-a50a-0b64a656d855" (UID: "5e194b3e-7d44-437b-a50a-0b64a656d855"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:02:52.733789 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:52.733673    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5e194b3e-7d44-437b-a50a-0b64a656d855-tls-certs" pod "5e194b3e-7d44-437b-a50a-0b64a656d855" (UID: "5e194b3e-7d44-437b-a50a-0b64a656d855"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 01:02:52.734191 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:52.734166    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5e194b3e-7d44-437b-a50a-0b64a656d855-kube-api-access-jjxvg" pod "5e194b3e-7d44-437b-a50a-0b64a656d855" (UID: "5e194b3e-7d44-437b-a50a-0b64a656d855"). InnerVolumeSpecName "kube-api-access-jjxvg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:02:52.734342 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:52.734308    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5e194b3e-7d44-437b-a50a-0b64a656d855-dshm" pod "5e194b3e-7d44-437b-a50a-0b64a656d855" (UID: "5e194b3e-7d44-437b-a50a-0b64a656d855"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:02:52.833110 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:52.833076    4284 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5e194b3e-7d44-437b-a50a-0b64a656d855-model-cache\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:02:52.833110 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:52.833110    4284 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jjxvg\" (UniqueName: \"kubernetes.io/projected/5e194b3e-7d44-437b-a50a-0b64a656d855-kube-api-access-jjxvg\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:02:52.833366 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:52.833125    4284 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5e194b3e-7d44-437b-a50a-0b64a656d855-home\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:02:52.833366 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:52.833138    4284 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5e194b3e-7d44-437b-a50a-0b64a656d855-tls-certs\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:02:52.833366 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:52.833150    4284 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5e194b3e-7d44-437b-a50a-0b64a656d855-dshm\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:02:53.430576 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:53.430534    4284 generic.go:356] "Generic (PLEG): container finished" podID="5e194b3e-7d44-437b-a50a-0b64a656d855" containerID="b2d1d7e11c0337b0f222660ff04e257e1fc42c322f8986c7c84b7a9029463657" exitCode=0
Oct 31 01:02:53.431008 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:53.430582    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-ac952a/ipp-route-ac952a-kserve-75c45558d9-fxngk"
Oct 31 01:02:53.431008 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:53.430601    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-ac952a/ipp-route-ac952a-kserve-75c45558d9-fxngk" event={"ID":"5e194b3e-7d44-437b-a50a-0b64a656d855","Type":"ContainerDied","Data":"b2d1d7e11c0337b0f222660ff04e257e1fc42c322f8986c7c84b7a9029463657"}
Oct 31 01:02:53.431008 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:53.430638    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-ac952a/ipp-route-ac952a-kserve-75c45558d9-fxngk" event={"ID":"5e194b3e-7d44-437b-a50a-0b64a656d855","Type":"ContainerDied","Data":"4d3056937dc8374606e09d68c8b5bbfdc28ab999a6b7d17df7c3b8282cfc97f1"}
Oct 31 01:02:53.431008 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:53.430656    4284 scope.go:122] "RemoveContainer" containerID="b2d1d7e11c0337b0f222660ff04e257e1fc42c322f8986c7c84b7a9029463657"
Oct 31 01:02:53.440987 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:53.440966    4284 scope.go:122] "RemoveContainer" containerID="b2d1d7e11c0337b0f222660ff04e257e1fc42c322f8986c7c84b7a9029463657"
Oct 31 01:02:53.441394 ip-10-0-132-173 kubenswrapper[4284]: E1031 01:02:53.441373    4284 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b2d1d7e11c0337b0f222660ff04e257e1fc42c322f8986c7c84b7a9029463657\": container with ID starting with b2d1d7e11c0337b0f222660ff04e257e1fc42c322f8986c7c84b7a9029463657 not found: ID does not exist" containerID="b2d1d7e11c0337b0f222660ff04e257e1fc42c322f8986c7c84b7a9029463657"
Oct 31 01:02:53.441475 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:53.441406    4284 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b2d1d7e11c0337b0f222660ff04e257e1fc42c322f8986c7c84b7a9029463657"} err="failed to get container status \"b2d1d7e11c0337b0f222660ff04e257e1fc42c322f8986c7c84b7a9029463657\": rpc error: code = NotFound desc = could not find container \"b2d1d7e11c0337b0f222660ff04e257e1fc42c322f8986c7c84b7a9029463657\": container with ID starting with b2d1d7e11c0337b0f222660ff04e257e1fc42c322f8986c7c84b7a9029463657 not found: ID does not exist"
Oct 31 01:02:53.589000 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:53.588969    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-ipp-a-ac952a/ipp-route-ac952a-kserve-75c45558d9-fxngk"]
Oct 31 01:02:53.636778 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:53.636746    4284 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-ipp-a-ac952a/ipp-route-ac952a-kserve-75c45558d9-fxngk"]
Oct 31 01:02:53.964106 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:53.964074    4284 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5e194b3e-7d44-437b-a50a-0b64a656d855" path="/var/lib/kubelet/pods/5e194b3e-7d44-437b-a50a-0b64a656d855/volumes"
Oct 31 01:02:54.015327 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:02:54.015292    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:03:00.386609 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:00.386571    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:03:00.638027 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:00.637941    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:03:05.298918 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:05.298880    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:03:10.191563 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:10.191517    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:03:11.673377 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:11.673336    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:03:13.576431 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:13.576399    4284 generic.go:356] "Generic (PLEG): container finished" podID="4b7ffc18-b208-499f-afed-cb71c96259e4" containerID="1acc08efa1ca567c1520e2bce6f783ec39e030c89612915e86fa5ddc750b6eea" exitCode=0
Oct 31 01:03:13.576928 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:13.576467    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-a-7ea4b0-openshift-default-65c5776677-nw4cq" event={"ID":"4b7ffc18-b208-499f-afed-cb71c96259e4","Type":"ContainerDied","Data":"1acc08efa1ca567c1520e2bce6f783ec39e030c89612915e86fa5ddc750b6eea"}
Oct 31 01:03:13.576928 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:13.576915    4284 scope.go:122] "RemoveContainer" containerID="1acc08efa1ca567c1520e2bce6f783ec39e030c89612915e86fa5ddc750b6eea"
Oct 31 01:03:13.578483 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:13.578453    4284 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 01:03:13.578599 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:13.578535    4284 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 01:03:13.578599 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:13.578586    4284 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 01:03:13.668284 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:13.668256    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:03:14.583959 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:14.583928    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-a-7ea4b0-openshift-default-65c5776677-nw4cq" event={"ID":"4b7ffc18-b208-499f-afed-cb71c96259e4","Type":"ContainerStarted","Data":"9944fd5115259a76279c2ea42ed37df527f8a384919631ff616c21cd0715148b"}
Oct 31 01:03:15.032569 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:15.032463    4284 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-praxis-a-7ea4b0-openshift-default-65c5776677-nw4cq"
Oct 31 01:03:15.032769 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:15.032653    4284 patch_prober.go:29] interesting pod/e2e-praxis-a-7ea4b0-openshift-default-65c5776677-nw4cq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.129:15021/healthz/ready\": dial tcp 10.134.0.129:15021: connect: connection refused" start-of-body=
Oct 31 01:03:15.032769 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:15.032704    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-a-7ea4b0-openshift-default-65c5776677-nw4cq" podUID="4b7ffc18-b208-499f-afed-cb71c96259e4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.129:15021/healthz/ready\": dial tcp 10.134.0.129:15021: connect: connection refused"
Oct 31 01:03:15.683319 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:15.683285    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:03:16.034538 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:16.034070    4284 patch_prober.go:29] interesting pod/e2e-praxis-a-7ea4b0-openshift-default-65c5776677-nw4cq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.129:15021/healthz/ready\": dial tcp 10.134.0.129:15021: connect: connection refused" start-of-body=
Oct 31 01:03:16.034538 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:16.034156    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-a-7ea4b0-openshift-default-65c5776677-nw4cq" podUID="4b7ffc18-b208-499f-afed-cb71c96259e4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.129:15021/healthz/ready\": dial tcp 10.134.0.129:15021: connect: connection refused"
Oct 31 01:03:16.061158 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:16.061122    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:03:17.032198 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:17.032112    4284 patch_prober.go:29] interesting pod/e2e-praxis-a-7ea4b0-openshift-default-65c5776677-nw4cq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.129:15021/healthz/ready\": dial tcp 10.134.0.129:15021: connect: connection refused" start-of-body=
Oct 31 01:03:17.032198 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:17.032168    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-a-7ea4b0-openshift-default-65c5776677-nw4cq" podUID="4b7ffc18-b208-499f-afed-cb71c96259e4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.129:15021/healthz/ready\": dial tcp 10.134.0.129:15021: connect: connection refused"
Oct 31 01:03:17.036751 ip-10-0-132-173 kubenswrapper[4284]: E1031 01:03:17.036721    4284 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fd8sp" containerName="extractor"
Oct 31 01:03:18.032309 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:18.032275    4284 patch_prober.go:29] interesting pod/e2e-praxis-a-7ea4b0-openshift-default-65c5776677-nw4cq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.129:15021/healthz/ready\": dial tcp 10.134.0.129:15021: connect: connection refused" start-of-body=
Oct 31 01:03:18.032743 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:18.032354    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-a-7ea4b0-openshift-default-65c5776677-nw4cq" podUID="4b7ffc18-b208-499f-afed-cb71c96259e4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.129:15021/healthz/ready\": dial tcp 10.134.0.129:15021: connect: connection refused"
Oct 31 01:03:19.032226 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:19.032176    4284 patch_prober.go:29] interesting pod/e2e-praxis-a-7ea4b0-openshift-default-65c5776677-nw4cq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.129:15021/healthz/ready\": dial tcp 10.134.0.129:15021: connect: connection refused" start-of-body=
Oct 31 01:03:19.032410 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:19.032243    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-a-7ea4b0-openshift-default-65c5776677-nw4cq" podUID="4b7ffc18-b208-499f-afed-cb71c96259e4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.129:15021/healthz/ready\": dial tcp 10.134.0.129:15021: connect: connection refused"
Oct 31 01:03:20.037284 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:20.037252    4284 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-praxis-a-7ea4b0-openshift-default-65c5776677-nw4cq"
Oct 31 01:03:20.037698 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:20.037653    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-praxis-a-7ea4b0-openshift-default-65c5776677-nw4cq"
Oct 31 01:03:20.038483 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:20.038459    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-praxis-a-7ea4b0-openshift-default-65c5776677-nw4cq"
Oct 31 01:03:20.879177 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:20.879141    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:03:21.170727 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:21.170675    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:03:21.368848 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:21.368816    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:03:21.552759 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:21.552658    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:03:25.579816 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:25.579775    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:03:25.872485 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:25.872412    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-eee7f1-77669c5769-fr9zj"]
Oct 31 01:03:26.224096 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:26.224056    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:03:26.385296 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:26.385262    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:03:30.752239 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:30.749290    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:03:30.873306 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:30.873252    4284 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-a-eee7f1-77669c5769-fr9zj" podUID="33b97842-74a9-4d0b-9887-d65273b2bbf8" containerName="payload-processing" containerID="cri-o://03cc268f1ad87ae4d82e4cd8f130cae4be152c9e7118c1cc91cd9f9b540b63c7" gracePeriod=25
Oct 31 01:03:31.142517 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:31.142485    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-eee7f1-77669c5769-fr9zj"
Oct 31 01:03:31.171595 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:31.171565    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/33b97842-74a9-4d0b-9887-d65273b2bbf8-kube-api-access-pqmxs\" (UniqueName: \"kubernetes.io/projected/33b97842-74a9-4d0b-9887-d65273b2bbf8-kube-api-access-pqmxs\") pod \"33b97842-74a9-4d0b-9887-d65273b2bbf8\" (UID: \"33b97842-74a9-4d0b-9887-d65273b2bbf8\") "
Oct 31 01:03:31.171766 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:31.171724    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/33b97842-74a9-4d0b-9887-d65273b2bbf8-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/33b97842-74a9-4d0b-9887-d65273b2bbf8-plugins-config-volume\") pod \"33b97842-74a9-4d0b-9887-d65273b2bbf8\" (UID: \"33b97842-74a9-4d0b-9887-d65273b2bbf8\") "
Oct 31 01:03:31.172076 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:31.172053    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/33b97842-74a9-4d0b-9887-d65273b2bbf8-plugins-config-volume" pod "33b97842-74a9-4d0b-9887-d65273b2bbf8" (UID: "33b97842-74a9-4d0b-9887-d65273b2bbf8"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 01:03:31.173932 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:31.173911    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/33b97842-74a9-4d0b-9887-d65273b2bbf8-kube-api-access-pqmxs" pod "33b97842-74a9-4d0b-9887-d65273b2bbf8" (UID: "33b97842-74a9-4d0b-9887-d65273b2bbf8"). InnerVolumeSpecName "kube-api-access-pqmxs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:03:31.273356 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:31.273303    4284 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/33b97842-74a9-4d0b-9887-d65273b2bbf8-plugins-config-volume\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:03:31.273356 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:31.273357    4284 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pqmxs\" (UniqueName: \"kubernetes.io/projected/33b97842-74a9-4d0b-9887-d65273b2bbf8-kube-api-access-pqmxs\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:03:31.771717 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:31.771688    4284 generic.go:356] "Generic (PLEG): container finished" podID="33b97842-74a9-4d0b-9887-d65273b2bbf8" containerID="03cc268f1ad87ae4d82e4cd8f130cae4be152c9e7118c1cc91cd9f9b540b63c7" exitCode=0
Oct 31 01:03:31.772179 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:31.771735    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-eee7f1-77669c5769-fr9zj"
Oct 31 01:03:31.772179 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:31.771787    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-eee7f1-77669c5769-fr9zj" event={"ID":"33b97842-74a9-4d0b-9887-d65273b2bbf8","Type":"ContainerDied","Data":"03cc268f1ad87ae4d82e4cd8f130cae4be152c9e7118c1cc91cd9f9b540b63c7"}
Oct 31 01:03:31.772179 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:31.771839    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-eee7f1-77669c5769-fr9zj" event={"ID":"33b97842-74a9-4d0b-9887-d65273b2bbf8","Type":"ContainerDied","Data":"a9c75c143347edbfa7404596281ec4059baf781d68e3edd20cad7fb6a076dc83"}
Oct 31 01:03:31.772179 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:31.771861    4284 scope.go:122] "RemoveContainer" containerID="03cc268f1ad87ae4d82e4cd8f130cae4be152c9e7118c1cc91cd9f9b540b63c7"
Oct 31 01:03:31.782477 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:31.782462    4284 scope.go:122] "RemoveContainer" containerID="03cc268f1ad87ae4d82e4cd8f130cae4be152c9e7118c1cc91cd9f9b540b63c7"
Oct 31 01:03:31.782768 ip-10-0-132-173 kubenswrapper[4284]: E1031 01:03:31.782749    4284 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"03cc268f1ad87ae4d82e4cd8f130cae4be152c9e7118c1cc91cd9f9b540b63c7\": container with ID starting with 03cc268f1ad87ae4d82e4cd8f130cae4be152c9e7118c1cc91cd9f9b540b63c7 not found: ID does not exist" containerID="03cc268f1ad87ae4d82e4cd8f130cae4be152c9e7118c1cc91cd9f9b540b63c7"
Oct 31 01:03:31.782816 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:31.782776    4284 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"03cc268f1ad87ae4d82e4cd8f130cae4be152c9e7118c1cc91cd9f9b540b63c7"} err="failed to get container status \"03cc268f1ad87ae4d82e4cd8f130cae4be152c9e7118c1cc91cd9f9b540b63c7\": rpc error: code = NotFound desc = could not find container \"03cc268f1ad87ae4d82e4cd8f130cae4be152c9e7118c1cc91cd9f9b540b63c7\": container with ID starting with 03cc268f1ad87ae4d82e4cd8f130cae4be152c9e7118c1cc91cd9f9b540b63c7 not found: ID does not exist"
Oct 31 01:03:31.933886 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:31.933849    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-eee7f1-77669c5769-fr9zj"]
Oct 31 01:03:31.981243 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:31.981191    4284 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-eee7f1-77669c5769-fr9zj"]
Oct 31 01:03:33.967777 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:33.967741    4284 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="33b97842-74a9-4d0b-9887-d65273b2bbf8" path="/var/lib/kubelet/pods/33b97842-74a9-4d0b-9887-d65273b2bbf8/volumes"
Oct 31 01:03:35.750065 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:35.747325    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:03:39.664396 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:39.664359    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:03:39.836003 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:39.835970    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:03:40.075036 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:40.074783    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:03:40.425805 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:40.425773    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-eb41d3-cbcb48f6b-tcknv"]
Oct 31 01:03:40.425997 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:40.425975    4284 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-b-eb41d3-cbcb48f6b-tcknv" podUID="5ac12b4b-3d75-4f4d-866e-6dec6a7e5616" containerName="payload-processing" containerID="cri-o://2be33e3187b5347719fa8ec5d2c0f1d8b42dfa0139af4dab46f70a5b65ff78e5" gracePeriod=30
Oct 31 01:03:40.530540 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:40.530501    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-eb41d3-764494467f-4rfrk"]
Oct 31 01:03:40.530782 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:40.530752    4284 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-eb41d3-764494467f-4rfrk" podUID="36405c36-d9a5-4627-b812-435b5632303b" containerName="payload-pre-processing" containerID="cri-o://50a2c2423cbd0df04ece969b399bcfaddb8dfe69e88a34ee669b2f2052b51258" gracePeriod=30
Oct 31 01:03:40.589250 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:40.587108    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:03:40.720883 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:40.720857    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-eb41d3-cbcb48f6b-tcknv"
Oct 31 01:03:40.740845 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:40.740809    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:03:40.788801 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:40.788773    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-eb41d3-764494467f-4rfrk"
Oct 31 01:03:40.819961 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:40.819926    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5ac12b4b-3d75-4f4d-866e-6dec6a7e5616-kube-api-access-bg5zx\" (UniqueName: \"kubernetes.io/projected/5ac12b4b-3d75-4f4d-866e-6dec6a7e5616-kube-api-access-bg5zx\") pod \"5ac12b4b-3d75-4f4d-866e-6dec6a7e5616\" (UID: \"5ac12b4b-3d75-4f4d-866e-6dec6a7e5616\") "
Oct 31 01:03:40.820163 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:40.820065    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/5ac12b4b-3d75-4f4d-866e-6dec6a7e5616-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5ac12b4b-3d75-4f4d-866e-6dec6a7e5616-plugins-config-volume\") pod \"5ac12b4b-3d75-4f4d-866e-6dec6a7e5616\" (UID: \"5ac12b4b-3d75-4f4d-866e-6dec6a7e5616\") "
Oct 31 01:03:40.820495 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:40.820468    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5ac12b4b-3d75-4f4d-866e-6dec6a7e5616-plugins-config-volume" pod "5ac12b4b-3d75-4f4d-866e-6dec6a7e5616" (UID: "5ac12b4b-3d75-4f4d-866e-6dec6a7e5616"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 01:03:40.822114 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:40.822085    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5ac12b4b-3d75-4f4d-866e-6dec6a7e5616-kube-api-access-bg5zx" pod "5ac12b4b-3d75-4f4d-866e-6dec6a7e5616" (UID: "5ac12b4b-3d75-4f4d-866e-6dec6a7e5616"). InnerVolumeSpecName "kube-api-access-bg5zx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:03:40.829539 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:40.829508    4284 generic.go:356] "Generic (PLEG): container finished" podID="5ac12b4b-3d75-4f4d-866e-6dec6a7e5616" containerID="2be33e3187b5347719fa8ec5d2c0f1d8b42dfa0139af4dab46f70a5b65ff78e5" exitCode=0
Oct 31 01:03:40.829680 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:40.829553    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-eb41d3-cbcb48f6b-tcknv"
Oct 31 01:03:40.829680 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:40.829593    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-eb41d3-cbcb48f6b-tcknv" event={"ID":"5ac12b4b-3d75-4f4d-866e-6dec6a7e5616","Type":"ContainerDied","Data":"2be33e3187b5347719fa8ec5d2c0f1d8b42dfa0139af4dab46f70a5b65ff78e5"}
Oct 31 01:03:40.829680 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:40.829649    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-eb41d3-cbcb48f6b-tcknv" event={"ID":"5ac12b4b-3d75-4f4d-866e-6dec6a7e5616","Type":"ContainerDied","Data":"637eee6cb49cb6cabcd713c06ec9080d0695398fdaec30d114a016937c8150b2"}
Oct 31 01:03:40.829812 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:40.829682    4284 scope.go:122] "RemoveContainer" containerID="2be33e3187b5347719fa8ec5d2c0f1d8b42dfa0139af4dab46f70a5b65ff78e5"
Oct 31 01:03:40.831127 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:40.831105    4284 generic.go:356] "Generic (PLEG): container finished" podID="36405c36-d9a5-4627-b812-435b5632303b" containerID="50a2c2423cbd0df04ece969b399bcfaddb8dfe69e88a34ee669b2f2052b51258" exitCode=0
Oct 31 01:03:40.831275 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:40.831160    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-eb41d3-764494467f-4rfrk"
Oct 31 01:03:40.831275 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:40.831173    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-eb41d3-764494467f-4rfrk" event={"ID":"36405c36-d9a5-4627-b812-435b5632303b","Type":"ContainerDied","Data":"50a2c2423cbd0df04ece969b399bcfaddb8dfe69e88a34ee669b2f2052b51258"}
Oct 31 01:03:40.831275 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:40.831261    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-eb41d3-764494467f-4rfrk" event={"ID":"36405c36-d9a5-4627-b812-435b5632303b","Type":"ContainerDied","Data":"a3a20cc579d94b040d01b058db75a55c51f849385ba427faa653f87333cff243"}
Oct 31 01:03:40.841333 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:40.841304    4284 scope.go:122] "RemoveContainer" containerID="2be33e3187b5347719fa8ec5d2c0f1d8b42dfa0139af4dab46f70a5b65ff78e5"
Oct 31 01:03:40.842514 ip-10-0-132-173 kubenswrapper[4284]: E1031 01:03:40.842488    4284 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2be33e3187b5347719fa8ec5d2c0f1d8b42dfa0139af4dab46f70a5b65ff78e5\": container with ID starting with 2be33e3187b5347719fa8ec5d2c0f1d8b42dfa0139af4dab46f70a5b65ff78e5 not found: ID does not exist" containerID="2be33e3187b5347719fa8ec5d2c0f1d8b42dfa0139af4dab46f70a5b65ff78e5"
Oct 31 01:03:40.842610 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:40.842525    4284 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2be33e3187b5347719fa8ec5d2c0f1d8b42dfa0139af4dab46f70a5b65ff78e5"} err="failed to get container status \"2be33e3187b5347719fa8ec5d2c0f1d8b42dfa0139af4dab46f70a5b65ff78e5\": rpc error: code = NotFound desc = could not find container \"2be33e3187b5347719fa8ec5d2c0f1d8b42dfa0139af4dab46f70a5b65ff78e5\": container with ID starting with 2be33e3187b5347719fa8ec5d2c0f1d8b42dfa0139af4dab46f70a5b65ff78e5 not found: ID does not exist"
Oct 31 01:03:40.842610 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:40.842550    4284 scope.go:122] "RemoveContainer" containerID="50a2c2423cbd0df04ece969b399bcfaddb8dfe69e88a34ee669b2f2052b51258"
Oct 31 01:03:40.853861 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:40.853833    4284 scope.go:122] "RemoveContainer" containerID="50a2c2423cbd0df04ece969b399bcfaddb8dfe69e88a34ee669b2f2052b51258"
Oct 31 01:03:40.854251 ip-10-0-132-173 kubenswrapper[4284]: E1031 01:03:40.854191    4284 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"50a2c2423cbd0df04ece969b399bcfaddb8dfe69e88a34ee669b2f2052b51258\": container with ID starting with 50a2c2423cbd0df04ece969b399bcfaddb8dfe69e88a34ee669b2f2052b51258 not found: ID does not exist" containerID="50a2c2423cbd0df04ece969b399bcfaddb8dfe69e88a34ee669b2f2052b51258"
Oct 31 01:03:40.854357 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:40.854261    4284 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"50a2c2423cbd0df04ece969b399bcfaddb8dfe69e88a34ee669b2f2052b51258"} err="failed to get container status \"50a2c2423cbd0df04ece969b399bcfaddb8dfe69e88a34ee669b2f2052b51258\": rpc error: code = NotFound desc = could not find container \"50a2c2423cbd0df04ece969b399bcfaddb8dfe69e88a34ee669b2f2052b51258\": container with ID starting with 50a2c2423cbd0df04ece969b399bcfaddb8dfe69e88a34ee669b2f2052b51258 not found: ID does not exist"
Oct 31 01:03:40.921623 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:40.921576    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/36405c36-d9a5-4627-b812-435b5632303b-kube-api-access-rxwdk\" (UniqueName: \"kubernetes.io/projected/36405c36-d9a5-4627-b812-435b5632303b-kube-api-access-rxwdk\") pod \"36405c36-d9a5-4627-b812-435b5632303b\" (UID: \"36405c36-d9a5-4627-b812-435b5632303b\") "
Oct 31 01:03:40.921868 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:40.921838    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/36405c36-d9a5-4627-b812-435b5632303b-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/36405c36-d9a5-4627-b812-435b5632303b-plugins-config-volume\") pod \"36405c36-d9a5-4627-b812-435b5632303b\" (UID: \"36405c36-d9a5-4627-b812-435b5632303b\") "
Oct 31 01:03:40.922331 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:40.922294    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/36405c36-d9a5-4627-b812-435b5632303b-plugins-config-volume" pod "36405c36-d9a5-4627-b812-435b5632303b" (UID: "36405c36-d9a5-4627-b812-435b5632303b"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 01:03:40.922469 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:40.922391    4284 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bg5zx\" (UniqueName: \"kubernetes.io/projected/5ac12b4b-3d75-4f4d-866e-6dec6a7e5616-kube-api-access-bg5zx\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:03:40.922469 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:40.922410    4284 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/36405c36-d9a5-4627-b812-435b5632303b-plugins-config-volume\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:03:40.922469 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:40.922419    4284 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5ac12b4b-3d75-4f4d-866e-6dec6a7e5616-plugins-config-volume\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:03:40.924084 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:40.924056    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/36405c36-d9a5-4627-b812-435b5632303b-kube-api-access-rxwdk" pod "36405c36-d9a5-4627-b812-435b5632303b" (UID: "36405c36-d9a5-4627-b812-435b5632303b"). InnerVolumeSpecName "kube-api-access-rxwdk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:03:41.028301 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:41.026311    4284 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rxwdk\" (UniqueName: \"kubernetes.io/projected/36405c36-d9a5-4627-b812-435b5632303b-kube-api-access-rxwdk\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:03:41.180184 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:41.180145    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-eb41d3-cbcb48f6b-tcknv"]
Oct 31 01:03:41.191907 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:41.191875    4284 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-eb41d3-cbcb48f6b-tcknv"]
Oct 31 01:03:41.300968 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:41.300875    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-eb41d3-764494467f-4rfrk"]
Oct 31 01:03:41.308781 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:41.308753    4284 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-eb41d3-764494467f-4rfrk"]
Oct 31 01:03:41.744056 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:41.744016    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:03:41.963428 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:41.963396    4284 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="36405c36-d9a5-4627-b812-435b5632303b" path="/var/lib/kubelet/pods/36405c36-d9a5-4627-b812-435b5632303b/volumes"
Oct 31 01:03:41.963821 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:41.963808    4284 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5ac12b4b-3d75-4f4d-866e-6dec6a7e5616" path="/var/lib/kubelet/pods/5ac12b4b-3d75-4f4d-866e-6dec6a7e5616/volumes"
Oct 31 01:03:41.993802 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:41.993767    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:03:42.186176 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:42.186126    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:03:42.694672 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:42.694638    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:03:44.141226 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:44.141170    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:03:45.439409 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:45.439373    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:03:46.651535 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:46.651500    4284 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-praxis-b-4ab4aa-openshift-default-799d6fd5-pqg68"]
Oct 31 01:03:46.652149 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:46.652059    4284 cpu_manager.go:414] "containerMap: removing container" podUID="5e194b3e-7d44-437b-a50a-0b64a656d855" containerName="main"
Oct 31 01:03:46.652149 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:46.652072    4284 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5e194b3e-7d44-437b-a50a-0b64a656d855" containerName="main"
Oct 31 01:03:46.652149 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:46.652094    4284 cpu_manager.go:414] "containerMap: removing container" podUID="33b97842-74a9-4d0b-9887-d65273b2bbf8" containerName="payload-processing"
Oct 31 01:03:46.652149 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:46.652099    4284 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="33b97842-74a9-4d0b-9887-d65273b2bbf8" containerName="payload-processing"
Oct 31 01:03:46.652149 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:46.652111    4284 cpu_manager.go:414] "containerMap: removing container" podUID="36405c36-d9a5-4627-b812-435b5632303b" containerName="payload-pre-processing"
Oct 31 01:03:46.652149 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:46.652116    4284 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="36405c36-d9a5-4627-b812-435b5632303b" containerName="payload-pre-processing"
Oct 31 01:03:46.652149 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:46.652124    4284 cpu_manager.go:414] "containerMap: removing container" podUID="5ac12b4b-3d75-4f4d-866e-6dec6a7e5616" containerName="payload-processing"
Oct 31 01:03:46.652149 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:46.652129    4284 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5ac12b4b-3d75-4f4d-866e-6dec6a7e5616" containerName="payload-processing"
Oct 31 01:03:46.652598 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:46.652201    4284 memory_manager.go:365] "RemoveStaleState removing state" podUID="5ac12b4b-3d75-4f4d-866e-6dec6a7e5616" containerName="payload-processing"
Oct 31 01:03:46.652598 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:46.652237    4284 memory_manager.go:365] "RemoveStaleState removing state" podUID="5e194b3e-7d44-437b-a50a-0b64a656d855" containerName="main"
Oct 31 01:03:46.652598 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:46.652246    4284 memory_manager.go:365] "RemoveStaleState removing state" podUID="33b97842-74a9-4d0b-9887-d65273b2bbf8" containerName="payload-processing"
Oct 31 01:03:46.652598 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:46.652254    4284 memory_manager.go:365] "RemoveStaleState removing state" podUID="36405c36-d9a5-4627-b812-435b5632303b" containerName="payload-pre-processing"
Oct 31 01:03:46.656126 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:46.656106    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-praxis-b-4ab4aa-openshift-default-799d6fd5-pqg68"
Oct 31 01:03:46.693537 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:46.693504    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/832a208d-8b57-444a-9ee2-fe30d01e1525-credential-socket\") pod \"e2e-praxis-b-4ab4aa-openshift-default-799d6fd5-pqg68\" (UID: \"832a208d-8b57-444a-9ee2-fe30d01e1525\") " pod="openshift-ingress/e2e-praxis-b-4ab4aa-openshift-default-799d6fd5-pqg68"
Oct 31 01:03:46.693725 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:46.693544    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/832a208d-8b57-444a-9ee2-fe30d01e1525-workload-socket\") pod \"e2e-praxis-b-4ab4aa-openshift-default-799d6fd5-pqg68\" (UID: \"832a208d-8b57-444a-9ee2-fe30d01e1525\") " pod="openshift-ingress/e2e-praxis-b-4ab4aa-openshift-default-799d6fd5-pqg68"
Oct 31 01:03:46.693725 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:46.693671    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/832a208d-8b57-444a-9ee2-fe30d01e1525-istio-podinfo\") pod \"e2e-praxis-b-4ab4aa-openshift-default-799d6fd5-pqg68\" (UID: \"832a208d-8b57-444a-9ee2-fe30d01e1525\") " pod="openshift-ingress/e2e-praxis-b-4ab4aa-openshift-default-799d6fd5-pqg68"
Oct 31 01:03:46.693725 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:46.693691    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/832a208d-8b57-444a-9ee2-fe30d01e1525-istiod-ca-cert\") pod \"e2e-praxis-b-4ab4aa-openshift-default-799d6fd5-pqg68\" (UID: \"832a208d-8b57-444a-9ee2-fe30d01e1525\") " pod="openshift-ingress/e2e-praxis-b-4ab4aa-openshift-default-799d6fd5-pqg68"
Oct 31 01:03:46.693725 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:46.693709    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/832a208d-8b57-444a-9ee2-fe30d01e1525-workload-certs\") pod \"e2e-praxis-b-4ab4aa-openshift-default-799d6fd5-pqg68\" (UID: \"832a208d-8b57-444a-9ee2-fe30d01e1525\") " pod="openshift-ingress/e2e-praxis-b-4ab4aa-openshift-default-799d6fd5-pqg68"
Oct 31 01:03:46.693959 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:46.693733    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/832a208d-8b57-444a-9ee2-fe30d01e1525-istio-data\") pod \"e2e-praxis-b-4ab4aa-openshift-default-799d6fd5-pqg68\" (UID: \"832a208d-8b57-444a-9ee2-fe30d01e1525\") " pod="openshift-ingress/e2e-praxis-b-4ab4aa-openshift-default-799d6fd5-pqg68"
Oct 31 01:03:46.693959 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:46.693855    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/832a208d-8b57-444a-9ee2-fe30d01e1525-istio-envoy\") pod \"e2e-praxis-b-4ab4aa-openshift-default-799d6fd5-pqg68\" (UID: \"832a208d-8b57-444a-9ee2-fe30d01e1525\") " pod="openshift-ingress/e2e-praxis-b-4ab4aa-openshift-default-799d6fd5-pqg68"
Oct 31 01:03:46.693959 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:46.693887    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8pt59\" (UniqueName: \"kubernetes.io/projected/832a208d-8b57-444a-9ee2-fe30d01e1525-kube-api-access-8pt59\") pod \"e2e-praxis-b-4ab4aa-openshift-default-799d6fd5-pqg68\" (UID: \"832a208d-8b57-444a-9ee2-fe30d01e1525\") " pod="openshift-ingress/e2e-praxis-b-4ab4aa-openshift-default-799d6fd5-pqg68"
Oct 31 01:03:46.693959 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:46.693925    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/832a208d-8b57-444a-9ee2-fe30d01e1525-istio-token\") pod \"e2e-praxis-b-4ab4aa-openshift-default-799d6fd5-pqg68\" (UID: \"832a208d-8b57-444a-9ee2-fe30d01e1525\") " pod="openshift-ingress/e2e-praxis-b-4ab4aa-openshift-default-799d6fd5-pqg68"
Oct 31 01:03:46.708268 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:46.707366    4284 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-praxis-b-4ab4aa-openshift-default-dockercfg-g922j\""
Oct 31 01:03:46.753599 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:46.753559    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:03:46.758533 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:46.758507    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-praxis-b-4ab4aa-openshift-default-799d6fd5-pqg68"]
Oct 31 01:03:46.811454 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:46.808998    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8pt59\" (UniqueName: \"kubernetes.io/projected/832a208d-8b57-444a-9ee2-fe30d01e1525-kube-api-access-8pt59\") pod \"e2e-praxis-b-4ab4aa-openshift-default-799d6fd5-pqg68\" (UID: \"832a208d-8b57-444a-9ee2-fe30d01e1525\") " pod="openshift-ingress/e2e-praxis-b-4ab4aa-openshift-default-799d6fd5-pqg68"
Oct 31 01:03:46.812329 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:46.812293    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/832a208d-8b57-444a-9ee2-fe30d01e1525-istio-token\") pod \"e2e-praxis-b-4ab4aa-openshift-default-799d6fd5-pqg68\" (UID: \"832a208d-8b57-444a-9ee2-fe30d01e1525\") " pod="openshift-ingress/e2e-praxis-b-4ab4aa-openshift-default-799d6fd5-pqg68"
Oct 31 01:03:46.813011 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:46.812977    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/832a208d-8b57-444a-9ee2-fe30d01e1525-credential-socket\") pod \"e2e-praxis-b-4ab4aa-openshift-default-799d6fd5-pqg68\" (UID: \"832a208d-8b57-444a-9ee2-fe30d01e1525\") " pod="openshift-ingress/e2e-praxis-b-4ab4aa-openshift-default-799d6fd5-pqg68"
Oct 31 01:03:46.813791 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:46.813748    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/832a208d-8b57-444a-9ee2-fe30d01e1525-workload-socket\") pod \"e2e-praxis-b-4ab4aa-openshift-default-799d6fd5-pqg68\" (UID: \"832a208d-8b57-444a-9ee2-fe30d01e1525\") " pod="openshift-ingress/e2e-praxis-b-4ab4aa-openshift-default-799d6fd5-pqg68"
Oct 31 01:03:46.813930 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:46.813681    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/832a208d-8b57-444a-9ee2-fe30d01e1525-credential-socket\") pod \"e2e-praxis-b-4ab4aa-openshift-default-799d6fd5-pqg68\" (UID: \"832a208d-8b57-444a-9ee2-fe30d01e1525\") " pod="openshift-ingress/e2e-praxis-b-4ab4aa-openshift-default-799d6fd5-pqg68"
Oct 31 01:03:46.814159 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:46.814136    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/832a208d-8b57-444a-9ee2-fe30d01e1525-istio-podinfo\") pod \"e2e-praxis-b-4ab4aa-openshift-default-799d6fd5-pqg68\" (UID: \"832a208d-8b57-444a-9ee2-fe30d01e1525\") " pod="openshift-ingress/e2e-praxis-b-4ab4aa-openshift-default-799d6fd5-pqg68"
Oct 31 01:03:46.814426 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:46.814391    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/832a208d-8b57-444a-9ee2-fe30d01e1525-workload-socket\") pod \"e2e-praxis-b-4ab4aa-openshift-default-799d6fd5-pqg68\" (UID: \"832a208d-8b57-444a-9ee2-fe30d01e1525\") " pod="openshift-ingress/e2e-praxis-b-4ab4aa-openshift-default-799d6fd5-pqg68"
Oct 31 01:03:46.817673 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:46.814360    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/832a208d-8b57-444a-9ee2-fe30d01e1525-istiod-ca-cert\") pod \"e2e-praxis-b-4ab4aa-openshift-default-799d6fd5-pqg68\" (UID: \"832a208d-8b57-444a-9ee2-fe30d01e1525\") " pod="openshift-ingress/e2e-praxis-b-4ab4aa-openshift-default-799d6fd5-pqg68"
Oct 31 01:03:46.818679 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:46.818655    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/832a208d-8b57-444a-9ee2-fe30d01e1525-workload-certs\") pod \"e2e-praxis-b-4ab4aa-openshift-default-799d6fd5-pqg68\" (UID: \"832a208d-8b57-444a-9ee2-fe30d01e1525\") " pod="openshift-ingress/e2e-praxis-b-4ab4aa-openshift-default-799d6fd5-pqg68"
Oct 31 01:03:46.818890 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:46.818731    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/832a208d-8b57-444a-9ee2-fe30d01e1525-workload-certs\") pod \"e2e-praxis-b-4ab4aa-openshift-default-799d6fd5-pqg68\" (UID: \"832a208d-8b57-444a-9ee2-fe30d01e1525\") " pod="openshift-ingress/e2e-praxis-b-4ab4aa-openshift-default-799d6fd5-pqg68"
Oct 31 01:03:46.818890 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:46.818768    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/832a208d-8b57-444a-9ee2-fe30d01e1525-istio-data\") pod \"e2e-praxis-b-4ab4aa-openshift-default-799d6fd5-pqg68\" (UID: \"832a208d-8b57-444a-9ee2-fe30d01e1525\") " pod="openshift-ingress/e2e-praxis-b-4ab4aa-openshift-default-799d6fd5-pqg68"
Oct 31 01:03:46.819037 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:46.818930    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/832a208d-8b57-444a-9ee2-fe30d01e1525-istio-envoy\") pod \"e2e-praxis-b-4ab4aa-openshift-default-799d6fd5-pqg68\" (UID: \"832a208d-8b57-444a-9ee2-fe30d01e1525\") " pod="openshift-ingress/e2e-praxis-b-4ab4aa-openshift-default-799d6fd5-pqg68"
Oct 31 01:03:46.819504 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:46.819470    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/832a208d-8b57-444a-9ee2-fe30d01e1525-istio-data\") pod \"e2e-praxis-b-4ab4aa-openshift-default-799d6fd5-pqg68\" (UID: \"832a208d-8b57-444a-9ee2-fe30d01e1525\") " pod="openshift-ingress/e2e-praxis-b-4ab4aa-openshift-default-799d6fd5-pqg68"
Oct 31 01:03:46.820683 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:46.820661    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/832a208d-8b57-444a-9ee2-fe30d01e1525-istiod-ca-cert\") pod \"e2e-praxis-b-4ab4aa-openshift-default-799d6fd5-pqg68\" (UID: \"832a208d-8b57-444a-9ee2-fe30d01e1525\") " pod="openshift-ingress/e2e-praxis-b-4ab4aa-openshift-default-799d6fd5-pqg68"
Oct 31 01:03:46.839241 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:46.837741    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/832a208d-8b57-444a-9ee2-fe30d01e1525-istio-envoy\") pod \"e2e-praxis-b-4ab4aa-openshift-default-799d6fd5-pqg68\" (UID: \"832a208d-8b57-444a-9ee2-fe30d01e1525\") " pod="openshift-ingress/e2e-praxis-b-4ab4aa-openshift-default-799d6fd5-pqg68"
Oct 31 01:03:46.851294 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:46.850203    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/832a208d-8b57-444a-9ee2-fe30d01e1525-istio-podinfo\") pod \"e2e-praxis-b-4ab4aa-openshift-default-799d6fd5-pqg68\" (UID: \"832a208d-8b57-444a-9ee2-fe30d01e1525\") " pod="openshift-ingress/e2e-praxis-b-4ab4aa-openshift-default-799d6fd5-pqg68"
Oct 31 01:03:46.872693 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:46.872666    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8pt59\" (UniqueName: \"kubernetes.io/projected/832a208d-8b57-444a-9ee2-fe30d01e1525-kube-api-access-8pt59\") pod \"e2e-praxis-b-4ab4aa-openshift-default-799d6fd5-pqg68\" (UID: \"832a208d-8b57-444a-9ee2-fe30d01e1525\") " pod="openshift-ingress/e2e-praxis-b-4ab4aa-openshift-default-799d6fd5-pqg68"
Oct 31 01:03:46.875625 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:46.875599    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/832a208d-8b57-444a-9ee2-fe30d01e1525-istio-token\") pod \"e2e-praxis-b-4ab4aa-openshift-default-799d6fd5-pqg68\" (UID: \"832a208d-8b57-444a-9ee2-fe30d01e1525\") " pod="openshift-ingress/e2e-praxis-b-4ab4aa-openshift-default-799d6fd5-pqg68"
Oct 31 01:03:46.949757 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:46.949709    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:03:46.968310 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:46.968283    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-praxis-b-4ab4aa-openshift-default-799d6fd5-pqg68"
Oct 31 01:03:47.026425 ip-10-0-132-173 kubenswrapper[4284]: E1031 01:03:47.026364    4284 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fd8sp" containerName="extractor"
Oct 31 01:03:47.248811 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:47.248733    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:03:47.401270 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:47.401239    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-praxis-b-4ab4aa-openshift-default-799d6fd5-pqg68"]
Oct 31 01:03:47.404381 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:47.404347    4284 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 01:03:47.404478 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:47.404414    4284 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 01:03:47.404478 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:47.404445    4284 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 01:03:47.898438 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:47.898397    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-b-4ab4aa-openshift-default-799d6fd5-pqg68" event={"ID":"832a208d-8b57-444a-9ee2-fe30d01e1525","Type":"ContainerStarted","Data":"5aac0cb912889747d7872fb30066bf839447f7474f3ed90c219f6b2730e0abe1"}
Oct 31 01:03:47.898854 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:47.898462    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-b-4ab4aa-openshift-default-799d6fd5-pqg68" event={"ID":"832a208d-8b57-444a-9ee2-fe30d01e1525","Type":"ContainerStarted","Data":"d192885d624693910fb1280edd90ffb1428a8584908fd8ca6da63d43bc9017a5"}
Oct 31 01:03:47.969000 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:47.968974    4284 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-praxis-b-4ab4aa-openshift-default-799d6fd5-pqg68"
Oct 31 01:03:48.065612 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:48.065555    4284 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-praxis-b-4ab4aa-openshift-default-799d6fd5-pqg68" podStartSLOduration=2.0655390750000002 podStartE2EDuration="2.065539075s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 01:03:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 01:03:48.015858724 +0000 UTC m=+2142.160865642" watchObservedRunningTime="2026-10-31 01:03:48.065539075 +0000 UTC m=+2142.210545990"
Oct 31 01:03:48.974322 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:48.974288    4284 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-praxis-b-4ab4aa-openshift-default-799d6fd5-pqg68"
Oct 31 01:03:49.772099 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:49.772054    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:03:49.911835 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:49.911795    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-praxis-b-4ab4aa-openshift-default-799d6fd5-pqg68"
Oct 31 01:03:49.912831 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:49.912813    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-praxis-b-4ab4aa-openshift-default-799d6fd5-pqg68"
Oct 31 01:03:49.978650 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:49.978619    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:03:50.137691 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:50.137610    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-eb41d3-openshift-default-5bdbd7758c-lqfzl"]
Oct 31 01:03:50.137919 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:50.137890    4284 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-b-eb41d3-openshift-default-5bdbd7758c-lqfzl" podUID="2728e644-cebe-42fe-88ef-3d17a8a5817e" containerName="istio-proxy" containerID="cri-o://cc61d765850e8845baa04fa2e46fbed9ffd8a7dcb42302d0233f86713dcd2156" gracePeriod=30
Oct 31 01:03:50.162718 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:50.162681    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:03:50.903834 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:50.903803    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:03:51.102005 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:51.101969    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:03:51.954983 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:51.954948    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:03:52.147930 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:52.147895    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-cb2b8a-77c6c7fdbf-mzb85"]
Oct 31 01:03:52.148369 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:52.148098    4284 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-cb2b8a-77c6c7fdbf-mzb85" podUID="aa9a9164-762f-4869-a04d-5cd82e3d8bfa" containerName="maas-api" containerID="cri-o://6d7206673a82d00907791d9c5b8a234bdf1a8e6f2f6349177aaa8a0dfe1b905b" gracePeriod=30
Oct 31 01:03:52.260157 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:52.260054    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:03:52.513724 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:52.513668    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-cb2b8a-77c6c7fdbf-mzb85"
Oct 31 01:03:52.651008 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:52.650969    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:03:52.658060 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:52.658032    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/aa9a9164-762f-4869-a04d-5cd82e3d8bfa-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/aa9a9164-762f-4869-a04d-5cd82e3d8bfa-maas-api-tls\") pod \"aa9a9164-762f-4869-a04d-5cd82e3d8bfa\" (UID: \"aa9a9164-762f-4869-a04d-5cd82e3d8bfa\") "
Oct 31 01:03:52.658252 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:52.658107    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/aa9a9164-762f-4869-a04d-5cd82e3d8bfa-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/aa9a9164-762f-4869-a04d-5cd82e3d8bfa-openshift-service-ca\") pod \"aa9a9164-762f-4869-a04d-5cd82e3d8bfa\" (UID: \"aa9a9164-762f-4869-a04d-5cd82e3d8bfa\") "
Oct 31 01:03:52.658252 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:52.658230    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/aa9a9164-762f-4869-a04d-5cd82e3d8bfa-metrics-certs\" (UniqueName: \"kubernetes.io/secret/aa9a9164-762f-4869-a04d-5cd82e3d8bfa-metrics-certs\") pod \"aa9a9164-762f-4869-a04d-5cd82e3d8bfa\" (UID: \"aa9a9164-762f-4869-a04d-5cd82e3d8bfa\") "
Oct 31 01:03:52.658342 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:52.658304    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/aa9a9164-762f-4869-a04d-5cd82e3d8bfa-kube-api-access-gmgnv\" (UniqueName: \"kubernetes.io/projected/aa9a9164-762f-4869-a04d-5cd82e3d8bfa-kube-api-access-gmgnv\") pod \"aa9a9164-762f-4869-a04d-5cd82e3d8bfa\" (UID: \"aa9a9164-762f-4869-a04d-5cd82e3d8bfa\") "
Oct 31 01:03:52.658484 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:52.658461    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/aa9a9164-762f-4869-a04d-5cd82e3d8bfa-openshift-service-ca" pod "aa9a9164-762f-4869-a04d-5cd82e3d8bfa" (UID: "aa9a9164-762f-4869-a04d-5cd82e3d8bfa"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 01:03:52.659013 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:52.658991    4284 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/aa9a9164-762f-4869-a04d-5cd82e3d8bfa-openshift-service-ca\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:03:52.660523 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:52.660496    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/aa9a9164-762f-4869-a04d-5cd82e3d8bfa-maas-api-tls" pod "aa9a9164-762f-4869-a04d-5cd82e3d8bfa" (UID: "aa9a9164-762f-4869-a04d-5cd82e3d8bfa"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 01:03:52.660611 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:52.660521    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/aa9a9164-762f-4869-a04d-5cd82e3d8bfa-metrics-certs" pod "aa9a9164-762f-4869-a04d-5cd82e3d8bfa" (UID: "aa9a9164-762f-4869-a04d-5cd82e3d8bfa"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 01:03:52.660611 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:52.660542    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aa9a9164-762f-4869-a04d-5cd82e3d8bfa-kube-api-access-gmgnv" pod "aa9a9164-762f-4869-a04d-5cd82e3d8bfa" (UID: "aa9a9164-762f-4869-a04d-5cd82e3d8bfa"). InnerVolumeSpecName "kube-api-access-gmgnv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:03:52.760004 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:52.759959    4284 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/aa9a9164-762f-4869-a04d-5cd82e3d8bfa-maas-api-tls\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:03:52.760004 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:52.760002    4284 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/aa9a9164-762f-4869-a04d-5cd82e3d8bfa-metrics-certs\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:03:52.760250 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:52.760014    4284 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gmgnv\" (UniqueName: \"kubernetes.io/projected/aa9a9164-762f-4869-a04d-5cd82e3d8bfa-kube-api-access-gmgnv\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:03:52.863987 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:52.863861    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:03:52.933605 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:52.933571    4284 generic.go:356] "Generic (PLEG): container finished" podID="aa9a9164-762f-4869-a04d-5cd82e3d8bfa" containerID="6d7206673a82d00907791d9c5b8a234bdf1a8e6f2f6349177aaa8a0dfe1b905b" exitCode=0
Oct 31 01:03:52.933776 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:52.933632    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-cb2b8a-77c6c7fdbf-mzb85"
Oct 31 01:03:52.933776 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:52.933648    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-cb2b8a-77c6c7fdbf-mzb85" event={"ID":"aa9a9164-762f-4869-a04d-5cd82e3d8bfa","Type":"ContainerDied","Data":"6d7206673a82d00907791d9c5b8a234bdf1a8e6f2f6349177aaa8a0dfe1b905b"}
Oct 31 01:03:52.933776 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:52.933729    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-cb2b8a-77c6c7fdbf-mzb85" event={"ID":"aa9a9164-762f-4869-a04d-5cd82e3d8bfa","Type":"ContainerDied","Data":"742e0e54dbd77bb4b241b3f173d1b89b80fbeb5f1feea715a5bcb7b69ca293c3"}
Oct 31 01:03:52.933776 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:52.933761    4284 scope.go:122] "RemoveContainer" containerID="6d7206673a82d00907791d9c5b8a234bdf1a8e6f2f6349177aaa8a0dfe1b905b"
Oct 31 01:03:52.944083 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:52.944067    4284 scope.go:122] "RemoveContainer" containerID="6d7206673a82d00907791d9c5b8a234bdf1a8e6f2f6349177aaa8a0dfe1b905b"
Oct 31 01:03:52.944384 ip-10-0-132-173 kubenswrapper[4284]: E1031 01:03:52.944360    4284 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6d7206673a82d00907791d9c5b8a234bdf1a8e6f2f6349177aaa8a0dfe1b905b\": container with ID starting with 6d7206673a82d00907791d9c5b8a234bdf1a8e6f2f6349177aaa8a0dfe1b905b not found: ID does not exist" containerID="6d7206673a82d00907791d9c5b8a234bdf1a8e6f2f6349177aaa8a0dfe1b905b"
Oct 31 01:03:52.944453 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:52.944395    4284 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6d7206673a82d00907791d9c5b8a234bdf1a8e6f2f6349177aaa8a0dfe1b905b"} err="failed to get container status \"6d7206673a82d00907791d9c5b8a234bdf1a8e6f2f6349177aaa8a0dfe1b905b\": rpc error: code = NotFound desc = could not find container \"6d7206673a82d00907791d9c5b8a234bdf1a8e6f2f6349177aaa8a0dfe1b905b\": container with ID starting with 6d7206673a82d00907791d9c5b8a234bdf1a8e6f2f6349177aaa8a0dfe1b905b not found: ID does not exist"
Oct 31 01:03:53.022529 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:53.022492    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:03:53.110408 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:53.110377    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-cb2b8a-77c6c7fdbf-mzb85"]
Oct 31 01:03:53.122652 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:53.122584    4284 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-cb2b8a-77c6c7fdbf-mzb85"]
Oct 31 01:03:53.963573 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:53.963538    4284 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="aa9a9164-762f-4869-a04d-5cd82e3d8bfa" path="/var/lib/kubelet/pods/aa9a9164-762f-4869-a04d-5cd82e3d8bfa/volumes"
Oct 31 01:03:54.422176 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:54.422140    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:03:54.634927 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:54.634885    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:03:54.925029 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:54.924988    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:03:55.422806 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:55.422777    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-eb41d3-openshift-default-5bdbd7758c-lqfzl"
Oct 31 01:03:55.508279 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:55.508179    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/2728e644-cebe-42fe-88ef-3d17a8a5817e-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2728e644-cebe-42fe-88ef-3d17a8a5817e-istio-podinfo\") pod \"2728e644-cebe-42fe-88ef-3d17a8a5817e\" (UID: \"2728e644-cebe-42fe-88ef-3d17a8a5817e\") "
Oct 31 01:03:55.508279 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:55.508276    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/2728e644-cebe-42fe-88ef-3d17a8a5817e-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2728e644-cebe-42fe-88ef-3d17a8a5817e-istiod-ca-cert\") pod \"2728e644-cebe-42fe-88ef-3d17a8a5817e\" (UID: \"2728e644-cebe-42fe-88ef-3d17a8a5817e\") "
Oct 31 01:03:55.508517 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:55.508431    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2728e644-cebe-42fe-88ef-3d17a8a5817e-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2728e644-cebe-42fe-88ef-3d17a8a5817e-istio-data\") pod \"2728e644-cebe-42fe-88ef-3d17a8a5817e\" (UID: \"2728e644-cebe-42fe-88ef-3d17a8a5817e\") "
Oct 31 01:03:55.508517 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:55.508482    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2728e644-cebe-42fe-88ef-3d17a8a5817e-kube-api-access-fpmfv\" (UniqueName: \"kubernetes.io/projected/2728e644-cebe-42fe-88ef-3d17a8a5817e-kube-api-access-fpmfv\") pod \"2728e644-cebe-42fe-88ef-3d17a8a5817e\" (UID: \"2728e644-cebe-42fe-88ef-3d17a8a5817e\") "
Oct 31 01:03:55.508637 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:55.508566    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2728e644-cebe-42fe-88ef-3d17a8a5817e-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2728e644-cebe-42fe-88ef-3d17a8a5817e-workload-certs\") pod \"2728e644-cebe-42fe-88ef-3d17a8a5817e\" (UID: \"2728e644-cebe-42fe-88ef-3d17a8a5817e\") "
Oct 31 01:03:55.508704 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:55.508648    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2728e644-cebe-42fe-88ef-3d17a8a5817e-istiod-ca-cert" pod "2728e644-cebe-42fe-88ef-3d17a8a5817e" (UID: "2728e644-cebe-42fe-88ef-3d17a8a5817e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 01:03:55.508704 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:55.508666    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2728e644-cebe-42fe-88ef-3d17a8a5817e-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2728e644-cebe-42fe-88ef-3d17a8a5817e-workload-socket\") pod \"2728e644-cebe-42fe-88ef-3d17a8a5817e\" (UID: \"2728e644-cebe-42fe-88ef-3d17a8a5817e\") "
Oct 31 01:03:55.508704 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:55.508671    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2728e644-cebe-42fe-88ef-3d17a8a5817e-istio-data" pod "2728e644-cebe-42fe-88ef-3d17a8a5817e" (UID: "2728e644-cebe-42fe-88ef-3d17a8a5817e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:03:55.508825 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:55.508743    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2728e644-cebe-42fe-88ef-3d17a8a5817e-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2728e644-cebe-42fe-88ef-3d17a8a5817e-credential-socket\") pod \"2728e644-cebe-42fe-88ef-3d17a8a5817e\" (UID: \"2728e644-cebe-42fe-88ef-3d17a8a5817e\") "
Oct 31 01:03:55.508825 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:55.508771    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2728e644-cebe-42fe-88ef-3d17a8a5817e-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2728e644-cebe-42fe-88ef-3d17a8a5817e-istio-envoy\") pod \"2728e644-cebe-42fe-88ef-3d17a8a5817e\" (UID: \"2728e644-cebe-42fe-88ef-3d17a8a5817e\") "
Oct 31 01:03:55.508904 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:55.508846    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2728e644-cebe-42fe-88ef-3d17a8a5817e-workload-certs" pod "2728e644-cebe-42fe-88ef-3d17a8a5817e" (UID: "2728e644-cebe-42fe-88ef-3d17a8a5817e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:03:55.508904 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:55.508859    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2728e644-cebe-42fe-88ef-3d17a8a5817e-istio-token\" (UniqueName: \"kubernetes.io/projected/2728e644-cebe-42fe-88ef-3d17a8a5817e-istio-token\") pod \"2728e644-cebe-42fe-88ef-3d17a8a5817e\" (UID: \"2728e644-cebe-42fe-88ef-3d17a8a5817e\") "
Oct 31 01:03:55.509007 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:55.508954    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2728e644-cebe-42fe-88ef-3d17a8a5817e-credential-socket" pod "2728e644-cebe-42fe-88ef-3d17a8a5817e" (UID: "2728e644-cebe-42fe-88ef-3d17a8a5817e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:03:55.509007 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:55.508987    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2728e644-cebe-42fe-88ef-3d17a8a5817e-workload-socket" pod "2728e644-cebe-42fe-88ef-3d17a8a5817e" (UID: "2728e644-cebe-42fe-88ef-3d17a8a5817e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:03:55.509351 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:55.509325    4284 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2728e644-cebe-42fe-88ef-3d17a8a5817e-workload-socket\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:03:55.509351 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:55.509352    4284 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2728e644-cebe-42fe-88ef-3d17a8a5817e-credential-socket\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:03:55.509712 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:55.509368    4284 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2728e644-cebe-42fe-88ef-3d17a8a5817e-istiod-ca-cert\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:03:55.509712 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:55.509384    4284 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2728e644-cebe-42fe-88ef-3d17a8a5817e-istio-data\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:03:55.509712 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:55.509405    4284 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2728e644-cebe-42fe-88ef-3d17a8a5817e-workload-certs\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:03:55.511065 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:55.511034    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2728e644-cebe-42fe-88ef-3d17a8a5817e-istio-envoy" pod "2728e644-cebe-42fe-88ef-3d17a8a5817e" (UID: "2728e644-cebe-42fe-88ef-3d17a8a5817e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:03:55.511178 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:55.511080    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/2728e644-cebe-42fe-88ef-3d17a8a5817e-istio-podinfo" pod "2728e644-cebe-42fe-88ef-3d17a8a5817e" (UID: "2728e644-cebe-42fe-88ef-3d17a8a5817e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 01:03:55.511325 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:55.511303    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2728e644-cebe-42fe-88ef-3d17a8a5817e-kube-api-access-fpmfv" pod "2728e644-cebe-42fe-88ef-3d17a8a5817e" (UID: "2728e644-cebe-42fe-88ef-3d17a8a5817e"). InnerVolumeSpecName "kube-api-access-fpmfv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:03:55.511438 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:55.511421    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2728e644-cebe-42fe-88ef-3d17a8a5817e-istio-token" pod "2728e644-cebe-42fe-88ef-3d17a8a5817e" (UID: "2728e644-cebe-42fe-88ef-3d17a8a5817e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:03:55.587707 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:55.587675    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:03:55.610644 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:55.610606    4284 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2728e644-cebe-42fe-88ef-3d17a8a5817e-istio-envoy\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:03:55.610644 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:55.610646    4284 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2728e644-cebe-42fe-88ef-3d17a8a5817e-istio-token\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:03:55.610897 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:55.610660    4284 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2728e644-cebe-42fe-88ef-3d17a8a5817e-istio-podinfo\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:03:55.610897 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:55.610675    4284 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fpmfv\" (UniqueName: \"kubernetes.io/projected/2728e644-cebe-42fe-88ef-3d17a8a5817e-kube-api-access-fpmfv\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:03:55.966325 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:55.966292    4284 generic.go:356] "Generic (PLEG): container finished" podID="2728e644-cebe-42fe-88ef-3d17a8a5817e" containerID="cc61d765850e8845baa04fa2e46fbed9ffd8a7dcb42302d0233f86713dcd2156" exitCode=0
Oct 31 01:03:55.966325 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:55.966326    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-eb41d3-openshift-default-5bdbd7758c-lqfzl"
Oct 31 01:03:55.966564 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:55.966416    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-eb41d3-openshift-default-5bdbd7758c-lqfzl" event={"ID":"2728e644-cebe-42fe-88ef-3d17a8a5817e","Type":"ContainerDied","Data":"cc61d765850e8845baa04fa2e46fbed9ffd8a7dcb42302d0233f86713dcd2156"}
Oct 31 01:03:55.966564 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:55.966455    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-eb41d3-openshift-default-5bdbd7758c-lqfzl" event={"ID":"2728e644-cebe-42fe-88ef-3d17a8a5817e","Type":"ContainerDied","Data":"ae713e97ffcad839b15942904e04134213e5c7e9974e0a1ab4bd5f4abdda2f28"}
Oct 31 01:03:55.966564 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:55.966478    4284 scope.go:122] "RemoveContainer" containerID="cc61d765850e8845baa04fa2e46fbed9ffd8a7dcb42302d0233f86713dcd2156"
Oct 31 01:03:55.977428 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:55.977411    4284 scope.go:122] "RemoveContainer" containerID="7d2e3989067ab9ebc481325739a6a6843c952e478c9b181b3e522b9af1458faf"
Oct 31 01:03:55.986163 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:55.986135    4284 scope.go:122] "RemoveContainer" containerID="cc61d765850e8845baa04fa2e46fbed9ffd8a7dcb42302d0233f86713dcd2156"
Oct 31 01:03:55.986487 ip-10-0-132-173 kubenswrapper[4284]: E1031 01:03:55.986461    4284 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cc61d765850e8845baa04fa2e46fbed9ffd8a7dcb42302d0233f86713dcd2156\": container with ID starting with cc61d765850e8845baa04fa2e46fbed9ffd8a7dcb42302d0233f86713dcd2156 not found: ID does not exist" containerID="cc61d765850e8845baa04fa2e46fbed9ffd8a7dcb42302d0233f86713dcd2156"
Oct 31 01:03:55.986544 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:55.986502    4284 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cc61d765850e8845baa04fa2e46fbed9ffd8a7dcb42302d0233f86713dcd2156"} err="failed to get container status \"cc61d765850e8845baa04fa2e46fbed9ffd8a7dcb42302d0233f86713dcd2156\": rpc error: code = NotFound desc = could not find container \"cc61d765850e8845baa04fa2e46fbed9ffd8a7dcb42302d0233f86713dcd2156\": container with ID starting with cc61d765850e8845baa04fa2e46fbed9ffd8a7dcb42302d0233f86713dcd2156 not found: ID does not exist"
Oct 31 01:03:55.986544 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:55.986532    4284 scope.go:122] "RemoveContainer" containerID="7d2e3989067ab9ebc481325739a6a6843c952e478c9b181b3e522b9af1458faf"
Oct 31 01:03:55.986829 ip-10-0-132-173 kubenswrapper[4284]: E1031 01:03:55.986809    4284 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7d2e3989067ab9ebc481325739a6a6843c952e478c9b181b3e522b9af1458faf\": container with ID starting with 7d2e3989067ab9ebc481325739a6a6843c952e478c9b181b3e522b9af1458faf not found: ID does not exist" containerID="7d2e3989067ab9ebc481325739a6a6843c952e478c9b181b3e522b9af1458faf"
Oct 31 01:03:55.986899 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:55.986838    4284 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7d2e3989067ab9ebc481325739a6a6843c952e478c9b181b3e522b9af1458faf"} err="failed to get container status \"7d2e3989067ab9ebc481325739a6a6843c952e478c9b181b3e522b9af1458faf\": rpc error: code = NotFound desc = could not find container \"7d2e3989067ab9ebc481325739a6a6843c952e478c9b181b3e522b9af1458faf\": container with ID starting with 7d2e3989067ab9ebc481325739a6a6843c952e478c9b181b3e522b9af1458faf not found: ID does not exist"
Oct 31 01:03:56.127493 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:56.127453    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-eb41d3-openshift-default-5bdbd7758c-lqfzl"]
Oct 31 01:03:56.130471 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:56.130446    4284 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-b-eb41d3-openshift-default-5bdbd7758c-lqfzl"]
Oct 31 01:03:57.425436 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:57.425403    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:03:57.636871 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:57.636839    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:03:58.008237 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:58.006537    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:03:58.054146 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:58.054105    4284 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2728e644-cebe-42fe-88ef-3d17a8a5817e" path="/var/lib/kubelet/pods/2728e644-cebe-42fe-88ef-3d17a8a5817e/volumes"
Oct 31 01:03:58.054599 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:58.054572    4284 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-praxis-b-4ab4aa-5d9969dd59-n4vgd"]
Oct 31 01:03:58.055231 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:58.055197    4284 cpu_manager.go:414] "containerMap: removing container" podUID="2728e644-cebe-42fe-88ef-3d17a8a5817e" containerName="istio-proxy"
Oct 31 01:03:58.055335 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:58.055232    4284 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2728e644-cebe-42fe-88ef-3d17a8a5817e" containerName="istio-proxy"
Oct 31 01:03:58.055335 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:58.055258    4284 cpu_manager.go:414] "containerMap: removing container" podUID="2728e644-cebe-42fe-88ef-3d17a8a5817e" containerName="istio-proxy"
Oct 31 01:03:58.055335 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:58.055264    4284 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2728e644-cebe-42fe-88ef-3d17a8a5817e" containerName="istio-proxy"
Oct 31 01:03:58.055335 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:58.055271    4284 cpu_manager.go:414] "containerMap: removing container" podUID="aa9a9164-762f-4869-a04d-5cd82e3d8bfa" containerName="maas-api"
Oct 31 01:03:58.055335 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:58.055284    4284 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="aa9a9164-762f-4869-a04d-5cd82e3d8bfa" containerName="maas-api"
Oct 31 01:03:58.055613 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:58.055379    4284 memory_manager.go:365] "RemoveStaleState removing state" podUID="2728e644-cebe-42fe-88ef-3d17a8a5817e" containerName="istio-proxy"
Oct 31 01:03:58.055613 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:58.055393    4284 memory_manager.go:365] "RemoveStaleState removing state" podUID="2728e644-cebe-42fe-88ef-3d17a8a5817e" containerName="istio-proxy"
Oct 31 01:03:58.055613 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:58.055402    4284 memory_manager.go:365] "RemoveStaleState removing state" podUID="aa9a9164-762f-4869-a04d-5cd82e3d8bfa" containerName="maas-api"
Oct 31 01:03:58.060702 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:58.060678    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-praxis-b-4ab4aa-5d9969dd59-n4vgd"
Oct 31 01:03:58.086176 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:58.086142    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-praxis-b-4ab4aa-5d9969dd59-n4vgd"]
Oct 31 01:03:58.116307 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:58.114253    4284 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-praxis-b-4ab4aa-dockercfg-5xjw2\""
Oct 31 01:03:58.130171 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:58.130138    4284 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-praxis-b-4ab4aa\""
Oct 31 01:03:58.151628 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:58.151594    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/82bd1e41-59c3-4dc5-80ac-e1c426f7b0c9-plugins-config-volume\") pod \"payload-processing-e2e-praxis-b-4ab4aa-5d9969dd59-n4vgd\" (UID: \"82bd1e41-59c3-4dc5-80ac-e1c426f7b0c9\") " pod="openshift-ingress/payload-processing-e2e-praxis-b-4ab4aa-5d9969dd59-n4vgd"
Oct 31 01:03:58.151827 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:58.151740    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xdjm5\" (UniqueName: \"kubernetes.io/projected/82bd1e41-59c3-4dc5-80ac-e1c426f7b0c9-kube-api-access-xdjm5\") pod \"payload-processing-e2e-praxis-b-4ab4aa-5d9969dd59-n4vgd\" (UID: \"82bd1e41-59c3-4dc5-80ac-e1c426f7b0c9\") " pod="openshift-ingress/payload-processing-e2e-praxis-b-4ab4aa-5d9969dd59-n4vgd"
Oct 31 01:03:58.169034 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:58.169002    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:03:58.252805 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:58.252771    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xdjm5\" (UniqueName: \"kubernetes.io/projected/82bd1e41-59c3-4dc5-80ac-e1c426f7b0c9-kube-api-access-xdjm5\") pod \"payload-processing-e2e-praxis-b-4ab4aa-5d9969dd59-n4vgd\" (UID: \"82bd1e41-59c3-4dc5-80ac-e1c426f7b0c9\") " pod="openshift-ingress/payload-processing-e2e-praxis-b-4ab4aa-5d9969dd59-n4vgd"
Oct 31 01:03:58.252998 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:58.252910    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/82bd1e41-59c3-4dc5-80ac-e1c426f7b0c9-plugins-config-volume\") pod \"payload-processing-e2e-praxis-b-4ab4aa-5d9969dd59-n4vgd\" (UID: \"82bd1e41-59c3-4dc5-80ac-e1c426f7b0c9\") " pod="openshift-ingress/payload-processing-e2e-praxis-b-4ab4aa-5d9969dd59-n4vgd"
Oct 31 01:03:58.253670 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:58.253644    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/82bd1e41-59c3-4dc5-80ac-e1c426f7b0c9-plugins-config-volume\") pod \"payload-processing-e2e-praxis-b-4ab4aa-5d9969dd59-n4vgd\" (UID: \"82bd1e41-59c3-4dc5-80ac-e1c426f7b0c9\") " pod="openshift-ingress/payload-processing-e2e-praxis-b-4ab4aa-5d9969dd59-n4vgd"
Oct 31 01:03:58.312028 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:58.311954    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xdjm5\" (UniqueName: \"kubernetes.io/projected/82bd1e41-59c3-4dc5-80ac-e1c426f7b0c9-kube-api-access-xdjm5\") pod \"payload-processing-e2e-praxis-b-4ab4aa-5d9969dd59-n4vgd\" (UID: \"82bd1e41-59c3-4dc5-80ac-e1c426f7b0c9\") " pod="openshift-ingress/payload-processing-e2e-praxis-b-4ab4aa-5d9969dd59-n4vgd"
Oct 31 01:03:58.375662 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:58.375630    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-praxis-b-4ab4aa-5d9969dd59-n4vgd"
Oct 31 01:03:58.771591 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:58.771561    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-praxis-b-4ab4aa-5d9969dd59-n4vgd"]
Oct 31 01:03:58.772525 ip-10-0-132-173 kubenswrapper[4284]: W1031 01:03:58.772502    4284 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod82bd1e41_59c3_4dc5_80ac_e1c426f7b0c9.slice/crio-913f8ba3fc35e0a23dc20d8ae9805670697ac6526faedcf0faa591794d48b782 WatchSource:0}: Error finding container 913f8ba3fc35e0a23dc20d8ae9805670697ac6526faedcf0faa591794d48b782: Status 404 returned error can't find the container with id 913f8ba3fc35e0a23dc20d8ae9805670697ac6526faedcf0faa591794d48b782
Oct 31 01:03:59.020115 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:59.020025    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-praxis-b-4ab4aa-5d9969dd59-n4vgd" event={"ID":"82bd1e41-59c3-4dc5-80ac-e1c426f7b0c9","Type":"ContainerStarted","Data":"db8c20fc4fb95c2d48ef59268f25cb7533fb49394a6d9cb5ebaab4ddb335c646"}
Oct 31 01:03:59.020115 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:59.020082    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-praxis-b-4ab4aa-5d9969dd59-n4vgd" event={"ID":"82bd1e41-59c3-4dc5-80ac-e1c426f7b0c9","Type":"ContainerStarted","Data":"913f8ba3fc35e0a23dc20d8ae9805670697ac6526faedcf0faa591794d48b782"}
Oct 31 01:03:59.020115 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:59.020104    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-praxis-b-4ab4aa-5d9969dd59-n4vgd"
Oct 31 01:03:59.136248 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:03:59.136183    4284 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-praxis-b-4ab4aa-5d9969dd59-n4vgd" podStartSLOduration=2.136168572 podStartE2EDuration="2.136168572s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 01:03:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 01:03:59.13375375 +0000 UTC m=+2153.278760667" watchObservedRunningTime="2026-10-31 01:03:59.136168572 +0000 UTC m=+2153.281175511"
Oct 31 01:04:00.006397 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:00.006364    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:04:00.947259 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:00.947226    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:04:02.278159 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:02.278100    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:04:02.463812 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:02.463779    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:04:02.862276 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:02.862235    4284 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-praxis-a-add5bf-564b7f99c5-dhn9p"]
Oct 31 01:04:02.866348 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:02.866327    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-praxis-a-add5bf-564b7f99c5-dhn9p"
Oct 31 01:04:02.920575 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:02.920540    4284 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-praxis-a-add5bf\""
Oct 31 01:04:02.923917 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:02.923882    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5f50d8f5-28a3-4289-a54d-79aa99424929-plugins-config-volume\") pod \"payload-processing-e2e-praxis-a-add5bf-564b7f99c5-dhn9p\" (UID: \"5f50d8f5-28a3-4289-a54d-79aa99424929\") " pod="openshift-ingress/payload-processing-e2e-praxis-a-add5bf-564b7f99c5-dhn9p"
Oct 31 01:04:02.924309 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:02.924285    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v5g7g\" (UniqueName: \"kubernetes.io/projected/5f50d8f5-28a3-4289-a54d-79aa99424929-kube-api-access-v5g7g\") pod \"payload-processing-e2e-praxis-a-add5bf-564b7f99c5-dhn9p\" (UID: \"5f50d8f5-28a3-4289-a54d-79aa99424929\") " pod="openshift-ingress/payload-processing-e2e-praxis-a-add5bf-564b7f99c5-dhn9p"
Oct 31 01:04:02.973526 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:02.973490    4284 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-praxis-a-add5bf-dockercfg-ffmd6\""
Oct 31 01:04:03.005848 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:03.005767    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-praxis-a-add5bf-564b7f99c5-dhn9p"]
Oct 31 01:04:03.025491 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:03.025460    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v5g7g\" (UniqueName: \"kubernetes.io/projected/5f50d8f5-28a3-4289-a54d-79aa99424929-kube-api-access-v5g7g\") pod \"payload-processing-e2e-praxis-a-add5bf-564b7f99c5-dhn9p\" (UID: \"5f50d8f5-28a3-4289-a54d-79aa99424929\") " pod="openshift-ingress/payload-processing-e2e-praxis-a-add5bf-564b7f99c5-dhn9p"
Oct 31 01:04:03.025707 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:03.025605    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5f50d8f5-28a3-4289-a54d-79aa99424929-plugins-config-volume\") pod \"payload-processing-e2e-praxis-a-add5bf-564b7f99c5-dhn9p\" (UID: \"5f50d8f5-28a3-4289-a54d-79aa99424929\") " pod="openshift-ingress/payload-processing-e2e-praxis-a-add5bf-564b7f99c5-dhn9p"
Oct 31 01:04:03.026748 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:03.026718    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5f50d8f5-28a3-4289-a54d-79aa99424929-plugins-config-volume\") pod \"payload-processing-e2e-praxis-a-add5bf-564b7f99c5-dhn9p\" (UID: \"5f50d8f5-28a3-4289-a54d-79aa99424929\") " pod="openshift-ingress/payload-processing-e2e-praxis-a-add5bf-564b7f99c5-dhn9p"
Oct 31 01:04:03.086243 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:03.086186    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v5g7g\" (UniqueName: \"kubernetes.io/projected/5f50d8f5-28a3-4289-a54d-79aa99424929-kube-api-access-v5g7g\") pod \"payload-processing-e2e-praxis-a-add5bf-564b7f99c5-dhn9p\" (UID: \"5f50d8f5-28a3-4289-a54d-79aa99424929\") " pod="openshift-ingress/payload-processing-e2e-praxis-a-add5bf-564b7f99c5-dhn9p"
Oct 31 01:04:03.182835 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:03.182802    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-praxis-a-add5bf-564b7f99c5-dhn9p"
Oct 31 01:04:03.510725 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:03.510621    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:04:03.581977 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:03.581950    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-praxis-a-add5bf-564b7f99c5-dhn9p"]
Oct 31 01:04:03.582389 ip-10-0-132-173 kubenswrapper[4284]: W1031 01:04:03.582363    4284 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5f50d8f5_28a3_4289_a54d_79aa99424929.slice/crio-d4c964ebd8700423fe4baa39eb451abd939af5cf8d5a96ad22d562787b4c6ede WatchSource:0}: Error finding container d4c964ebd8700423fe4baa39eb451abd939af5cf8d5a96ad22d562787b4c6ede: Status 404 returned error can't find the container with id d4c964ebd8700423fe4baa39eb451abd939af5cf8d5a96ad22d562787b4c6ede
Oct 31 01:04:03.677881 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:03.677842    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:04:03.842471 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:03.842384    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-cb2b8a-openshift-default-75dd976c57-4jgr7"]
Oct 31 01:04:03.842649 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:03.842617    4284 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-b-cb2b8a-openshift-default-75dd976c57-4jgr7" podUID="d999d6fd-4e88-49eb-b8ac-f9c1e03baa4c" containerName="istio-proxy" containerID="cri-o://7285956f5dd8d2eb76fe948a272aea4828a3b3ba3268cdeaba9e4d3c418eee90" gracePeriod=30
Oct 31 01:04:03.901991 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:03.901949    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:04:04.068257 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:04.066491    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-praxis-a-add5bf-564b7f99c5-dhn9p" event={"ID":"5f50d8f5-28a3-4289-a54d-79aa99424929","Type":"ContainerStarted","Data":"92a3f7096b4983e643717cf0924e49173f48b7052411ad9cbfbc5285e44e0192"}
Oct 31 01:04:04.068257 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:04.066558    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-praxis-a-add5bf-564b7f99c5-dhn9p" event={"ID":"5f50d8f5-28a3-4289-a54d-79aa99424929","Type":"ContainerStarted","Data":"d4c964ebd8700423fe4baa39eb451abd939af5cf8d5a96ad22d562787b4c6ede"}
Oct 31 01:04:04.068257 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:04.067707    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-praxis-a-add5bf-564b7f99c5-dhn9p"
Oct 31 01:04:04.187326 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:04.187266    4284 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-praxis-a-add5bf-564b7f99c5-dhn9p" podStartSLOduration=2.187250179 podStartE2EDuration="2.187250179s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 01:04:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 01:04:04.185084528 +0000 UTC m=+2158.330091443" watchObservedRunningTime="2026-10-31 01:04:04.187250179 +0000 UTC m=+2158.332257118"
Oct 31 01:04:05.319177 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:05.319140    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:04:09.110907 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:09.110837    4284 generic.go:356] "Generic (PLEG): container finished" podID="d999d6fd-4e88-49eb-b8ac-f9c1e03baa4c" containerID="7285956f5dd8d2eb76fe948a272aea4828a3b3ba3268cdeaba9e4d3c418eee90" exitCode=0
Oct 31 01:04:09.111365 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:09.110915    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-cb2b8a-openshift-default-75dd976c57-4jgr7" event={"ID":"d999d6fd-4e88-49eb-b8ac-f9c1e03baa4c","Type":"ContainerDied","Data":"7285956f5dd8d2eb76fe948a272aea4828a3b3ba3268cdeaba9e4d3c418eee90"}
Oct 31 01:04:09.111365 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:09.111009    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-cb2b8a-openshift-default-75dd976c57-4jgr7" event={"ID":"d999d6fd-4e88-49eb-b8ac-f9c1e03baa4c","Type":"ContainerDied","Data":"c994076e13d596bf71e6cc04e075ebce889bdfc41e8914381fe964883d1fb440"}
Oct 31 01:04:09.111365 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:09.111033    4284 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c994076e13d596bf71e6cc04e075ebce889bdfc41e8914381fe964883d1fb440"
Oct 31 01:04:09.111365 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:09.111052    4284 scope.go:122] "RemoveContainer" containerID="37925e0229115d194189405ddd0fef06de3719e8b611f08a2d0a091046aaa467"
Oct 31 01:04:09.128546 ip-10-0-132-173 kubenswrapper[4284]: E1031 01:04:09.128511    4284 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"37925e0229115d194189405ddd0fef06de3719e8b611f08a2d0a091046aaa467\": container with ID starting with 37925e0229115d194189405ddd0fef06de3719e8b611f08a2d0a091046aaa467 not found: ID does not exist" containerID="37925e0229115d194189405ddd0fef06de3719e8b611f08a2d0a091046aaa467"
Oct 31 01:04:09.132148 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:09.132125    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-cb2b8a-openshift-default-75dd976c57-4jgr7"
Oct 31 01:04:09.195618 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:09.195588    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d999d6fd-4e88-49eb-b8ac-f9c1e03baa4c-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d999d6fd-4e88-49eb-b8ac-f9c1e03baa4c-istio-data\") pod \"d999d6fd-4e88-49eb-b8ac-f9c1e03baa4c\" (UID: \"d999d6fd-4e88-49eb-b8ac-f9c1e03baa4c\") "
Oct 31 01:04:09.195812 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:09.195624    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d999d6fd-4e88-49eb-b8ac-f9c1e03baa4c-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d999d6fd-4e88-49eb-b8ac-f9c1e03baa4c-credential-socket\") pod \"d999d6fd-4e88-49eb-b8ac-f9c1e03baa4c\" (UID: \"d999d6fd-4e88-49eb-b8ac-f9c1e03baa4c\") "
Oct 31 01:04:09.195812 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:09.195696    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d999d6fd-4e88-49eb-b8ac-f9c1e03baa4c-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d999d6fd-4e88-49eb-b8ac-f9c1e03baa4c-istiod-ca-cert\") pod \"d999d6fd-4e88-49eb-b8ac-f9c1e03baa4c\" (UID: \"d999d6fd-4e88-49eb-b8ac-f9c1e03baa4c\") "
Oct 31 01:04:09.195812 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:09.195753    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d999d6fd-4e88-49eb-b8ac-f9c1e03baa4c-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d999d6fd-4e88-49eb-b8ac-f9c1e03baa4c-workload-certs\") pod \"d999d6fd-4e88-49eb-b8ac-f9c1e03baa4c\" (UID: \"d999d6fd-4e88-49eb-b8ac-f9c1e03baa4c\") "
Oct 31 01:04:09.195812 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:09.195810    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d999d6fd-4e88-49eb-b8ac-f9c1e03baa4c-kube-api-access-mqbkt\" (UniqueName: \"kubernetes.io/projected/d999d6fd-4e88-49eb-b8ac-f9c1e03baa4c-kube-api-access-mqbkt\") pod \"d999d6fd-4e88-49eb-b8ac-f9c1e03baa4c\" (UID: \"d999d6fd-4e88-49eb-b8ac-f9c1e03baa4c\") "
Oct 31 01:04:09.196026 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:09.195922    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d999d6fd-4e88-49eb-b8ac-f9c1e03baa4c-istio-data" pod "d999d6fd-4e88-49eb-b8ac-f9c1e03baa4c" (UID: "d999d6fd-4e88-49eb-b8ac-f9c1e03baa4c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:04:09.196026 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:09.195938    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d999d6fd-4e88-49eb-b8ac-f9c1e03baa4c-credential-socket" pod "d999d6fd-4e88-49eb-b8ac-f9c1e03baa4c" (UID: "d999d6fd-4e88-49eb-b8ac-f9c1e03baa4c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:04:09.196026 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:09.195946    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d999d6fd-4e88-49eb-b8ac-f9c1e03baa4c-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d999d6fd-4e88-49eb-b8ac-f9c1e03baa4c-istio-envoy\") pod \"d999d6fd-4e88-49eb-b8ac-f9c1e03baa4c\" (UID: \"d999d6fd-4e88-49eb-b8ac-f9c1e03baa4c\") "
Oct 31 01:04:09.196186 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:09.196026    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d999d6fd-4e88-49eb-b8ac-f9c1e03baa4c-istio-token\" (UniqueName: \"kubernetes.io/projected/d999d6fd-4e88-49eb-b8ac-f9c1e03baa4c-istio-token\") pod \"d999d6fd-4e88-49eb-b8ac-f9c1e03baa4c\" (UID: \"d999d6fd-4e88-49eb-b8ac-f9c1e03baa4c\") "
Oct 31 01:04:09.196186 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:09.196051    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d999d6fd-4e88-49eb-b8ac-f9c1e03baa4c-workload-certs" pod "d999d6fd-4e88-49eb-b8ac-f9c1e03baa4c" (UID: "d999d6fd-4e88-49eb-b8ac-f9c1e03baa4c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:04:09.196186 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:09.196058    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/d999d6fd-4e88-49eb-b8ac-f9c1e03baa4c-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d999d6fd-4e88-49eb-b8ac-f9c1e03baa4c-istio-podinfo\") pod \"d999d6fd-4e88-49eb-b8ac-f9c1e03baa4c\" (UID: \"d999d6fd-4e88-49eb-b8ac-f9c1e03baa4c\") "
Oct 31 01:04:09.196186 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:09.196112    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d999d6fd-4e88-49eb-b8ac-f9c1e03baa4c-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d999d6fd-4e88-49eb-b8ac-f9c1e03baa4c-workload-socket\") pod \"d999d6fd-4e88-49eb-b8ac-f9c1e03baa4c\" (UID: \"d999d6fd-4e88-49eb-b8ac-f9c1e03baa4c\") "
Oct 31 01:04:09.196436 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:09.196206    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d999d6fd-4e88-49eb-b8ac-f9c1e03baa4c-istiod-ca-cert" pod "d999d6fd-4e88-49eb-b8ac-f9c1e03baa4c" (UID: "d999d6fd-4e88-49eb-b8ac-f9c1e03baa4c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 01:04:09.196436 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:09.196423    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d999d6fd-4e88-49eb-b8ac-f9c1e03baa4c-workload-socket" pod "d999d6fd-4e88-49eb-b8ac-f9c1e03baa4c" (UID: "d999d6fd-4e88-49eb-b8ac-f9c1e03baa4c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:04:09.196617 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:09.196601    4284 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d999d6fd-4e88-49eb-b8ac-f9c1e03baa4c-istio-data\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:04:09.196692 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:09.196620    4284 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d999d6fd-4e88-49eb-b8ac-f9c1e03baa4c-credential-socket\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:04:09.196692 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:09.196629    4284 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d999d6fd-4e88-49eb-b8ac-f9c1e03baa4c-istiod-ca-cert\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:04:09.196692 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:09.196637    4284 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d999d6fd-4e88-49eb-b8ac-f9c1e03baa4c-workload-certs\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:04:09.196692 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:09.196646    4284 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d999d6fd-4e88-49eb-b8ac-f9c1e03baa4c-workload-socket\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:04:09.198191 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:09.198157    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d999d6fd-4e88-49eb-b8ac-f9c1e03baa4c-istio-envoy" pod "d999d6fd-4e88-49eb-b8ac-f9c1e03baa4c" (UID: "d999d6fd-4e88-49eb-b8ac-f9c1e03baa4c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:04:09.198440 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:09.198414    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d999d6fd-4e88-49eb-b8ac-f9c1e03baa4c-istio-token" pod "d999d6fd-4e88-49eb-b8ac-f9c1e03baa4c" (UID: "d999d6fd-4e88-49eb-b8ac-f9c1e03baa4c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:04:09.198727 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:09.198699    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d999d6fd-4e88-49eb-b8ac-f9c1e03baa4c-istio-podinfo" pod "d999d6fd-4e88-49eb-b8ac-f9c1e03baa4c" (UID: "d999d6fd-4e88-49eb-b8ac-f9c1e03baa4c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 01:04:09.198784 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:09.198728    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d999d6fd-4e88-49eb-b8ac-f9c1e03baa4c-kube-api-access-mqbkt" pod "d999d6fd-4e88-49eb-b8ac-f9c1e03baa4c" (UID: "d999d6fd-4e88-49eb-b8ac-f9c1e03baa4c"). InnerVolumeSpecName "kube-api-access-mqbkt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:04:09.297907 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:09.297873    4284 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mqbkt\" (UniqueName: \"kubernetes.io/projected/d999d6fd-4e88-49eb-b8ac-f9c1e03baa4c-kube-api-access-mqbkt\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:04:09.297907 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:09.297905    4284 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d999d6fd-4e88-49eb-b8ac-f9c1e03baa4c-istio-envoy\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:04:09.297907 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:09.297915    4284 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d999d6fd-4e88-49eb-b8ac-f9c1e03baa4c-istio-token\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:04:09.298271 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:09.297923    4284 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d999d6fd-4e88-49eb-b8ac-f9c1e03baa4c-istio-podinfo\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:04:10.034814 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:10.034786    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-praxis-b-4ab4aa-5d9969dd59-n4vgd"
Oct 31 01:04:10.118175 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:10.118136    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-cb2b8a-openshift-default-75dd976c57-4jgr7"
Oct 31 01:04:10.258156 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:10.258076    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-cb2b8a-openshift-default-75dd976c57-4jgr7"]
Oct 31 01:04:10.282426 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:10.282388    4284 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-b-cb2b8a-openshift-default-75dd976c57-4jgr7"]
Oct 31 01:04:10.297919 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:10.297885    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:04:11.964342 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:11.964308    4284 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d999d6fd-4e88-49eb-b8ac-f9c1e03baa4c" path="/var/lib/kubelet/pods/d999d6fd-4e88-49eb-b8ac-f9c1e03baa4c/volumes"
Oct 31 01:04:12.636461 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:12.636430    4284 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-praxis-a-8c08e7/praxis-route-8c08e7-kserve-7966879b6c-bpw2m"]
Oct 31 01:04:12.636995 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:12.636983    4284 cpu_manager.go:414] "containerMap: removing container" podUID="d999d6fd-4e88-49eb-b8ac-f9c1e03baa4c" containerName="istio-proxy"
Oct 31 01:04:12.637040 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:12.636997    4284 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d999d6fd-4e88-49eb-b8ac-f9c1e03baa4c" containerName="istio-proxy"
Oct 31 01:04:12.637040 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:12.637011    4284 cpu_manager.go:414] "containerMap: removing container" podUID="d999d6fd-4e88-49eb-b8ac-f9c1e03baa4c" containerName="istio-proxy"
Oct 31 01:04:12.637040 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:12.637016    4284 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d999d6fd-4e88-49eb-b8ac-f9c1e03baa4c" containerName="istio-proxy"
Oct 31 01:04:12.637140 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:12.637110    4284 memory_manager.go:365] "RemoveStaleState removing state" podUID="d999d6fd-4e88-49eb-b8ac-f9c1e03baa4c" containerName="istio-proxy"
Oct 31 01:04:12.637140 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:12.637117    4284 memory_manager.go:365] "RemoveStaleState removing state" podUID="d999d6fd-4e88-49eb-b8ac-f9c1e03baa4c" containerName="istio-proxy"
Oct 31 01:04:12.641577 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:12.641557    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-praxis-a-8c08e7/praxis-route-8c08e7-kserve-7966879b6c-bpw2m"
Oct 31 01:04:12.698077 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:12.698041    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-praxis-a-8c08e7/praxis-route-8c08e7-kserve-7966879b6c-bpw2m"]
Oct 31 01:04:12.736906 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:12.736870    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:04:12.738045 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:12.738023    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f6f3f062-5d9e-4051-9c1a-9cf20b93ba0a-home\") pod \"praxis-route-8c08e7-kserve-7966879b6c-bpw2m\" (UID: \"f6f3f062-5d9e-4051-9c1a-9cf20b93ba0a\") " pod="ai-tenant-e2e-praxis-a-8c08e7/praxis-route-8c08e7-kserve-7966879b6c-bpw2m"
Oct 31 01:04:12.738103 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:12.738055    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qvrb6\" (UniqueName: \"kubernetes.io/projected/f6f3f062-5d9e-4051-9c1a-9cf20b93ba0a-kube-api-access-qvrb6\") pod \"praxis-route-8c08e7-kserve-7966879b6c-bpw2m\" (UID: \"f6f3f062-5d9e-4051-9c1a-9cf20b93ba0a\") " pod="ai-tenant-e2e-praxis-a-8c08e7/praxis-route-8c08e7-kserve-7966879b6c-bpw2m"
Oct 31 01:04:12.738145 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:12.738132    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f6f3f062-5d9e-4051-9c1a-9cf20b93ba0a-tmp-dir\") pod \"praxis-route-8c08e7-kserve-7966879b6c-bpw2m\" (UID: \"f6f3f062-5d9e-4051-9c1a-9cf20b93ba0a\") " pod="ai-tenant-e2e-praxis-a-8c08e7/praxis-route-8c08e7-kserve-7966879b6c-bpw2m"
Oct 31 01:04:12.738191 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:12.738165    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f6f3f062-5d9e-4051-9c1a-9cf20b93ba0a-dshm\") pod \"praxis-route-8c08e7-kserve-7966879b6c-bpw2m\" (UID: \"f6f3f062-5d9e-4051-9c1a-9cf20b93ba0a\") " pod="ai-tenant-e2e-praxis-a-8c08e7/praxis-route-8c08e7-kserve-7966879b6c-bpw2m"
Oct 31 01:04:12.738289 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:12.738274    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f6f3f062-5d9e-4051-9c1a-9cf20b93ba0a-model-cache\") pod \"praxis-route-8c08e7-kserve-7966879b6c-bpw2m\" (UID: \"f6f3f062-5d9e-4051-9c1a-9cf20b93ba0a\") " pod="ai-tenant-e2e-praxis-a-8c08e7/praxis-route-8c08e7-kserve-7966879b6c-bpw2m"
Oct 31 01:04:12.738406 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:12.738379    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f6f3f062-5d9e-4051-9c1a-9cf20b93ba0a-tls-certs\") pod \"praxis-route-8c08e7-kserve-7966879b6c-bpw2m\" (UID: \"f6f3f062-5d9e-4051-9c1a-9cf20b93ba0a\") " pod="ai-tenant-e2e-praxis-a-8c08e7/praxis-route-8c08e7-kserve-7966879b6c-bpw2m"
Oct 31 01:04:12.742343 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:12.742321    4284 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-praxis-a-8c08e7\"/\"openshift-service-ca.crt\""
Oct 31 01:04:12.742343 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:12.742331    4284 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-praxis-a-8c08e7\"/\"default-dockercfg-f8shw\""
Oct 31 01:04:12.742548 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:12.742376    4284 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-praxis-a-8c08e7\"/\"praxis-route-8c08e7-kserve-self-signed-certs\""
Oct 31 01:04:12.742548 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:12.742434    4284 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-praxis-a-8c08e7\"/\"kube-root-ca.crt\""
Oct 31 01:04:12.839804 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:12.839772    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f6f3f062-5d9e-4051-9c1a-9cf20b93ba0a-model-cache\") pod \"praxis-route-8c08e7-kserve-7966879b6c-bpw2m\" (UID: \"f6f3f062-5d9e-4051-9c1a-9cf20b93ba0a\") " pod="ai-tenant-e2e-praxis-a-8c08e7/praxis-route-8c08e7-kserve-7966879b6c-bpw2m"
Oct 31 01:04:12.840016 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:12.839832    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f6f3f062-5d9e-4051-9c1a-9cf20b93ba0a-tls-certs\") pod \"praxis-route-8c08e7-kserve-7966879b6c-bpw2m\" (UID: \"f6f3f062-5d9e-4051-9c1a-9cf20b93ba0a\") " pod="ai-tenant-e2e-praxis-a-8c08e7/praxis-route-8c08e7-kserve-7966879b6c-bpw2m"
Oct 31 01:04:12.840016 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:12.839945    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f6f3f062-5d9e-4051-9c1a-9cf20b93ba0a-home\") pod \"praxis-route-8c08e7-kserve-7966879b6c-bpw2m\" (UID: \"f6f3f062-5d9e-4051-9c1a-9cf20b93ba0a\") " pod="ai-tenant-e2e-praxis-a-8c08e7/praxis-route-8c08e7-kserve-7966879b6c-bpw2m"
Oct 31 01:04:12.840016 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:12.839970    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qvrb6\" (UniqueName: \"kubernetes.io/projected/f6f3f062-5d9e-4051-9c1a-9cf20b93ba0a-kube-api-access-qvrb6\") pod \"praxis-route-8c08e7-kserve-7966879b6c-bpw2m\" (UID: \"f6f3f062-5d9e-4051-9c1a-9cf20b93ba0a\") " pod="ai-tenant-e2e-praxis-a-8c08e7/praxis-route-8c08e7-kserve-7966879b6c-bpw2m"
Oct 31 01:04:12.840183 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:12.840014    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f6f3f062-5d9e-4051-9c1a-9cf20b93ba0a-tmp-dir\") pod \"praxis-route-8c08e7-kserve-7966879b6c-bpw2m\" (UID: \"f6f3f062-5d9e-4051-9c1a-9cf20b93ba0a\") " pod="ai-tenant-e2e-praxis-a-8c08e7/praxis-route-8c08e7-kserve-7966879b6c-bpw2m"
Oct 31 01:04:12.840183 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:12.840061    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f6f3f062-5d9e-4051-9c1a-9cf20b93ba0a-dshm\") pod \"praxis-route-8c08e7-kserve-7966879b6c-bpw2m\" (UID: \"f6f3f062-5d9e-4051-9c1a-9cf20b93ba0a\") " pod="ai-tenant-e2e-praxis-a-8c08e7/praxis-route-8c08e7-kserve-7966879b6c-bpw2m"
Oct 31 01:04:12.840304 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:12.840258    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f6f3f062-5d9e-4051-9c1a-9cf20b93ba0a-model-cache\") pod \"praxis-route-8c08e7-kserve-7966879b6c-bpw2m\" (UID: \"f6f3f062-5d9e-4051-9c1a-9cf20b93ba0a\") " pod="ai-tenant-e2e-praxis-a-8c08e7/praxis-route-8c08e7-kserve-7966879b6c-bpw2m"
Oct 31 01:04:12.840428 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:12.840404    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f6f3f062-5d9e-4051-9c1a-9cf20b93ba0a-tmp-dir\") pod \"praxis-route-8c08e7-kserve-7966879b6c-bpw2m\" (UID: \"f6f3f062-5d9e-4051-9c1a-9cf20b93ba0a\") " pod="ai-tenant-e2e-praxis-a-8c08e7/praxis-route-8c08e7-kserve-7966879b6c-bpw2m"
Oct 31 01:04:12.840561 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:12.840538    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f6f3f062-5d9e-4051-9c1a-9cf20b93ba0a-home\") pod \"praxis-route-8c08e7-kserve-7966879b6c-bpw2m\" (UID: \"f6f3f062-5d9e-4051-9c1a-9cf20b93ba0a\") " pod="ai-tenant-e2e-praxis-a-8c08e7/praxis-route-8c08e7-kserve-7966879b6c-bpw2m"
Oct 31 01:04:12.842393 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:12.842374    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f6f3f062-5d9e-4051-9c1a-9cf20b93ba0a-dshm\") pod \"praxis-route-8c08e7-kserve-7966879b6c-bpw2m\" (UID: \"f6f3f062-5d9e-4051-9c1a-9cf20b93ba0a\") " pod="ai-tenant-e2e-praxis-a-8c08e7/praxis-route-8c08e7-kserve-7966879b6c-bpw2m"
Oct 31 01:04:12.842596 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:12.842581    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f6f3f062-5d9e-4051-9c1a-9cf20b93ba0a-tls-certs\") pod \"praxis-route-8c08e7-kserve-7966879b6c-bpw2m\" (UID: \"f6f3f062-5d9e-4051-9c1a-9cf20b93ba0a\") " pod="ai-tenant-e2e-praxis-a-8c08e7/praxis-route-8c08e7-kserve-7966879b6c-bpw2m"
Oct 31 01:04:12.899039 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:12.898975    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qvrb6\" (UniqueName: \"kubernetes.io/projected/f6f3f062-5d9e-4051-9c1a-9cf20b93ba0a-kube-api-access-qvrb6\") pod \"praxis-route-8c08e7-kserve-7966879b6c-bpw2m\" (UID: \"f6f3f062-5d9e-4051-9c1a-9cf20b93ba0a\") " pod="ai-tenant-e2e-praxis-a-8c08e7/praxis-route-8c08e7-kserve-7966879b6c-bpw2m"
Oct 31 01:04:12.953046 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:12.953016    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-praxis-a-8c08e7/praxis-route-8c08e7-kserve-7966879b6c-bpw2m"
Oct 31 01:04:13.327171 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:13.327150    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-praxis-a-8c08e7/praxis-route-8c08e7-kserve-7966879b6c-bpw2m"]
Oct 31 01:04:13.329248 ip-10-0-132-173 kubenswrapper[4284]: W1031 01:04:13.329195    4284 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf6f3f062_5d9e_4051_9c1a_9cf20b93ba0a.slice/crio-f18ff32598c388332c67c4e7e436e0f96477c419b1b244717fd1fe1e6cc10a38 WatchSource:0}: Error finding container f18ff32598c388332c67c4e7e436e0f96477c419b1b244717fd1fe1e6cc10a38: Status 404 returned error can't find the container with id f18ff32598c388332c67c4e7e436e0f96477c419b1b244717fd1fe1e6cc10a38
Oct 31 01:04:14.155361 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:14.155327    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-praxis-a-8c08e7/praxis-route-8c08e7-kserve-7966879b6c-bpw2m" event={"ID":"f6f3f062-5d9e-4051-9c1a-9cf20b93ba0a","Type":"ContainerStarted","Data":"ce30606a226b452e005599060768efa00a35cbd3f569aff22c4a48db86695359"}
Oct 31 01:04:14.155558 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:14.155369    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-praxis-a-8c08e7/praxis-route-8c08e7-kserve-7966879b6c-bpw2m" event={"ID":"f6f3f062-5d9e-4051-9c1a-9cf20b93ba0a","Type":"ContainerStarted","Data":"f18ff32598c388332c67c4e7e436e0f96477c419b1b244717fd1fe1e6cc10a38"}
Oct 31 01:04:14.276354 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:14.276306    4284 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="ai-tenant-e2e-praxis-a-8c08e7/praxis-route-8c08e7-kserve-7966879b6c-bpw2m" podStartSLOduration=2.276293101 podStartE2EDuration="2.276293101s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 01:04:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 01:04:14.274399655 +0000 UTC m=+2168.419406582" watchObservedRunningTime="2026-10-31 01:04:14.276293101 +0000 UTC m=+2168.421300017"
Oct 31 01:04:14.954225 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:14.954190    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-praxis-a-8c08e7/praxis-route-8c08e7-kserve-7966879b6c-bpw2m"
Oct 31 01:04:15.226137 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:15.226052    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:04:16.090003 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:16.089978    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-praxis-a-add5bf-564b7f99c5-dhn9p"
Oct 31 01:04:17.016163 ip-10-0-132-173 kubenswrapper[4284]: E1031 01:04:17.016128    4284 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fd8sp" containerName="extractor"
Oct 31 01:04:18.515196 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:18.515149    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:04:20.160609 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:20.160550    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:04:22.953500 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:22.953466    4284 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-praxis-a-8c08e7/praxis-route-8c08e7-kserve-7966879b6c-bpw2m"
Oct 31 01:04:22.966315 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:22.966289    4284 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-praxis-a-8c08e7/praxis-route-8c08e7-kserve-7966879b6c-bpw2m"
Oct 31 01:04:23.223001 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:23.222925    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-praxis-a-8c08e7/praxis-route-8c08e7-kserve-7966879b6c-bpw2m"
Oct 31 01:04:25.122632 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:25.122585    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:04:27.375605 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:27.375567    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:04:30.221116 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:30.221075    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:04:35.548102 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:35.548061    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:04:40.519676 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:40.519640    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:04:41.467339 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:41.467307    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-vqsxn"]
Oct 31 01:04:41.467623 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:41.467522    4284 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-vqsxn" podUID="d3800ace-ae94-47da-ac65-218d3830b13d" containerName="istio-proxy" containerID="cri-o://5ced87cca8c6f86b39c0ff7814529364b6bb73441ac5e8905e92c7629fe5d76d" gracePeriod=30
Oct 31 01:04:43.736674 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:43.736636    4284 patch_prober.go:29] interesting pod/e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-vqsxn container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 31 01:04:43.737060 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:43.736704    4284 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-vqsxn" podUID="d3800ace-ae94-47da-ac65-218d3830b13d" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 31 01:04:45.141866 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:45.141829    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-29ece5-openshift-default-6fcb86dbc-f75sz"]
Oct 31 01:04:45.142365 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:45.142024    4284 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w2-29ece5-openshift-default-6fcb86dbc-f75sz" podUID="ffd0eb46-1e6b-4bcc-95bc-eaca5254c056" containerName="istio-proxy" containerID="cri-o://6cf3ff5d8abe6d686ff1ea7ab91616d4b561140c6ea5cf6e905eb5c37ed8b79c" gracePeriod=30
Oct 31 01:04:45.307950 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:45.307916    4284 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-b-4ab4aa-7d9ff55bd6-g7vx5"]
Oct 31 01:04:45.312207 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:45.312188    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-4ab4aa-7d9ff55bd6-g7vx5"
Oct 31 01:04:45.366316 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:45.366286    4284 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-praxis-b-4ab4aa-serving-cert\""
Oct 31 01:04:45.381568 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:45.381540    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-b-4ab4aa-7d9ff55bd6-g7vx5"]
Oct 31 01:04:45.440507 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:45.440475    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1000e907-d373-47f0-ac62-db82b1d816b3-openshift-service-ca\") pod \"maas-api-e2e-praxis-b-4ab4aa-7d9ff55bd6-g7vx5\" (UID: \"1000e907-d373-47f0-ac62-db82b1d816b3\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-4ab4aa-7d9ff55bd6-g7vx5"
Oct 31 01:04:45.440700 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:45.440615    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bd86x\" (UniqueName: \"kubernetes.io/projected/1000e907-d373-47f0-ac62-db82b1d816b3-kube-api-access-bd86x\") pod \"maas-api-e2e-praxis-b-4ab4aa-7d9ff55bd6-g7vx5\" (UID: \"1000e907-d373-47f0-ac62-db82b1d816b3\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-4ab4aa-7d9ff55bd6-g7vx5"
Oct 31 01:04:45.440767 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:45.440701    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1000e907-d373-47f0-ac62-db82b1d816b3-metrics-certs\") pod \"maas-api-e2e-praxis-b-4ab4aa-7d9ff55bd6-g7vx5\" (UID: \"1000e907-d373-47f0-ac62-db82b1d816b3\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-4ab4aa-7d9ff55bd6-g7vx5"
Oct 31 01:04:45.440767 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:45.440754    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1000e907-d373-47f0-ac62-db82b1d816b3-maas-api-tls\") pod \"maas-api-e2e-praxis-b-4ab4aa-7d9ff55bd6-g7vx5\" (UID: \"1000e907-d373-47f0-ac62-db82b1d816b3\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-4ab4aa-7d9ff55bd6-g7vx5"
Oct 31 01:04:45.494409 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:45.494363    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:04:45.542306 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:45.542265    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1000e907-d373-47f0-ac62-db82b1d816b3-openshift-service-ca\") pod \"maas-api-e2e-praxis-b-4ab4aa-7d9ff55bd6-g7vx5\" (UID: \"1000e907-d373-47f0-ac62-db82b1d816b3\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-4ab4aa-7d9ff55bd6-g7vx5"
Oct 31 01:04:45.542515 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:45.542371    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bd86x\" (UniqueName: \"kubernetes.io/projected/1000e907-d373-47f0-ac62-db82b1d816b3-kube-api-access-bd86x\") pod \"maas-api-e2e-praxis-b-4ab4aa-7d9ff55bd6-g7vx5\" (UID: \"1000e907-d373-47f0-ac62-db82b1d816b3\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-4ab4aa-7d9ff55bd6-g7vx5"
Oct 31 01:04:45.542515 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:45.542414    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1000e907-d373-47f0-ac62-db82b1d816b3-metrics-certs\") pod \"maas-api-e2e-praxis-b-4ab4aa-7d9ff55bd6-g7vx5\" (UID: \"1000e907-d373-47f0-ac62-db82b1d816b3\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-4ab4aa-7d9ff55bd6-g7vx5"
Oct 31 01:04:45.542515 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:45.542443    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1000e907-d373-47f0-ac62-db82b1d816b3-maas-api-tls\") pod \"maas-api-e2e-praxis-b-4ab4aa-7d9ff55bd6-g7vx5\" (UID: \"1000e907-d373-47f0-ac62-db82b1d816b3\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-4ab4aa-7d9ff55bd6-g7vx5"
Oct 31 01:04:45.542684 ip-10-0-132-173 kubenswrapper[4284]: E1031 01:04:45.542611    4284 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-praxis-b-4ab4aa-serving-cert: secret "maas-api-e2e-praxis-b-4ab4aa-serving-cert" not found
Oct 31 01:04:45.542732 ip-10-0-132-173 kubenswrapper[4284]: E1031 01:04:45.542687    4284 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1000e907-d373-47f0-ac62-db82b1d816b3-maas-api-tls podName:1000e907-d373-47f0-ac62-db82b1d816b3 nodeName:}" failed. No retries permitted until 2026-10-31 01:04:46.042664067 +0000 UTC m=+2200.187670977 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/1000e907-d373-47f0-ac62-db82b1d816b3-maas-api-tls") pod "maas-api-e2e-praxis-b-4ab4aa-7d9ff55bd6-g7vx5" (UID: "1000e907-d373-47f0-ac62-db82b1d816b3") : secret "maas-api-e2e-praxis-b-4ab4aa-serving-cert" not found
Oct 31 01:04:45.543114 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:45.543035    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1000e907-d373-47f0-ac62-db82b1d816b3-openshift-service-ca\") pod \"maas-api-e2e-praxis-b-4ab4aa-7d9ff55bd6-g7vx5\" (UID: \"1000e907-d373-47f0-ac62-db82b1d816b3\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-4ab4aa-7d9ff55bd6-g7vx5"
Oct 31 01:04:45.544925 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:45.544903    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1000e907-d373-47f0-ac62-db82b1d816b3-metrics-certs\") pod \"maas-api-e2e-praxis-b-4ab4aa-7d9ff55bd6-g7vx5\" (UID: \"1000e907-d373-47f0-ac62-db82b1d816b3\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-4ab4aa-7d9ff55bd6-g7vx5"
Oct 31 01:04:45.609192 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:45.609153    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bd86x\" (UniqueName: \"kubernetes.io/projected/1000e907-d373-47f0-ac62-db82b1d816b3-kube-api-access-bd86x\") pod \"maas-api-e2e-praxis-b-4ab4aa-7d9ff55bd6-g7vx5\" (UID: \"1000e907-d373-47f0-ac62-db82b1d816b3\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-4ab4aa-7d9ff55bd6-g7vx5"
Oct 31 01:04:45.693391 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:45.693301    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:04:45.849830 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:45.849786    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:04:46.052796 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:46.052705    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1000e907-d373-47f0-ac62-db82b1d816b3-maas-api-tls\") pod \"maas-api-e2e-praxis-b-4ab4aa-7d9ff55bd6-g7vx5\" (UID: \"1000e907-d373-47f0-ac62-db82b1d816b3\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-4ab4aa-7d9ff55bd6-g7vx5"
Oct 31 01:04:46.055291 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:46.055263    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1000e907-d373-47f0-ac62-db82b1d816b3-maas-api-tls\") pod \"maas-api-e2e-praxis-b-4ab4aa-7d9ff55bd6-g7vx5\" (UID: \"1000e907-d373-47f0-ac62-db82b1d816b3\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-4ab4aa-7d9ff55bd6-g7vx5"
Oct 31 01:04:46.226610 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:46.226576    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-4ab4aa-7d9ff55bd6-g7vx5"
Oct 31 01:04:46.602593 ip-10-0-132-173 kubenswrapper[4284]: W1031 01:04:46.602554    4284 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1000e907_d373_47f0_ac62_db82b1d816b3.slice/crio-f8c8e8cb4e2ccb35c70ea6a9040c0bcce7d2c528805cb0e98fe3b09e4bb733ab WatchSource:0}: Error finding container f8c8e8cb4e2ccb35c70ea6a9040c0bcce7d2c528805cb0e98fe3b09e4bb733ab: Status 404 returned error can't find the container with id f8c8e8cb4e2ccb35c70ea6a9040c0bcce7d2c528805cb0e98fe3b09e4bb733ab
Oct 31 01:04:46.602728 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:46.602591    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-b-4ab4aa-7d9ff55bd6-g7vx5"]
Oct 31 01:04:46.746341 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:46.746278    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-vqsxn"
Oct 31 01:04:46.863502 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:46.863466    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d3800ace-ae94-47da-ac65-218d3830b13d-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d3800ace-ae94-47da-ac65-218d3830b13d-workload-socket\") pod \"d3800ace-ae94-47da-ac65-218d3830b13d\" (UID: \"d3800ace-ae94-47da-ac65-218d3830b13d\") "
Oct 31 01:04:46.863685 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:46.863510    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d3800ace-ae94-47da-ac65-218d3830b13d-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d3800ace-ae94-47da-ac65-218d3830b13d-istio-data\") pod \"d3800ace-ae94-47da-ac65-218d3830b13d\" (UID: \"d3800ace-ae94-47da-ac65-218d3830b13d\") "
Oct 31 01:04:46.863685 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:46.863557    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d3800ace-ae94-47da-ac65-218d3830b13d-kube-api-access-kg8bn\" (UniqueName: \"kubernetes.io/projected/d3800ace-ae94-47da-ac65-218d3830b13d-kube-api-access-kg8bn\") pod \"d3800ace-ae94-47da-ac65-218d3830b13d\" (UID: \"d3800ace-ae94-47da-ac65-218d3830b13d\") "
Oct 31 01:04:46.863685 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:46.863585    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/d3800ace-ae94-47da-ac65-218d3830b13d-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d3800ace-ae94-47da-ac65-218d3830b13d-istio-podinfo\") pod \"d3800ace-ae94-47da-ac65-218d3830b13d\" (UID: \"d3800ace-ae94-47da-ac65-218d3830b13d\") "
Oct 31 01:04:46.863685 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:46.863603    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d3800ace-ae94-47da-ac65-218d3830b13d-istio-token\" (UniqueName: \"kubernetes.io/projected/d3800ace-ae94-47da-ac65-218d3830b13d-istio-token\") pod \"d3800ace-ae94-47da-ac65-218d3830b13d\" (UID: \"d3800ace-ae94-47da-ac65-218d3830b13d\") "
Oct 31 01:04:46.863685 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:46.863679    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d3800ace-ae94-47da-ac65-218d3830b13d-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d3800ace-ae94-47da-ac65-218d3830b13d-workload-certs\") pod \"d3800ace-ae94-47da-ac65-218d3830b13d\" (UID: \"d3800ace-ae94-47da-ac65-218d3830b13d\") "
Oct 31 01:04:46.863945 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:46.863755    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d3800ace-ae94-47da-ac65-218d3830b13d-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d3800ace-ae94-47da-ac65-218d3830b13d-istio-envoy\") pod \"d3800ace-ae94-47da-ac65-218d3830b13d\" (UID: \"d3800ace-ae94-47da-ac65-218d3830b13d\") "
Oct 31 01:04:46.863945 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:46.863814    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d3800ace-ae94-47da-ac65-218d3830b13d-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d3800ace-ae94-47da-ac65-218d3830b13d-credential-socket\") pod \"d3800ace-ae94-47da-ac65-218d3830b13d\" (UID: \"d3800ace-ae94-47da-ac65-218d3830b13d\") "
Oct 31 01:04:46.863945 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:46.863845    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d3800ace-ae94-47da-ac65-218d3830b13d-istio-data" pod "d3800ace-ae94-47da-ac65-218d3830b13d" (UID: "d3800ace-ae94-47da-ac65-218d3830b13d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:04:46.864128 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:46.863960    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d3800ace-ae94-47da-ac65-218d3830b13d-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d3800ace-ae94-47da-ac65-218d3830b13d-istiod-ca-cert\") pod \"d3800ace-ae94-47da-ac65-218d3830b13d\" (UID: \"d3800ace-ae94-47da-ac65-218d3830b13d\") "
Oct 31 01:04:46.864128 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:46.864064    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d3800ace-ae94-47da-ac65-218d3830b13d-workload-certs" pod "d3800ace-ae94-47da-ac65-218d3830b13d" (UID: "d3800ace-ae94-47da-ac65-218d3830b13d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:04:46.864274 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:46.864160    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d3800ace-ae94-47da-ac65-218d3830b13d-workload-socket" pod "d3800ace-ae94-47da-ac65-218d3830b13d" (UID: "d3800ace-ae94-47da-ac65-218d3830b13d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:04:46.864469 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:46.864351    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d3800ace-ae94-47da-ac65-218d3830b13d-credential-socket" pod "d3800ace-ae94-47da-ac65-218d3830b13d" (UID: "d3800ace-ae94-47da-ac65-218d3830b13d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:04:46.864564 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:46.864535    4284 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d3800ace-ae94-47da-ac65-218d3830b13d-workload-socket\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:04:46.864564 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:46.864555    4284 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d3800ace-ae94-47da-ac65-218d3830b13d-istio-data\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:04:46.864689 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:46.864572    4284 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d3800ace-ae94-47da-ac65-218d3830b13d-workload-certs\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:04:46.864689 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:46.864586    4284 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d3800ace-ae94-47da-ac65-218d3830b13d-credential-socket\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:04:46.864689 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:46.864660    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d3800ace-ae94-47da-ac65-218d3830b13d-istiod-ca-cert" pod "d3800ace-ae94-47da-ac65-218d3830b13d" (UID: "d3800ace-ae94-47da-ac65-218d3830b13d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 01:04:46.866069 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:46.866037    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d3800ace-ae94-47da-ac65-218d3830b13d-istio-envoy" pod "d3800ace-ae94-47da-ac65-218d3830b13d" (UID: "d3800ace-ae94-47da-ac65-218d3830b13d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:04:46.866381 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:46.866085    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d3800ace-ae94-47da-ac65-218d3830b13d-kube-api-access-kg8bn" pod "d3800ace-ae94-47da-ac65-218d3830b13d" (UID: "d3800ace-ae94-47da-ac65-218d3830b13d"). InnerVolumeSpecName "kube-api-access-kg8bn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:04:46.866381 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:46.866095    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d3800ace-ae94-47da-ac65-218d3830b13d-istio-podinfo" pod "d3800ace-ae94-47da-ac65-218d3830b13d" (UID: "d3800ace-ae94-47da-ac65-218d3830b13d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 01:04:46.866381 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:46.866359    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d3800ace-ae94-47da-ac65-218d3830b13d-istio-token" pod "d3800ace-ae94-47da-ac65-218d3830b13d" (UID: "d3800ace-ae94-47da-ac65-218d3830b13d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:04:46.966134 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:46.966100    4284 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kg8bn\" (UniqueName: \"kubernetes.io/projected/d3800ace-ae94-47da-ac65-218d3830b13d-kube-api-access-kg8bn\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:04:46.966134 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:46.966131    4284 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d3800ace-ae94-47da-ac65-218d3830b13d-istio-podinfo\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:04:46.966134 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:46.966141    4284 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d3800ace-ae94-47da-ac65-218d3830b13d-istio-token\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:04:46.966480 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:46.966152    4284 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d3800ace-ae94-47da-ac65-218d3830b13d-istio-envoy\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:04:46.966480 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:46.966161    4284 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d3800ace-ae94-47da-ac65-218d3830b13d-istiod-ca-cert\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:04:47.018043 ip-10-0-132-173 kubenswrapper[4284]: E1031 01:04:47.017303    4284 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fd8sp" containerName="extractor"
Oct 31 01:04:47.390170 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:47.390072    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-4ab4aa-7d9ff55bd6-g7vx5" event={"ID":"1000e907-d373-47f0-ac62-db82b1d816b3","Type":"ContainerStarted","Data":"f8c8e8cb4e2ccb35c70ea6a9040c0bcce7d2c528805cb0e98fe3b09e4bb733ab"}
Oct 31 01:04:47.391536 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:47.391510    4284 generic.go:356] "Generic (PLEG): container finished" podID="d3800ace-ae94-47da-ac65-218d3830b13d" containerID="5ced87cca8c6f86b39c0ff7814529364b6bb73441ac5e8905e92c7629fe5d76d" exitCode=0
Oct 31 01:04:47.391680 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:47.391566    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-vqsxn"
Oct 31 01:04:47.391680 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:47.391582    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-vqsxn" event={"ID":"d3800ace-ae94-47da-ac65-218d3830b13d","Type":"ContainerDied","Data":"5ced87cca8c6f86b39c0ff7814529364b6bb73441ac5e8905e92c7629fe5d76d"}
Oct 31 01:04:47.391680 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:47.391625    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-vqsxn" event={"ID":"d3800ace-ae94-47da-ac65-218d3830b13d","Type":"ContainerDied","Data":"a12058e8f0f7027e584d94cec8efb1aab88277a2733f3de23399a1fc7afef8fa"}
Oct 31 01:04:47.391680 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:47.391657    4284 scope.go:122] "RemoveContainer" containerID="5ced87cca8c6f86b39c0ff7814529364b6bb73441ac5e8905e92c7629fe5d76d"
Oct 31 01:04:47.402722 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:47.402706    4284 scope.go:122] "RemoveContainer" containerID="5ced87cca8c6f86b39c0ff7814529364b6bb73441ac5e8905e92c7629fe5d76d"
Oct 31 01:04:47.402990 ip-10-0-132-173 kubenswrapper[4284]: E1031 01:04:47.402966    4284 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5ced87cca8c6f86b39c0ff7814529364b6bb73441ac5e8905e92c7629fe5d76d\": container with ID starting with 5ced87cca8c6f86b39c0ff7814529364b6bb73441ac5e8905e92c7629fe5d76d not found: ID does not exist" containerID="5ced87cca8c6f86b39c0ff7814529364b6bb73441ac5e8905e92c7629fe5d76d"
Oct 31 01:04:47.403037 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:47.402997    4284 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5ced87cca8c6f86b39c0ff7814529364b6bb73441ac5e8905e92c7629fe5d76d"} err="failed to get container status \"5ced87cca8c6f86b39c0ff7814529364b6bb73441ac5e8905e92c7629fe5d76d\": rpc error: code = NotFound desc = could not find container \"5ced87cca8c6f86b39c0ff7814529364b6bb73441ac5e8905e92c7629fe5d76d\": container with ID starting with 5ced87cca8c6f86b39c0ff7814529364b6bb73441ac5e8905e92c7629fe5d76d not found: ID does not exist"
Oct 31 01:04:47.568026 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:47.563862    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-vqsxn"]
Oct 31 01:04:47.608342 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:47.608314    4284 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-vqsxn"]
Oct 31 01:04:47.964247 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:47.964189    4284 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d3800ace-ae94-47da-ac65-218d3830b13d" path="/var/lib/kubelet/pods/d3800ace-ae94-47da-ac65-218d3830b13d/volumes"
Oct 31 01:04:48.399759 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:48.399658    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-4ab4aa-7d9ff55bd6-g7vx5" event={"ID":"1000e907-d373-47f0-ac62-db82b1d816b3","Type":"ContainerStarted","Data":"e4ecd2e35dc6e866147c71f439b1a6035e375ca4d77b8a98f47cfd48f8fc2d65"}
Oct 31 01:04:48.399759 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:48.399728    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-4ab4aa-7d9ff55bd6-g7vx5"
Oct 31 01:04:48.518011 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:48.517949    4284 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-4ab4aa-7d9ff55bd6-g7vx5" podStartSLOduration=2.567569938 podStartE2EDuration="3.517932425s" totalImagesPullingTime="950.362487ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 01:04:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 01:04:48.516171874 +0000 UTC m=+2202.661178790" watchObservedRunningTime="2026-10-31 01:04:48.517932425 +0000 UTC m=+2202.662939340"
Oct 31 01:04:50.273933 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:50.273896    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-eee7f1-d9bf6db74-ggvqp"]
Oct 31 01:04:50.274300 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:50.274148    4284 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-eee7f1-d9bf6db74-ggvqp" podUID="d5b7cbb4-de93-4a9a-abd3-b69636f595dd" containerName="maas-api" containerID="cri-o://1ee37fad3f92b4642d7b48b8f35f45711b99249469ab146987ada980b9b06ab0" gracePeriod=30
Oct 31 01:04:50.379593 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:50.379516    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:04:50.422661 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:50.422635    4284 generic.go:356] "Generic (PLEG): container finished" podID="ffd0eb46-1e6b-4bcc-95bc-eaca5254c056" containerID="6cf3ff5d8abe6d686ff1ea7ab91616d4b561140c6ea5cf6e905eb5c37ed8b79c" exitCode=0
Oct 31 01:04:50.422795 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:50.422710    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-29ece5-openshift-default-6fcb86dbc-f75sz" event={"ID":"ffd0eb46-1e6b-4bcc-95bc-eaca5254c056","Type":"ContainerDied","Data":"6cf3ff5d8abe6d686ff1ea7ab91616d4b561140c6ea5cf6e905eb5c37ed8b79c"}
Oct 31 01:04:50.424678 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:50.424648    4284 generic.go:356] "Generic (PLEG): container finished" podID="d5b7cbb4-de93-4a9a-abd3-b69636f595dd" containerID="1ee37fad3f92b4642d7b48b8f35f45711b99249469ab146987ada980b9b06ab0" exitCode=0
Oct 31 01:04:50.424776 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:50.424694    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-eee7f1-d9bf6db74-ggvqp" event={"ID":"d5b7cbb4-de93-4a9a-abd3-b69636f595dd","Type":"ContainerDied","Data":"1ee37fad3f92b4642d7b48b8f35f45711b99249469ab146987ada980b9b06ab0"}
Oct 31 01:04:50.439240 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:50.439189    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-29ece5-openshift-default-6fcb86dbc-f75sz"
Oct 31 01:04:50.519269 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:50.513611    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ffd0eb46-1e6b-4bcc-95bc-eaca5254c056-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ffd0eb46-1e6b-4bcc-95bc-eaca5254c056-istio-data\") pod \"ffd0eb46-1e6b-4bcc-95bc-eaca5254c056\" (UID: \"ffd0eb46-1e6b-4bcc-95bc-eaca5254c056\") "
Oct 31 01:04:50.519269 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:50.513681    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ffd0eb46-1e6b-4bcc-95bc-eaca5254c056-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ffd0eb46-1e6b-4bcc-95bc-eaca5254c056-workload-certs\") pod \"ffd0eb46-1e6b-4bcc-95bc-eaca5254c056\" (UID: \"ffd0eb46-1e6b-4bcc-95bc-eaca5254c056\") "
Oct 31 01:04:50.519269 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:50.513754    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ffd0eb46-1e6b-4bcc-95bc-eaca5254c056-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ffd0eb46-1e6b-4bcc-95bc-eaca5254c056-istio-envoy\") pod \"ffd0eb46-1e6b-4bcc-95bc-eaca5254c056\" (UID: \"ffd0eb46-1e6b-4bcc-95bc-eaca5254c056\") "
Oct 31 01:04:50.519269 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:50.513794    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ffd0eb46-1e6b-4bcc-95bc-eaca5254c056-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ffd0eb46-1e6b-4bcc-95bc-eaca5254c056-istiod-ca-cert\") pod \"ffd0eb46-1e6b-4bcc-95bc-eaca5254c056\" (UID: \"ffd0eb46-1e6b-4bcc-95bc-eaca5254c056\") "
Oct 31 01:04:50.519269 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:50.513833    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ffd0eb46-1e6b-4bcc-95bc-eaca5254c056-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ffd0eb46-1e6b-4bcc-95bc-eaca5254c056-workload-socket\") pod \"ffd0eb46-1e6b-4bcc-95bc-eaca5254c056\" (UID: \"ffd0eb46-1e6b-4bcc-95bc-eaca5254c056\") "
Oct 31 01:04:50.519269 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:50.513890    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ffd0eb46-1e6b-4bcc-95bc-eaca5254c056-istio-token\" (UniqueName: \"kubernetes.io/projected/ffd0eb46-1e6b-4bcc-95bc-eaca5254c056-istio-token\") pod \"ffd0eb46-1e6b-4bcc-95bc-eaca5254c056\" (UID: \"ffd0eb46-1e6b-4bcc-95bc-eaca5254c056\") "
Oct 31 01:04:50.519269 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:50.513918    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ffd0eb46-1e6b-4bcc-95bc-eaca5254c056-kube-api-access-vcml2\" (UniqueName: \"kubernetes.io/projected/ffd0eb46-1e6b-4bcc-95bc-eaca5254c056-kube-api-access-vcml2\") pod \"ffd0eb46-1e6b-4bcc-95bc-eaca5254c056\" (UID: \"ffd0eb46-1e6b-4bcc-95bc-eaca5254c056\") "
Oct 31 01:04:50.519269 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:50.513975    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ffd0eb46-1e6b-4bcc-95bc-eaca5254c056-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ffd0eb46-1e6b-4bcc-95bc-eaca5254c056-credential-socket\") pod \"ffd0eb46-1e6b-4bcc-95bc-eaca5254c056\" (UID: \"ffd0eb46-1e6b-4bcc-95bc-eaca5254c056\") "
Oct 31 01:04:50.519269 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:50.514040    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/ffd0eb46-1e6b-4bcc-95bc-eaca5254c056-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ffd0eb46-1e6b-4bcc-95bc-eaca5254c056-istio-podinfo\") pod \"ffd0eb46-1e6b-4bcc-95bc-eaca5254c056\" (UID: \"ffd0eb46-1e6b-4bcc-95bc-eaca5254c056\") "
Oct 31 01:04:50.528976 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:50.525124    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ffd0eb46-1e6b-4bcc-95bc-eaca5254c056-workload-certs" pod "ffd0eb46-1e6b-4bcc-95bc-eaca5254c056" (UID: "ffd0eb46-1e6b-4bcc-95bc-eaca5254c056"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:04:50.533896 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:50.533851    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ffd0eb46-1e6b-4bcc-95bc-eaca5254c056-istio-data" pod "ffd0eb46-1e6b-4bcc-95bc-eaca5254c056" (UID: "ffd0eb46-1e6b-4bcc-95bc-eaca5254c056"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:04:50.534282 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:50.534256    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ffd0eb46-1e6b-4bcc-95bc-eaca5254c056-credential-socket" pod "ffd0eb46-1e6b-4bcc-95bc-eaca5254c056" (UID: "ffd0eb46-1e6b-4bcc-95bc-eaca5254c056"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:04:50.534422 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:50.534336    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ffd0eb46-1e6b-4bcc-95bc-eaca5254c056-workload-socket" pod "ffd0eb46-1e6b-4bcc-95bc-eaca5254c056" (UID: "ffd0eb46-1e6b-4bcc-95bc-eaca5254c056"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:04:50.535355 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:50.535329    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ffd0eb46-1e6b-4bcc-95bc-eaca5254c056-istiod-ca-cert" pod "ffd0eb46-1e6b-4bcc-95bc-eaca5254c056" (UID: "ffd0eb46-1e6b-4bcc-95bc-eaca5254c056"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 01:04:50.545156 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:50.545089    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ffd0eb46-1e6b-4bcc-95bc-eaca5254c056-istio-podinfo" pod "ffd0eb46-1e6b-4bcc-95bc-eaca5254c056" (UID: "ffd0eb46-1e6b-4bcc-95bc-eaca5254c056"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 01:04:50.545591 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:50.545540    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ffd0eb46-1e6b-4bcc-95bc-eaca5254c056-kube-api-access-vcml2" pod "ffd0eb46-1e6b-4bcc-95bc-eaca5254c056" (UID: "ffd0eb46-1e6b-4bcc-95bc-eaca5254c056"). InnerVolumeSpecName "kube-api-access-vcml2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:04:50.548356 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:50.547049    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ffd0eb46-1e6b-4bcc-95bc-eaca5254c056-istio-token" pod "ffd0eb46-1e6b-4bcc-95bc-eaca5254c056" (UID: "ffd0eb46-1e6b-4bcc-95bc-eaca5254c056"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:04:50.548356 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:50.548183    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ffd0eb46-1e6b-4bcc-95bc-eaca5254c056-istio-envoy" pod "ffd0eb46-1e6b-4bcc-95bc-eaca5254c056" (UID: "ffd0eb46-1e6b-4bcc-95bc-eaca5254c056"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:04:50.585289 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:50.585263    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-eee7f1-d9bf6db74-ggvqp"
Oct 31 01:04:50.615552 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:50.615515    4284 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ffd0eb46-1e6b-4bcc-95bc-eaca5254c056-istio-envoy\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:04:50.615552 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:50.615541    4284 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ffd0eb46-1e6b-4bcc-95bc-eaca5254c056-istiod-ca-cert\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:04:50.615552 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:50.615553    4284 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ffd0eb46-1e6b-4bcc-95bc-eaca5254c056-workload-socket\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:04:50.615552 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:50.615561    4284 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ffd0eb46-1e6b-4bcc-95bc-eaca5254c056-istio-token\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:04:50.615914 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:50.615570    4284 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vcml2\" (UniqueName: \"kubernetes.io/projected/ffd0eb46-1e6b-4bcc-95bc-eaca5254c056-kube-api-access-vcml2\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:04:50.615914 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:50.615579    4284 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ffd0eb46-1e6b-4bcc-95bc-eaca5254c056-credential-socket\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:04:50.615914 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:50.615587    4284 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ffd0eb46-1e6b-4bcc-95bc-eaca5254c056-istio-podinfo\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:04:50.615914 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:50.615596    4284 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ffd0eb46-1e6b-4bcc-95bc-eaca5254c056-istio-data\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:04:50.615914 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:50.615603    4284 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ffd0eb46-1e6b-4bcc-95bc-eaca5254c056-workload-certs\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:04:50.716392 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:50.716311    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/d5b7cbb4-de93-4a9a-abd3-b69636f595dd-metrics-certs\" (UniqueName: \"kubernetes.io/secret/d5b7cbb4-de93-4a9a-abd3-b69636f595dd-metrics-certs\") pod \"d5b7cbb4-de93-4a9a-abd3-b69636f595dd\" (UID: \"d5b7cbb4-de93-4a9a-abd3-b69636f595dd\") "
Oct 31 01:04:50.716563 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:50.716398    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/d5b7cbb4-de93-4a9a-abd3-b69636f595dd-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d5b7cbb4-de93-4a9a-abd3-b69636f595dd-maas-api-tls\") pod \"d5b7cbb4-de93-4a9a-abd3-b69636f595dd\" (UID: \"d5b7cbb4-de93-4a9a-abd3-b69636f595dd\") "
Oct 31 01:04:50.716563 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:50.716437    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d5b7cbb4-de93-4a9a-abd3-b69636f595dd-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d5b7cbb4-de93-4a9a-abd3-b69636f595dd-openshift-service-ca\") pod \"d5b7cbb4-de93-4a9a-abd3-b69636f595dd\" (UID: \"d5b7cbb4-de93-4a9a-abd3-b69636f595dd\") "
Oct 31 01:04:50.716563 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:50.716486    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d5b7cbb4-de93-4a9a-abd3-b69636f595dd-kube-api-access-2hssj\" (UniqueName: \"kubernetes.io/projected/d5b7cbb4-de93-4a9a-abd3-b69636f595dd-kube-api-access-2hssj\") pod \"d5b7cbb4-de93-4a9a-abd3-b69636f595dd\" (UID: \"d5b7cbb4-de93-4a9a-abd3-b69636f595dd\") "
Oct 31 01:04:50.716912 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:50.716874    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d5b7cbb4-de93-4a9a-abd3-b69636f595dd-openshift-service-ca" pod "d5b7cbb4-de93-4a9a-abd3-b69636f595dd" (UID: "d5b7cbb4-de93-4a9a-abd3-b69636f595dd"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 01:04:50.717141 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:50.717091    4284 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d5b7cbb4-de93-4a9a-abd3-b69636f595dd-openshift-service-ca\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:04:50.719387 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:50.719362    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d5b7cbb4-de93-4a9a-abd3-b69636f595dd-metrics-certs" pod "d5b7cbb4-de93-4a9a-abd3-b69636f595dd" (UID: "d5b7cbb4-de93-4a9a-abd3-b69636f595dd"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 01:04:50.719473 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:50.719398    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d5b7cbb4-de93-4a9a-abd3-b69636f595dd-kube-api-access-2hssj" pod "d5b7cbb4-de93-4a9a-abd3-b69636f595dd" (UID: "d5b7cbb4-de93-4a9a-abd3-b69636f595dd"). InnerVolumeSpecName "kube-api-access-2hssj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:04:50.719473 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:50.719443    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d5b7cbb4-de93-4a9a-abd3-b69636f595dd-maas-api-tls" pod "d5b7cbb4-de93-4a9a-abd3-b69636f595dd" (UID: "d5b7cbb4-de93-4a9a-abd3-b69636f595dd"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 01:04:50.835259 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:50.833637    4284 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d5b7cbb4-de93-4a9a-abd3-b69636f595dd-metrics-certs\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:04:50.835259 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:50.833678    4284 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d5b7cbb4-de93-4a9a-abd3-b69636f595dd-maas-api-tls\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:04:50.835259 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:50.833697    4284 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2hssj\" (UniqueName: \"kubernetes.io/projected/d5b7cbb4-de93-4a9a-abd3-b69636f595dd-kube-api-access-2hssj\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:04:51.430788 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:51.430711    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-29ece5-openshift-default-6fcb86dbc-f75sz" event={"ID":"ffd0eb46-1e6b-4bcc-95bc-eaca5254c056","Type":"ContainerDied","Data":"ee06bb1149ffaf293ae579d6309b8ea9bf16d0398bb6fa43454f20f5a729389c"}
Oct 31 01:04:51.431250 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:51.430719    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-29ece5-openshift-default-6fcb86dbc-f75sz"
Oct 31 01:04:51.431250 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:51.430800    4284 scope.go:122] "RemoveContainer" containerID="6cf3ff5d8abe6d686ff1ea7ab91616d4b561140c6ea5cf6e905eb5c37ed8b79c"
Oct 31 01:04:51.432172 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:51.432159    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-eee7f1-d9bf6db74-ggvqp" event={"ID":"d5b7cbb4-de93-4a9a-abd3-b69636f595dd","Type":"ContainerDied","Data":"d431b8e5d2aa6c3abfd8181b7d98240bf4f993a48df600b0a3df0d8576c141bb"}
Oct 31 01:04:51.432250 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:51.432177    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-eee7f1-d9bf6db74-ggvqp"
Oct 31 01:04:51.442537 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:51.442519    4284 scope.go:122] "RemoveContainer" containerID="1ee37fad3f92b4642d7b48b8f35f45711b99249469ab146987ada980b9b06ab0"
Oct 31 01:04:51.591322 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:51.591283    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-eee7f1-d9bf6db74-ggvqp"]
Oct 31 01:04:51.638266 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:51.638233    4284 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-eee7f1-d9bf6db74-ggvqp"]
Oct 31 01:04:51.720323 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:51.720206    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-29ece5-openshift-default-6fcb86dbc-f75sz"]
Oct 31 01:04:51.727773 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:51.727745    4284 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w2-29ece5-openshift-default-6fcb86dbc-f75sz"]
Oct 31 01:04:51.949659 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:51.949632    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:04:51.963377 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:51.963349    4284 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d5b7cbb4-de93-4a9a-abd3-b69636f595dd" path="/var/lib/kubelet/pods/d5b7cbb4-de93-4a9a-abd3-b69636f595dd/volumes"
Oct 31 01:04:51.963882 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:51.963860    4284 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ffd0eb46-1e6b-4bcc-95bc-eaca5254c056" path="/var/lib/kubelet/pods/ffd0eb46-1e6b-4bcc-95bc-eaca5254c056/volumes"
Oct 31 01:04:52.169461 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:52.169426    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:04:52.336078 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:52.336044    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:04:52.914548 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:52.914510    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:04:54.413160 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:54.413126    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-4ab4aa-7d9ff55bd6-g7vx5"
Oct 31 01:04:57.298734 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:57.298695    4284 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-a-add5bf-5f65cc9f84-2cwpd"]
Oct 31 01:04:57.299273 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:57.299258    4284 cpu_manager.go:414] "containerMap: removing container" podUID="d3800ace-ae94-47da-ac65-218d3830b13d" containerName="istio-proxy"
Oct 31 01:04:57.299273 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:57.299273    4284 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d3800ace-ae94-47da-ac65-218d3830b13d" containerName="istio-proxy"
Oct 31 01:04:57.299418 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:57.299306    4284 cpu_manager.go:414] "containerMap: removing container" podUID="d5b7cbb4-de93-4a9a-abd3-b69636f595dd" containerName="maas-api"
Oct 31 01:04:57.299418 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:57.299311    4284 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d5b7cbb4-de93-4a9a-abd3-b69636f595dd" containerName="maas-api"
Oct 31 01:04:57.299418 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:57.299319    4284 cpu_manager.go:414] "containerMap: removing container" podUID="ffd0eb46-1e6b-4bcc-95bc-eaca5254c056" containerName="istio-proxy"
Oct 31 01:04:57.299418 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:57.299324    4284 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ffd0eb46-1e6b-4bcc-95bc-eaca5254c056" containerName="istio-proxy"
Oct 31 01:04:57.299418 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:57.299408    4284 memory_manager.go:365] "RemoveStaleState removing state" podUID="d3800ace-ae94-47da-ac65-218d3830b13d" containerName="istio-proxy"
Oct 31 01:04:57.299418 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:57.299416    4284 memory_manager.go:365] "RemoveStaleState removing state" podUID="ffd0eb46-1e6b-4bcc-95bc-eaca5254c056" containerName="istio-proxy"
Oct 31 01:04:57.299725 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:57.299424    4284 memory_manager.go:365] "RemoveStaleState removing state" podUID="d5b7cbb4-de93-4a9a-abd3-b69636f595dd" containerName="maas-api"
Oct 31 01:04:57.304526 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:57.304507    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-add5bf-5f65cc9f84-2cwpd"
Oct 31 01:04:57.355745 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:57.355717    4284 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-praxis-a-add5bf-serving-cert\""
Oct 31 01:04:57.405468 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:57.405439    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-a-add5bf-5f65cc9f84-2cwpd"]
Oct 31 01:04:57.413057 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:57.413035    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2gq5c\" (UniqueName: \"kubernetes.io/projected/1761233b-1619-498b-a762-edbbcb0e967a-kube-api-access-2gq5c\") pod \"maas-api-e2e-praxis-a-add5bf-5f65cc9f84-2cwpd\" (UID: \"1761233b-1619-498b-a762-edbbcb0e967a\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-add5bf-5f65cc9f84-2cwpd"
Oct 31 01:04:57.413187 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:57.413075    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1761233b-1619-498b-a762-edbbcb0e967a-maas-api-tls\") pod \"maas-api-e2e-praxis-a-add5bf-5f65cc9f84-2cwpd\" (UID: \"1761233b-1619-498b-a762-edbbcb0e967a\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-add5bf-5f65cc9f84-2cwpd"
Oct 31 01:04:57.413255 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:57.413229    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1761233b-1619-498b-a762-edbbcb0e967a-metrics-certs\") pod \"maas-api-e2e-praxis-a-add5bf-5f65cc9f84-2cwpd\" (UID: \"1761233b-1619-498b-a762-edbbcb0e967a\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-add5bf-5f65cc9f84-2cwpd"
Oct 31 01:04:57.413382 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:57.413363    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1761233b-1619-498b-a762-edbbcb0e967a-openshift-service-ca\") pod \"maas-api-e2e-praxis-a-add5bf-5f65cc9f84-2cwpd\" (UID: \"1761233b-1619-498b-a762-edbbcb0e967a\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-add5bf-5f65cc9f84-2cwpd"
Oct 31 01:04:57.514069 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:57.514029    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1761233b-1619-498b-a762-edbbcb0e967a-metrics-certs\") pod \"maas-api-e2e-praxis-a-add5bf-5f65cc9f84-2cwpd\" (UID: \"1761233b-1619-498b-a762-edbbcb0e967a\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-add5bf-5f65cc9f84-2cwpd"
Oct 31 01:04:57.514302 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:57.514138    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1761233b-1619-498b-a762-edbbcb0e967a-openshift-service-ca\") pod \"maas-api-e2e-praxis-a-add5bf-5f65cc9f84-2cwpd\" (UID: \"1761233b-1619-498b-a762-edbbcb0e967a\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-add5bf-5f65cc9f84-2cwpd"
Oct 31 01:04:57.514302 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:57.514185    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2gq5c\" (UniqueName: \"kubernetes.io/projected/1761233b-1619-498b-a762-edbbcb0e967a-kube-api-access-2gq5c\") pod \"maas-api-e2e-praxis-a-add5bf-5f65cc9f84-2cwpd\" (UID: \"1761233b-1619-498b-a762-edbbcb0e967a\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-add5bf-5f65cc9f84-2cwpd"
Oct 31 01:04:57.514302 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:57.514250    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1761233b-1619-498b-a762-edbbcb0e967a-maas-api-tls\") pod \"maas-api-e2e-praxis-a-add5bf-5f65cc9f84-2cwpd\" (UID: \"1761233b-1619-498b-a762-edbbcb0e967a\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-add5bf-5f65cc9f84-2cwpd"
Oct 31 01:04:57.514513 ip-10-0-132-173 kubenswrapper[4284]: E1031 01:04:57.514455    4284 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-praxis-a-add5bf-serving-cert: secret "maas-api-e2e-praxis-a-add5bf-serving-cert" not found
Oct 31 01:04:57.514619 ip-10-0-132-173 kubenswrapper[4284]: E1031 01:04:57.514522    4284 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1761233b-1619-498b-a762-edbbcb0e967a-maas-api-tls podName:1761233b-1619-498b-a762-edbbcb0e967a nodeName:}" failed. No retries permitted until 2026-10-31 01:04:58.014507411 +0000 UTC m=+2212.159514304 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/1761233b-1619-498b-a762-edbbcb0e967a-maas-api-tls") pod "maas-api-e2e-praxis-a-add5bf-5f65cc9f84-2cwpd" (UID: "1761233b-1619-498b-a762-edbbcb0e967a") : secret "maas-api-e2e-praxis-a-add5bf-serving-cert" not found
Oct 31 01:04:57.514714 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:57.514700    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:04:57.515273 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:57.515240    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1761233b-1619-498b-a762-edbbcb0e967a-openshift-service-ca\") pod \"maas-api-e2e-praxis-a-add5bf-5f65cc9f84-2cwpd\" (UID: \"1761233b-1619-498b-a762-edbbcb0e967a\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-add5bf-5f65cc9f84-2cwpd"
Oct 31 01:04:57.516754 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:57.516736    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1761233b-1619-498b-a762-edbbcb0e967a-metrics-certs\") pod \"maas-api-e2e-praxis-a-add5bf-5f65cc9f84-2cwpd\" (UID: \"1761233b-1619-498b-a762-edbbcb0e967a\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-add5bf-5f65cc9f84-2cwpd"
Oct 31 01:04:57.602185 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:57.600955    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2gq5c\" (UniqueName: \"kubernetes.io/projected/1761233b-1619-498b-a762-edbbcb0e967a-kube-api-access-2gq5c\") pod \"maas-api-e2e-praxis-a-add5bf-5f65cc9f84-2cwpd\" (UID: \"1761233b-1619-498b-a762-edbbcb0e967a\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-add5bf-5f65cc9f84-2cwpd"
Oct 31 01:04:57.683203 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:57.683169    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:04:57.960835 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:57.960800    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:04:58.021162 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:58.021130    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1761233b-1619-498b-a762-edbbcb0e967a-maas-api-tls\") pod \"maas-api-e2e-praxis-a-add5bf-5f65cc9f84-2cwpd\" (UID: \"1761233b-1619-498b-a762-edbbcb0e967a\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-add5bf-5f65cc9f84-2cwpd"
Oct 31 01:04:58.023936 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:58.023908    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1761233b-1619-498b-a762-edbbcb0e967a-maas-api-tls\") pod \"maas-api-e2e-praxis-a-add5bf-5f65cc9f84-2cwpd\" (UID: \"1761233b-1619-498b-a762-edbbcb0e967a\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-add5bf-5f65cc9f84-2cwpd"
Oct 31 01:04:58.218429 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:58.218348    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-add5bf-5f65cc9f84-2cwpd"
Oct 31 01:04:58.596802 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:58.596772    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-a-add5bf-5f65cc9f84-2cwpd"]
Oct 31 01:04:58.598019 ip-10-0-132-173 kubenswrapper[4284]: W1031 01:04:58.597988    4284 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1761233b_1619_498b_a762_edbbcb0e967a.slice/crio-60339042f50779e0c5ca81573a9b30a39bc01a56af23f658439f53fa0b167be3 WatchSource:0}: Error finding container 60339042f50779e0c5ca81573a9b30a39bc01a56af23f658439f53fa0b167be3: Status 404 returned error can't find the container with id 60339042f50779e0c5ca81573a9b30a39bc01a56af23f658439f53fa0b167be3
Oct 31 01:04:59.493177 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:04:59.493145    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-add5bf-5f65cc9f84-2cwpd" event={"ID":"1761233b-1619-498b-a762-edbbcb0e967a","Type":"ContainerStarted","Data":"60339042f50779e0c5ca81573a9b30a39bc01a56af23f658439f53fa0b167be3"}
Oct 31 01:05:00.075720 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:05:00.075680    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:05:00.252486 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:05:00.252449    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:05:00.459809 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:05:00.459777    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-eee7f1-openshift-default-78d7b57c9-cn4g6"]
Oct 31 01:05:00.460031 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:05:00.460007    4284 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-eee7f1-openshift-default-78d7b57c9-cn4g6" podUID="4fe7b9bb-dcce-4bb5-b37a-82fe1639b77e" containerName="istio-proxy" containerID="cri-o://0c5d8907e60cf1cf12a88d576187cbcaf5d85130c988488329a87151c5a76960" gracePeriod=30
Oct 31 01:05:00.468527 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:05:00.468499    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:05:00.500392 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:05:00.500364    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-add5bf-5f65cc9f84-2cwpd" event={"ID":"1761233b-1619-498b-a762-edbbcb0e967a","Type":"ContainerStarted","Data":"160379e20885f721958b10bf6e5e06891a75930b483698bca4b1717354333bb5"}
Oct 31 01:05:00.500538 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:05:00.500447    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-add5bf-5f65cc9f84-2cwpd"
Oct 31 01:05:00.723077 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:05:00.722983    4284 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-add5bf-5f65cc9f84-2cwpd" podStartSLOduration=2.71559887 podStartE2EDuration="3.722961126s" totalImagesPullingTime="1.007362256s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 01:04:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 01:05:00.72242934 +0000 UTC m=+2214.867436282" watchObservedRunningTime="2026-10-31 01:05:00.722961126 +0000 UTC m=+2214.867968039"
Oct 31 01:05:01.344398 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:05:01.344361    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:05:03.229002 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:05:03.228960    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:05:05.742058 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:05:05.741647    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-eee7f1-openshift-default-78d7b57c9-cn4g6"
Oct 31 01:05:05.830660 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:05:05.830625    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4fe7b9bb-dcce-4bb5-b37a-82fe1639b77e-istio-token\" (UniqueName: \"kubernetes.io/projected/4fe7b9bb-dcce-4bb5-b37a-82fe1639b77e-istio-token\") pod \"4fe7b9bb-dcce-4bb5-b37a-82fe1639b77e\" (UID: \"4fe7b9bb-dcce-4bb5-b37a-82fe1639b77e\") "
Oct 31 01:05:05.830860 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:05:05.830695    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4fe7b9bb-dcce-4bb5-b37a-82fe1639b77e-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4fe7b9bb-dcce-4bb5-b37a-82fe1639b77e-workload-certs\") pod \"4fe7b9bb-dcce-4bb5-b37a-82fe1639b77e\" (UID: \"4fe7b9bb-dcce-4bb5-b37a-82fe1639b77e\") "
Oct 31 01:05:05.830860 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:05:05.830719    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4fe7b9bb-dcce-4bb5-b37a-82fe1639b77e-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4fe7b9bb-dcce-4bb5-b37a-82fe1639b77e-istio-data\") pod \"4fe7b9bb-dcce-4bb5-b37a-82fe1639b77e\" (UID: \"4fe7b9bb-dcce-4bb5-b37a-82fe1639b77e\") "
Oct 31 01:05:05.830860 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:05:05.830767    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/4fe7b9bb-dcce-4bb5-b37a-82fe1639b77e-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4fe7b9bb-dcce-4bb5-b37a-82fe1639b77e-istio-podinfo\") pod \"4fe7b9bb-dcce-4bb5-b37a-82fe1639b77e\" (UID: \"4fe7b9bb-dcce-4bb5-b37a-82fe1639b77e\") "
Oct 31 01:05:05.830860 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:05:05.830807    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/4fe7b9bb-dcce-4bb5-b37a-82fe1639b77e-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4fe7b9bb-dcce-4bb5-b37a-82fe1639b77e-istiod-ca-cert\") pod \"4fe7b9bb-dcce-4bb5-b37a-82fe1639b77e\" (UID: \"4fe7b9bb-dcce-4bb5-b37a-82fe1639b77e\") "
Oct 31 01:05:05.831156 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:05:05.830849    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4fe7b9bb-dcce-4bb5-b37a-82fe1639b77e-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4fe7b9bb-dcce-4bb5-b37a-82fe1639b77e-workload-socket\") pod \"4fe7b9bb-dcce-4bb5-b37a-82fe1639b77e\" (UID: \"4fe7b9bb-dcce-4bb5-b37a-82fe1639b77e\") "
Oct 31 01:05:05.831156 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:05:05.830897    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4fe7b9bb-dcce-4bb5-b37a-82fe1639b77e-kube-api-access-8scvs\" (UniqueName: \"kubernetes.io/projected/4fe7b9bb-dcce-4bb5-b37a-82fe1639b77e-kube-api-access-8scvs\") pod \"4fe7b9bb-dcce-4bb5-b37a-82fe1639b77e\" (UID: \"4fe7b9bb-dcce-4bb5-b37a-82fe1639b77e\") "
Oct 31 01:05:05.831156 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:05:05.831005    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4fe7b9bb-dcce-4bb5-b37a-82fe1639b77e-istio-data" pod "4fe7b9bb-dcce-4bb5-b37a-82fe1639b77e" (UID: "4fe7b9bb-dcce-4bb5-b37a-82fe1639b77e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:05:05.831156 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:05:05.831021    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4fe7b9bb-dcce-4bb5-b37a-82fe1639b77e-workload-certs" pod "4fe7b9bb-dcce-4bb5-b37a-82fe1639b77e" (UID: "4fe7b9bb-dcce-4bb5-b37a-82fe1639b77e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:05:05.831156 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:05:05.831098    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4fe7b9bb-dcce-4bb5-b37a-82fe1639b77e-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4fe7b9bb-dcce-4bb5-b37a-82fe1639b77e-credential-socket\") pod \"4fe7b9bb-dcce-4bb5-b37a-82fe1639b77e\" (UID: \"4fe7b9bb-dcce-4bb5-b37a-82fe1639b77e\") "
Oct 31 01:05:05.831156 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:05:05.831130    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4fe7b9bb-dcce-4bb5-b37a-82fe1639b77e-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4fe7b9bb-dcce-4bb5-b37a-82fe1639b77e-istio-envoy\") pod \"4fe7b9bb-dcce-4bb5-b37a-82fe1639b77e\" (UID: \"4fe7b9bb-dcce-4bb5-b37a-82fe1639b77e\") "
Oct 31 01:05:05.831514 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:05:05.831153    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4fe7b9bb-dcce-4bb5-b37a-82fe1639b77e-workload-socket" pod "4fe7b9bb-dcce-4bb5-b37a-82fe1639b77e" (UID: "4fe7b9bb-dcce-4bb5-b37a-82fe1639b77e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:05:05.831514 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:05:05.831295    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4fe7b9bb-dcce-4bb5-b37a-82fe1639b77e-istiod-ca-cert" pod "4fe7b9bb-dcce-4bb5-b37a-82fe1639b77e" (UID: "4fe7b9bb-dcce-4bb5-b37a-82fe1639b77e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 01:05:05.831514 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:05:05.831486    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4fe7b9bb-dcce-4bb5-b37a-82fe1639b77e-credential-socket" pod "4fe7b9bb-dcce-4bb5-b37a-82fe1639b77e" (UID: "4fe7b9bb-dcce-4bb5-b37a-82fe1639b77e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:05:05.831670 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:05:05.831655    4284 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4fe7b9bb-dcce-4bb5-b37a-82fe1639b77e-credential-socket\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:05:05.831714 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:05:05.831673    4284 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4fe7b9bb-dcce-4bb5-b37a-82fe1639b77e-workload-certs\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:05:05.831714 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:05:05.831682    4284 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4fe7b9bb-dcce-4bb5-b37a-82fe1639b77e-istio-data\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:05:05.831714 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:05:05.831692    4284 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4fe7b9bb-dcce-4bb5-b37a-82fe1639b77e-istiod-ca-cert\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:05:05.831714 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:05:05.831702    4284 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4fe7b9bb-dcce-4bb5-b37a-82fe1639b77e-workload-socket\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:05:05.833206 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:05:05.833181    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/4fe7b9bb-dcce-4bb5-b37a-82fe1639b77e-istio-podinfo" pod "4fe7b9bb-dcce-4bb5-b37a-82fe1639b77e" (UID: "4fe7b9bb-dcce-4bb5-b37a-82fe1639b77e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 01:05:05.833552 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:05:05.833533    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4fe7b9bb-dcce-4bb5-b37a-82fe1639b77e-istio-envoy" pod "4fe7b9bb-dcce-4bb5-b37a-82fe1639b77e" (UID: "4fe7b9bb-dcce-4bb5-b37a-82fe1639b77e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:05:05.833887 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:05:05.833864    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4fe7b9bb-dcce-4bb5-b37a-82fe1639b77e-istio-token" pod "4fe7b9bb-dcce-4bb5-b37a-82fe1639b77e" (UID: "4fe7b9bb-dcce-4bb5-b37a-82fe1639b77e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:05:05.833970 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:05:05.833901    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4fe7b9bb-dcce-4bb5-b37a-82fe1639b77e-kube-api-access-8scvs" pod "4fe7b9bb-dcce-4bb5-b37a-82fe1639b77e" (UID: "4fe7b9bb-dcce-4bb5-b37a-82fe1639b77e"). InnerVolumeSpecName "kube-api-access-8scvs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:05:05.933100 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:05:05.933066    4284 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4fe7b9bb-dcce-4bb5-b37a-82fe1639b77e-istio-envoy\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:05:05.933100 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:05:05.933100    4284 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4fe7b9bb-dcce-4bb5-b37a-82fe1639b77e-istio-token\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:05:05.933330 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:05:05.933109    4284 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4fe7b9bb-dcce-4bb5-b37a-82fe1639b77e-istio-podinfo\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:05:05.933330 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:05:05.933118    4284 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8scvs\" (UniqueName: \"kubernetes.io/projected/4fe7b9bb-dcce-4bb5-b37a-82fe1639b77e-kube-api-access-8scvs\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:05:06.246751 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:05:06.246707    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:05:06.405239 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:05:06.404200    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:05:06.518443 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:05:06.518364    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-add5bf-5f65cc9f84-2cwpd"
Oct 31 01:05:06.541413 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:05:06.541381    4284 generic.go:356] "Generic (PLEG): container finished" podID="4fe7b9bb-dcce-4bb5-b37a-82fe1639b77e" containerID="0c5d8907e60cf1cf12a88d576187cbcaf5d85130c988488329a87151c5a76960" exitCode=0
Oct 31 01:05:06.541584 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:05:06.541501    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-eee7f1-openshift-default-78d7b57c9-cn4g6" event={"ID":"4fe7b9bb-dcce-4bb5-b37a-82fe1639b77e","Type":"ContainerDied","Data":"0c5d8907e60cf1cf12a88d576187cbcaf5d85130c988488329a87151c5a76960"}
Oct 31 01:05:06.541584 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:05:06.541542    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-eee7f1-openshift-default-78d7b57c9-cn4g6" event={"ID":"4fe7b9bb-dcce-4bb5-b37a-82fe1639b77e","Type":"ContainerDied","Data":"f25a3cada30c511faae376abea6bc2307f0d6fba63f615e9e53e544f7c736bff"}
Oct 31 01:05:06.541584 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:05:06.541562    4284 scope.go:122] "RemoveContainer" containerID="0c5d8907e60cf1cf12a88d576187cbcaf5d85130c988488329a87151c5a76960"
Oct 31 01:05:06.541721 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:05:06.541710    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-eee7f1-openshift-default-78d7b57c9-cn4g6"
Oct 31 01:05:06.553799 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:05:06.553773    4284 scope.go:122] "RemoveContainer" containerID="dbae75592cdb87487f1720b192cbb998410a7c09334712b8ca46c5e262be5ebe"
Oct 31 01:05:06.563342 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:05:06.563314    4284 scope.go:122] "RemoveContainer" containerID="0c5d8907e60cf1cf12a88d576187cbcaf5d85130c988488329a87151c5a76960"
Oct 31 01:05:06.563698 ip-10-0-132-173 kubenswrapper[4284]: E1031 01:05:06.563677    4284 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0c5d8907e60cf1cf12a88d576187cbcaf5d85130c988488329a87151c5a76960\": container with ID starting with 0c5d8907e60cf1cf12a88d576187cbcaf5d85130c988488329a87151c5a76960 not found: ID does not exist" containerID="0c5d8907e60cf1cf12a88d576187cbcaf5d85130c988488329a87151c5a76960"
Oct 31 01:05:06.563780 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:05:06.563707    4284 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0c5d8907e60cf1cf12a88d576187cbcaf5d85130c988488329a87151c5a76960"} err="failed to get container status \"0c5d8907e60cf1cf12a88d576187cbcaf5d85130c988488329a87151c5a76960\": rpc error: code = NotFound desc = could not find container \"0c5d8907e60cf1cf12a88d576187cbcaf5d85130c988488329a87151c5a76960\": container with ID starting with 0c5d8907e60cf1cf12a88d576187cbcaf5d85130c988488329a87151c5a76960 not found: ID does not exist"
Oct 31 01:05:06.563780 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:05:06.563727    4284 scope.go:122] "RemoveContainer" containerID="dbae75592cdb87487f1720b192cbb998410a7c09334712b8ca46c5e262be5ebe"
Oct 31 01:05:06.564027 ip-10-0-132-173 kubenswrapper[4284]: E1031 01:05:06.564011    4284 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dbae75592cdb87487f1720b192cbb998410a7c09334712b8ca46c5e262be5ebe\": container with ID starting with dbae75592cdb87487f1720b192cbb998410a7c09334712b8ca46c5e262be5ebe not found: ID does not exist" containerID="dbae75592cdb87487f1720b192cbb998410a7c09334712b8ca46c5e262be5ebe"
Oct 31 01:05:06.564070 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:05:06.564038    4284 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dbae75592cdb87487f1720b192cbb998410a7c09334712b8ca46c5e262be5ebe"} err="failed to get container status \"dbae75592cdb87487f1720b192cbb998410a7c09334712b8ca46c5e262be5ebe\": rpc error: code = NotFound desc = could not find container \"dbae75592cdb87487f1720b192cbb998410a7c09334712b8ca46c5e262be5ebe\": container with ID starting with dbae75592cdb87487f1720b192cbb998410a7c09334712b8ca46c5e262be5ebe not found: ID does not exist"
Oct 31 01:05:06.684193 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:05:06.684163    4284 scope.go:122] "RemoveContainer" containerID="7285956f5dd8d2eb76fe948a272aea4828a3b3ba3268cdeaba9e4d3c418eee90"
Oct 31 01:05:06.813239 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:05:06.803282    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-eee7f1-openshift-default-78d7b57c9-cn4g6"]
Oct 31 01:05:06.813239 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:05:06.811748    4284 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-eee7f1-openshift-default-78d7b57c9-cn4g6"]
Oct 31 01:05:07.965386 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:05:07.965352    4284 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="4fe7b9bb-dcce-4bb5-b37a-82fe1639b77e" path="/var/lib/kubelet/pods/4fe7b9bb-dcce-4bb5-b37a-82fe1639b77e/volumes"
Oct 31 01:05:08.605238 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:05:08.602775    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:05:13.282601 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:05:13.282561    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:05:13.484574 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:05:13.484523    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:05:13.801863 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:05:13.801826    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:05:15.599296 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:05:15.599261    4284 generic.go:356] "Generic (PLEG): container finished" podID="832a208d-8b57-444a-9ee2-fe30d01e1525" containerID="5aac0cb912889747d7872fb30066bf839447f7474f3ed90c219f6b2730e0abe1" exitCode=0
Oct 31 01:05:15.599842 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:05:15.599339    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-b-4ab4aa-openshift-default-799d6fd5-pqg68" event={"ID":"832a208d-8b57-444a-9ee2-fe30d01e1525","Type":"ContainerDied","Data":"5aac0cb912889747d7872fb30066bf839447f7474f3ed90c219f6b2730e0abe1"}
Oct 31 01:05:15.599908 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:05:15.599847    4284 scope.go:122] "RemoveContainer" containerID="5aac0cb912889747d7872fb30066bf839447f7474f3ed90c219f6b2730e0abe1"
Oct 31 01:05:15.601115 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:05:15.601090    4284 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 01:05:15.601181 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:05:15.601155    4284 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 01:05:15.601252 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:05:15.601188    4284 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 01:05:16.614744 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:05:16.614710    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-b-4ab4aa-openshift-default-799d6fd5-pqg68" event={"ID":"832a208d-8b57-444a-9ee2-fe30d01e1525","Type":"ContainerStarted","Data":"ae92d5f931e923073a29138a26dd8255d4b4e43aff31c99e60bcf2a52f8ef39e"}
Oct 31 01:05:16.969336 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:05:16.969294    4284 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-praxis-b-4ab4aa-openshift-default-799d6fd5-pqg68"
Oct 31 01:05:16.970579 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:05:16.970550    4284 patch_prober.go:29] interesting pod/e2e-praxis-b-4ab4aa-openshift-default-799d6fd5-pqg68 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.136:15021/healthz/ready\": dial tcp 10.134.0.136:15021: connect: connection refused" start-of-body=
Oct 31 01:05:16.970695 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:05:16.970613    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-b-4ab4aa-openshift-default-799d6fd5-pqg68" podUID="832a208d-8b57-444a-9ee2-fe30d01e1525" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.136:15021/healthz/ready\": dial tcp 10.134.0.136:15021: connect: connection refused"
Oct 31 01:05:17.037012 ip-10-0-132-173 kubenswrapper[4284]: E1031 01:05:17.036971    4284 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fd8sp" containerName="extractor"
Oct 31 01:05:17.969454 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:05:17.969425    4284 patch_prober.go:29] interesting pod/e2e-praxis-b-4ab4aa-openshift-default-799d6fd5-pqg68 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.136:15021/healthz/ready\": dial tcp 10.134.0.136:15021: connect: connection refused" start-of-body=
Oct 31 01:05:17.969961 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:05:17.969495    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-b-4ab4aa-openshift-default-799d6fd5-pqg68" podUID="832a208d-8b57-444a-9ee2-fe30d01e1525" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.136:15021/healthz/ready\": dial tcp 10.134.0.136:15021: connect: connection refused"
Oct 31 01:05:18.929876 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:05:18.929838    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:05:18.971238 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:05:18.970683    4284 patch_prober.go:29] interesting pod/e2e-praxis-b-4ab4aa-openshift-default-799d6fd5-pqg68 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.136:15021/healthz/ready\": dial tcp 10.134.0.136:15021: connect: connection refused" start-of-body=
Oct 31 01:05:18.971238 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:05:18.970750    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-b-4ab4aa-openshift-default-799d6fd5-pqg68" podUID="832a208d-8b57-444a-9ee2-fe30d01e1525" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.136:15021/healthz/ready\": dial tcp 10.134.0.136:15021: connect: connection refused"
Oct 31 01:05:19.912016 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:05:19.911985    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-praxis-b-4ab4aa-openshift-default-799d6fd5-pqg68"
Oct 31 01:05:19.969506 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:05:19.969475    4284 patch_prober.go:29] interesting pod/e2e-praxis-b-4ab4aa-openshift-default-799d6fd5-pqg68 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.136:15021/healthz/ready\": dial tcp 10.134.0.136:15021: connect: connection refused" start-of-body=
Oct 31 01:05:19.969672 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:05:19.969540    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-b-4ab4aa-openshift-default-799d6fd5-pqg68" podUID="832a208d-8b57-444a-9ee2-fe30d01e1525" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.136:15021/healthz/ready\": dial tcp 10.134.0.136:15021: connect: connection refused"
Oct 31 01:05:20.969263 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:05:20.969201    4284 patch_prober.go:29] interesting pod/e2e-praxis-b-4ab4aa-openshift-default-799d6fd5-pqg68 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.136:15021/healthz/ready\": dial tcp 10.134.0.136:15021: connect: connection refused" start-of-body=
Oct 31 01:05:20.969678 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:05:20.969285    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-b-4ab4aa-openshift-default-799d6fd5-pqg68" podUID="832a208d-8b57-444a-9ee2-fe30d01e1525" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.136:15021/healthz/ready\": dial tcp 10.134.0.136:15021: connect: connection refused"
Oct 31 01:05:21.973025 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:05:21.972993    4284 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-praxis-b-4ab4aa-openshift-default-799d6fd5-pqg68"
Oct 31 01:05:21.973998 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:05:21.973981    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-praxis-b-4ab4aa-openshift-default-799d6fd5-pqg68"
Oct 31 01:05:23.988341 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:05:23.988279    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:05:29.207546 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:05:29.207498    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:05:30.040683 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:05:30.040649    4284 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w1-b63c24-openshift-default-d8c64855f-snk9h"]
Oct 31 01:05:30.041235 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:05:30.041199    4284 cpu_manager.go:414] "containerMap: removing container" podUID="4fe7b9bb-dcce-4bb5-b37a-82fe1639b77e" containerName="istio-proxy"
Oct 31 01:05:30.041235 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:05:30.041229    4284 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4fe7b9bb-dcce-4bb5-b37a-82fe1639b77e" containerName="istio-proxy"
Oct 31 01:05:30.041343 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:05:30.041333    4284 memory_manager.go:365] "RemoveStaleState removing state" podUID="4fe7b9bb-dcce-4bb5-b37a-82fe1639b77e" containerName="istio-proxy"
Oct 31 01:05:30.041384 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:05:30.041346    4284 memory_manager.go:365] "RemoveStaleState removing state" podUID="4fe7b9bb-dcce-4bb5-b37a-82fe1639b77e" containerName="istio-proxy"
Oct 31 01:05:30.044644 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:05:30.044629    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-b63c24-openshift-default-d8c64855f-snk9h"
Oct 31 01:05:30.087227 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:05:30.087183    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8b30791d-d7f2-425d-ad30-078d7ff079ee-istio-token\") pod \"e2e-worker-w1-b63c24-openshift-default-d8c64855f-snk9h\" (UID: \"8b30791d-d7f2-425d-ad30-078d7ff079ee\") " pod="openshift-ingress/e2e-worker-w1-b63c24-openshift-default-d8c64855f-snk9h"
Oct 31 01:05:30.087389 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:05:30.087242    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8b30791d-d7f2-425d-ad30-078d7ff079ee-istio-data\") pod \"e2e-worker-w1-b63c24-openshift-default-d8c64855f-snk9h\" (UID: \"8b30791d-d7f2-425d-ad30-078d7ff079ee\") " pod="openshift-ingress/e2e-worker-w1-b63c24-openshift-default-d8c64855f-snk9h"
Oct 31 01:05:30.087389 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:05:30.087375    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8b30791d-d7f2-425d-ad30-078d7ff079ee-istiod-ca-cert\") pod \"e2e-worker-w1-b63c24-openshift-default-d8c64855f-snk9h\" (UID: \"8b30791d-d7f2-425d-ad30-078d7ff079ee\") " pod="openshift-ingress/e2e-worker-w1-b63c24-openshift-default-d8c64855f-snk9h"
Oct 31 01:05:30.087552 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:05:30.087435    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8b30791d-d7f2-425d-ad30-078d7ff079ee-credential-socket\") pod \"e2e-worker-w1-b63c24-openshift-default-d8c64855f-snk9h\" (UID: \"8b30791d-d7f2-425d-ad30-078d7ff079ee\") " pod="openshift-ingress/e2e-worker-w1-b63c24-openshift-default-d8c64855f-snk9h"
Oct 31 01:05:30.087552 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:05:30.087522    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9kh62\" (UniqueName: \"kubernetes.io/projected/8b30791d-d7f2-425d-ad30-078d7ff079ee-kube-api-access-9kh62\") pod \"e2e-worker-w1-b63c24-openshift-default-d8c64855f-snk9h\" (UID: \"8b30791d-d7f2-425d-ad30-078d7ff079ee\") " pod="openshift-ingress/e2e-worker-w1-b63c24-openshift-default-d8c64855f-snk9h"
Oct 31 01:05:30.087662 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:05:30.087574    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8b30791d-d7f2-425d-ad30-078d7ff079ee-workload-socket\") pod \"e2e-worker-w1-b63c24-openshift-default-d8c64855f-snk9h\" (UID: \"8b30791d-d7f2-425d-ad30-078d7ff079ee\") " pod="openshift-ingress/e2e-worker-w1-b63c24-openshift-default-d8c64855f-snk9h"
Oct 31 01:05:30.087662 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:05:30.087631    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8b30791d-d7f2-425d-ad30-078d7ff079ee-istio-envoy\") pod \"e2e-worker-w1-b63c24-openshift-default-d8c64855f-snk9h\" (UID: \"8b30791d-d7f2-425d-ad30-078d7ff079ee\") " pod="openshift-ingress/e2e-worker-w1-b63c24-openshift-default-d8c64855f-snk9h"
Oct 31 01:05:30.087770 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:05:30.087678    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8b30791d-d7f2-425d-ad30-078d7ff079ee-workload-certs\") pod \"e2e-worker-w1-b63c24-openshift-default-d8c64855f-snk9h\" (UID: \"8b30791d-d7f2-425d-ad30-078d7ff079ee\") " pod="openshift-ingress/e2e-worker-w1-b63c24-openshift-default-d8c64855f-snk9h"
Oct 31 01:05:30.087770 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:05:30.087740    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8b30791d-d7f2-425d-ad30-078d7ff079ee-istio-podinfo\") pod \"e2e-worker-w1-b63c24-openshift-default-d8c64855f-snk9h\" (UID: \"8b30791d-d7f2-425d-ad30-078d7ff079ee\") " pod="openshift-ingress/e2e-worker-w1-b63c24-openshift-default-d8c64855f-snk9h"
Oct 31 01:05:30.149016 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:05:30.148982    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-b63c24-openshift-default-d8c64855f-snk9h"]
Oct 31 01:05:30.188884 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:05:30.188847    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8b30791d-d7f2-425d-ad30-078d7ff079ee-istiod-ca-cert\") pod \"e2e-worker-w1-b63c24-openshift-default-d8c64855f-snk9h\" (UID: \"8b30791d-d7f2-425d-ad30-078d7ff079ee\") " pod="openshift-ingress/e2e-worker-w1-b63c24-openshift-default-d8c64855f-snk9h"
Oct 31 01:05:30.189093 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:05:30.188904    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8b30791d-d7f2-425d-ad30-078d7ff079ee-credential-socket\") pod \"e2e-worker-w1-b63c24-openshift-default-d8c64855f-snk9h\" (UID: \"8b30791d-d7f2-425d-ad30-078d7ff079ee\") " pod="openshift-ingress/e2e-worker-w1-b63c24-openshift-default-d8c64855f-snk9h"
Oct 31 01:05:30.189093 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:05:30.188942    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9kh62\" (UniqueName: \"kubernetes.io/projected/8b30791d-d7f2-425d-ad30-078d7ff079ee-kube-api-access-9kh62\") pod \"e2e-worker-w1-b63c24-openshift-default-d8c64855f-snk9h\" (UID: \"8b30791d-d7f2-425d-ad30-078d7ff079ee\") " pod="openshift-ingress/e2e-worker-w1-b63c24-openshift-default-d8c64855f-snk9h"
Oct 31 01:05:30.189093 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:05:30.188981    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8b30791d-d7f2-425d-ad30-078d7ff079ee-workload-socket\") pod \"e2e-worker-w1-b63c24-openshift-default-d8c64855f-snk9h\" (UID: \"8b30791d-d7f2-425d-ad30-078d7ff079ee\") " pod="openshift-ingress/e2e-worker-w1-b63c24-openshift-default-d8c64855f-snk9h"
Oct 31 01:05:30.189093 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:05:30.189032    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8b30791d-d7f2-425d-ad30-078d7ff079ee-istio-envoy\") pod \"e2e-worker-w1-b63c24-openshift-default-d8c64855f-snk9h\" (UID: \"8b30791d-d7f2-425d-ad30-078d7ff079ee\") " pod="openshift-ingress/e2e-worker-w1-b63c24-openshift-default-d8c64855f-snk9h"
Oct 31 01:05:30.189093 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:05:30.189060    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8b30791d-d7f2-425d-ad30-078d7ff079ee-workload-certs\") pod \"e2e-worker-w1-b63c24-openshift-default-d8c64855f-snk9h\" (UID: \"8b30791d-d7f2-425d-ad30-078d7ff079ee\") " pod="openshift-ingress/e2e-worker-w1-b63c24-openshift-default-d8c64855f-snk9h"
Oct 31 01:05:30.189426 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:05:30.189103    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8b30791d-d7f2-425d-ad30-078d7ff079ee-istio-podinfo\") pod \"e2e-worker-w1-b63c24-openshift-default-d8c64855f-snk9h\" (UID: \"8b30791d-d7f2-425d-ad30-078d7ff079ee\") " pod="openshift-ingress/e2e-worker-w1-b63c24-openshift-default-d8c64855f-snk9h"
Oct 31 01:05:30.189426 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:05:30.189280    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8b30791d-d7f2-425d-ad30-078d7ff079ee-istio-token\") pod \"e2e-worker-w1-b63c24-openshift-default-d8c64855f-snk9h\" (UID: \"8b30791d-d7f2-425d-ad30-078d7ff079ee\") " pod="openshift-ingress/e2e-worker-w1-b63c24-openshift-default-d8c64855f-snk9h"
Oct 31 01:05:30.189426 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:05:30.189311    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8b30791d-d7f2-425d-ad30-078d7ff079ee-istio-data\") pod \"e2e-worker-w1-b63c24-openshift-default-d8c64855f-snk9h\" (UID: \"8b30791d-d7f2-425d-ad30-078d7ff079ee\") " pod="openshift-ingress/e2e-worker-w1-b63c24-openshift-default-d8c64855f-snk9h"
Oct 31 01:05:30.189426 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:05:30.189394    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8b30791d-d7f2-425d-ad30-078d7ff079ee-credential-socket\") pod \"e2e-worker-w1-b63c24-openshift-default-d8c64855f-snk9h\" (UID: \"8b30791d-d7f2-425d-ad30-078d7ff079ee\") " pod="openshift-ingress/e2e-worker-w1-b63c24-openshift-default-d8c64855f-snk9h"
Oct 31 01:05:30.189662 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:05:30.189494    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8b30791d-d7f2-425d-ad30-078d7ff079ee-workload-certs\") pod \"e2e-worker-w1-b63c24-openshift-default-d8c64855f-snk9h\" (UID: \"8b30791d-d7f2-425d-ad30-078d7ff079ee\") " pod="openshift-ingress/e2e-worker-w1-b63c24-openshift-default-d8c64855f-snk9h"
Oct 31 01:05:30.189733 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:05:30.189717    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8b30791d-d7f2-425d-ad30-078d7ff079ee-istio-data\") pod \"e2e-worker-w1-b63c24-openshift-default-d8c64855f-snk9h\" (UID: \"8b30791d-d7f2-425d-ad30-078d7ff079ee\") " pod="openshift-ingress/e2e-worker-w1-b63c24-openshift-default-d8c64855f-snk9h"
Oct 31 01:05:30.189785 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:05:30.189756    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8b30791d-d7f2-425d-ad30-078d7ff079ee-istiod-ca-cert\") pod \"e2e-worker-w1-b63c24-openshift-default-d8c64855f-snk9h\" (UID: \"8b30791d-d7f2-425d-ad30-078d7ff079ee\") " pod="openshift-ingress/e2e-worker-w1-b63c24-openshift-default-d8c64855f-snk9h"
Oct 31 01:05:30.189785 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:05:30.189767    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8b30791d-d7f2-425d-ad30-078d7ff079ee-workload-socket\") pod \"e2e-worker-w1-b63c24-openshift-default-d8c64855f-snk9h\" (UID: \"8b30791d-d7f2-425d-ad30-078d7ff079ee\") " pod="openshift-ingress/e2e-worker-w1-b63c24-openshift-default-d8c64855f-snk9h"
Oct 31 01:05:30.191663 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:05:30.191640    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8b30791d-d7f2-425d-ad30-078d7ff079ee-istio-envoy\") pod \"e2e-worker-w1-b63c24-openshift-default-d8c64855f-snk9h\" (UID: \"8b30791d-d7f2-425d-ad30-078d7ff079ee\") " pod="openshift-ingress/e2e-worker-w1-b63c24-openshift-default-d8c64855f-snk9h"
Oct 31 01:05:30.191794 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:05:30.191778    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8b30791d-d7f2-425d-ad30-078d7ff079ee-istio-podinfo\") pod \"e2e-worker-w1-b63c24-openshift-default-d8c64855f-snk9h\" (UID: \"8b30791d-d7f2-425d-ad30-078d7ff079ee\") " pod="openshift-ingress/e2e-worker-w1-b63c24-openshift-default-d8c64855f-snk9h"
Oct 31 01:05:30.248330 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:05:30.248301    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9kh62\" (UniqueName: \"kubernetes.io/projected/8b30791d-d7f2-425d-ad30-078d7ff079ee-kube-api-access-9kh62\") pod \"e2e-worker-w1-b63c24-openshift-default-d8c64855f-snk9h\" (UID: \"8b30791d-d7f2-425d-ad30-078d7ff079ee\") " pod="openshift-ingress/e2e-worker-w1-b63c24-openshift-default-d8c64855f-snk9h"
Oct 31 01:05:30.251695 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:05:30.251672    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8b30791d-d7f2-425d-ad30-078d7ff079ee-istio-token\") pod \"e2e-worker-w1-b63c24-openshift-default-d8c64855f-snk9h\" (UID: \"8b30791d-d7f2-425d-ad30-078d7ff079ee\") " pod="openshift-ingress/e2e-worker-w1-b63c24-openshift-default-d8c64855f-snk9h"
Oct 31 01:05:30.358298 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:05:30.358196    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-b63c24-openshift-default-d8c64855f-snk9h"
Oct 31 01:05:30.782207 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:05:30.782175    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-b63c24-openshift-default-d8c64855f-snk9h"]
Oct 31 01:05:30.812886 ip-10-0-132-173 kubenswrapper[4284]: W1031 01:05:30.812859    4284 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8b30791d_d7f2_425d_ad30_078d7ff079ee.slice/crio-e7f937b5618782f746b4dd478705572591df47cac6c43dfd274450082676bdae WatchSource:0}: Error finding container e7f937b5618782f746b4dd478705572591df47cac6c43dfd274450082676bdae: Status 404 returned error can't find the container with id e7f937b5618782f746b4dd478705572591df47cac6c43dfd274450082676bdae
Oct 31 01:05:30.815517 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:05:30.815487    4284 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 01:05:30.815601 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:05:30.815555    4284 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 01:05:30.815601 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:05:30.815583    4284 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 01:05:31.716336 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:05:31.716298    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-b63c24-openshift-default-d8c64855f-snk9h" event={"ID":"8b30791d-d7f2-425d-ad30-078d7ff079ee","Type":"ContainerStarted","Data":"ecc0a5bd9132f158bfc1c997618730ff87f033074322c55c0d6a84961317fbf6"}
Oct 31 01:05:31.716817 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:05:31.716354    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-b63c24-openshift-default-d8c64855f-snk9h" event={"ID":"8b30791d-d7f2-425d-ad30-078d7ff079ee","Type":"ContainerStarted","Data":"e7f937b5618782f746b4dd478705572591df47cac6c43dfd274450082676bdae"}
Oct 31 01:05:31.928432 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:05:31.928376    4284 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w1-b63c24-openshift-default-d8c64855f-snk9h" podStartSLOduration=2.928361277 podStartE2EDuration="2.928361277s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 01:05:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 01:05:31.88016998 +0000 UTC m=+2246.025176897" watchObservedRunningTime="2026-10-31 01:05:31.928361277 +0000 UTC m=+2246.073368265"
Oct 31 01:05:32.359037 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:05:32.359004    4284 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w1-b63c24-openshift-default-d8c64855f-snk9h"
Oct 31 01:05:32.360723 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:05:32.360697    4284 patch_prober.go:29] interesting pod/e2e-worker-w1-b63c24-openshift-default-d8c64855f-snk9h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.142:15021/healthz/ready\": dial tcp 10.134.0.142:15021: connect: connection refused" start-of-body=
Oct 31 01:05:32.360842 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:05:32.360744    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-b63c24-openshift-default-d8c64855f-snk9h" podUID="8b30791d-d7f2-425d-ad30-078d7ff079ee" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.142:15021/healthz/ready\": dial tcp 10.134.0.142:15021: connect: connection refused"
Oct 31 01:05:33.359032 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:05:33.358993    4284 patch_prober.go:29] interesting pod/e2e-worker-w1-b63c24-openshift-default-d8c64855f-snk9h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.142:15021/healthz/ready\": dial tcp 10.134.0.142:15021: connect: connection refused" start-of-body=
Oct 31 01:05:33.359408 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:05:33.359070    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-b63c24-openshift-default-d8c64855f-snk9h" podUID="8b30791d-d7f2-425d-ad30-078d7ff079ee" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.142:15021/healthz/ready\": dial tcp 10.134.0.142:15021: connect: connection refused"
Oct 31 01:05:34.126685 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:05:34.126647    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:05:34.358551 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:05:34.358520    4284 patch_prober.go:29] interesting pod/e2e-worker-w1-b63c24-openshift-default-d8c64855f-snk9h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.142:15021/healthz/ready\": dial tcp 10.134.0.142:15021: connect: connection refused" start-of-body=
Oct 31 01:05:34.358740 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:05:34.358569    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-b63c24-openshift-default-d8c64855f-snk9h" podUID="8b30791d-d7f2-425d-ad30-078d7ff079ee" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.142:15021/healthz/ready\": dial tcp 10.134.0.142:15021: connect: connection refused"
Oct 31 01:05:35.362369 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:05:35.362332    4284 patch_prober.go:29] interesting pod/e2e-worker-w1-b63c24-openshift-default-d8c64855f-snk9h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.142:15021/healthz/ready\": dial tcp 10.134.0.142:15021: connect: connection refused" start-of-body=
Oct 31 01:05:35.362777 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:05:35.362385    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-b63c24-openshift-default-d8c64855f-snk9h" podUID="8b30791d-d7f2-425d-ad30-078d7ff079ee" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.142:15021/healthz/ready\": dial tcp 10.134.0.142:15021: connect: connection refused"
Oct 31 01:05:36.359485 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:05:36.359442    4284 patch_prober.go:29] interesting pod/e2e-worker-w1-b63c24-openshift-default-d8c64855f-snk9h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.142:15021/healthz/ready\": dial tcp 10.134.0.142:15021: connect: connection refused" start-of-body=
Oct 31 01:05:36.359666 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:05:36.359500    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-b63c24-openshift-default-d8c64855f-snk9h" podUID="8b30791d-d7f2-425d-ad30-078d7ff079ee" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.142:15021/healthz/ready\": dial tcp 10.134.0.142:15021: connect: connection refused"
Oct 31 01:05:37.358966 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:05:37.358927    4284 patch_prober.go:29] interesting pod/e2e-worker-w1-b63c24-openshift-default-d8c64855f-snk9h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.142:15021/healthz/ready\": dial tcp 10.134.0.142:15021: connect: connection refused" start-of-body=
Oct 31 01:05:37.359524 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:05:37.358992    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-b63c24-openshift-default-d8c64855f-snk9h" podUID="8b30791d-d7f2-425d-ad30-078d7ff079ee" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.142:15021/healthz/ready\": dial tcp 10.134.0.142:15021: connect: connection refused"
Oct 31 01:05:38.363049 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:05:38.363019    4284 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w1-b63c24-openshift-default-d8c64855f-snk9h"
Oct 31 01:05:38.363491 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:05:38.363468    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-b63c24-openshift-default-d8c64855f-snk9h"
Oct 31 01:05:38.364120 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:05:38.364100    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w1-b63c24-openshift-default-d8c64855f-snk9h"
Oct 31 01:05:38.933104 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:05:38.933059    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:05:43.777408 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:05:43.777373    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:05:47.014386 ip-10-0-132-173 kubenswrapper[4284]: E1031 01:05:47.014346    4284 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fd8sp" containerName="extractor"
Oct 31 01:05:48.595050 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:05:48.595019    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:05:53.492591 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:05:53.492557    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:05:58.676416 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:05:58.676379    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:05:59.102867 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:05:59.102746    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:06:01.650043 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:01.650005    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:06:03.785833 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:03.785793    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:06:05.483567 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:05.483533    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:06:06.876641 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:06.876599    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:06:07.398706 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:07.398601    4284 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-praxis-b-3f25cd-openshift-default-576b49df5c-j48jb"]
Oct 31 01:06:07.399262 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:07.399247    4284 cpu_manager.go:414] "containerMap: removing container" podUID="4fe7b9bb-dcce-4bb5-b37a-82fe1639b77e" containerName="istio-proxy"
Oct 31 01:06:07.399338 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:07.399263    4284 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4fe7b9bb-dcce-4bb5-b37a-82fe1639b77e" containerName="istio-proxy"
Oct 31 01:06:07.403430 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:07.403410    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-praxis-b-3f25cd-openshift-default-576b49df5c-j48jb"
Oct 31 01:06:07.453778 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:07.453725    4284 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-praxis-b-3f25cd-openshift-default-dockercfg-p2kt7\""
Oct 31 01:06:07.504344 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:07.504312    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-praxis-b-3f25cd-openshift-default-576b49df5c-j48jb"]
Oct 31 01:06:07.523692 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:07.523657    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:06:07.555371 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:07.555339    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f8091219-d375-4a3c-9bea-5178857c8aeb-istio-data\") pod \"e2e-praxis-b-3f25cd-openshift-default-576b49df5c-j48jb\" (UID: \"f8091219-d375-4a3c-9bea-5178857c8aeb\") " pod="openshift-ingress/e2e-praxis-b-3f25cd-openshift-default-576b49df5c-j48jb"
Oct 31 01:06:07.555549 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:07.555392    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f8091219-d375-4a3c-9bea-5178857c8aeb-istio-token\") pod \"e2e-praxis-b-3f25cd-openshift-default-576b49df5c-j48jb\" (UID: \"f8091219-d375-4a3c-9bea-5178857c8aeb\") " pod="openshift-ingress/e2e-praxis-b-3f25cd-openshift-default-576b49df5c-j48jb"
Oct 31 01:06:07.555549 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:07.555528    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f8091219-d375-4a3c-9bea-5178857c8aeb-credential-socket\") pod \"e2e-praxis-b-3f25cd-openshift-default-576b49df5c-j48jb\" (UID: \"f8091219-d375-4a3c-9bea-5178857c8aeb\") " pod="openshift-ingress/e2e-praxis-b-3f25cd-openshift-default-576b49df5c-j48jb"
Oct 31 01:06:07.555645 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:07.555561    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hk8tm\" (UniqueName: \"kubernetes.io/projected/f8091219-d375-4a3c-9bea-5178857c8aeb-kube-api-access-hk8tm\") pod \"e2e-praxis-b-3f25cd-openshift-default-576b49df5c-j48jb\" (UID: \"f8091219-d375-4a3c-9bea-5178857c8aeb\") " pod="openshift-ingress/e2e-praxis-b-3f25cd-openshift-default-576b49df5c-j48jb"
Oct 31 01:06:07.555645 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:07.555600    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f8091219-d375-4a3c-9bea-5178857c8aeb-istiod-ca-cert\") pod \"e2e-praxis-b-3f25cd-openshift-default-576b49df5c-j48jb\" (UID: \"f8091219-d375-4a3c-9bea-5178857c8aeb\") " pod="openshift-ingress/e2e-praxis-b-3f25cd-openshift-default-576b49df5c-j48jb"
Oct 31 01:06:07.555739 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:07.555715    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f8091219-d375-4a3c-9bea-5178857c8aeb-workload-socket\") pod \"e2e-praxis-b-3f25cd-openshift-default-576b49df5c-j48jb\" (UID: \"f8091219-d375-4a3c-9bea-5178857c8aeb\") " pod="openshift-ingress/e2e-praxis-b-3f25cd-openshift-default-576b49df5c-j48jb"
Oct 31 01:06:07.555854 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:07.555836    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f8091219-d375-4a3c-9bea-5178857c8aeb-istio-podinfo\") pod \"e2e-praxis-b-3f25cd-openshift-default-576b49df5c-j48jb\" (UID: \"f8091219-d375-4a3c-9bea-5178857c8aeb\") " pod="openshift-ingress/e2e-praxis-b-3f25cd-openshift-default-576b49df5c-j48jb"
Oct 31 01:06:07.555930 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:07.555875    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f8091219-d375-4a3c-9bea-5178857c8aeb-workload-certs\") pod \"e2e-praxis-b-3f25cd-openshift-default-576b49df5c-j48jb\" (UID: \"f8091219-d375-4a3c-9bea-5178857c8aeb\") " pod="openshift-ingress/e2e-praxis-b-3f25cd-openshift-default-576b49df5c-j48jb"
Oct 31 01:06:07.555991 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:07.555978    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f8091219-d375-4a3c-9bea-5178857c8aeb-istio-envoy\") pod \"e2e-praxis-b-3f25cd-openshift-default-576b49df5c-j48jb\" (UID: \"f8091219-d375-4a3c-9bea-5178857c8aeb\") " pod="openshift-ingress/e2e-praxis-b-3f25cd-openshift-default-576b49df5c-j48jb"
Oct 31 01:06:07.659237 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:07.656794    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f8091219-d375-4a3c-9bea-5178857c8aeb-credential-socket\") pod \"e2e-praxis-b-3f25cd-openshift-default-576b49df5c-j48jb\" (UID: \"f8091219-d375-4a3c-9bea-5178857c8aeb\") " pod="openshift-ingress/e2e-praxis-b-3f25cd-openshift-default-576b49df5c-j48jb"
Oct 31 01:06:07.659237 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:07.656846    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hk8tm\" (UniqueName: \"kubernetes.io/projected/f8091219-d375-4a3c-9bea-5178857c8aeb-kube-api-access-hk8tm\") pod \"e2e-praxis-b-3f25cd-openshift-default-576b49df5c-j48jb\" (UID: \"f8091219-d375-4a3c-9bea-5178857c8aeb\") " pod="openshift-ingress/e2e-praxis-b-3f25cd-openshift-default-576b49df5c-j48jb"
Oct 31 01:06:07.659237 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:07.656881    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f8091219-d375-4a3c-9bea-5178857c8aeb-istiod-ca-cert\") pod \"e2e-praxis-b-3f25cd-openshift-default-576b49df5c-j48jb\" (UID: \"f8091219-d375-4a3c-9bea-5178857c8aeb\") " pod="openshift-ingress/e2e-praxis-b-3f25cd-openshift-default-576b49df5c-j48jb"
Oct 31 01:06:07.659237 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:07.656907    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f8091219-d375-4a3c-9bea-5178857c8aeb-workload-socket\") pod \"e2e-praxis-b-3f25cd-openshift-default-576b49df5c-j48jb\" (UID: \"f8091219-d375-4a3c-9bea-5178857c8aeb\") " pod="openshift-ingress/e2e-praxis-b-3f25cd-openshift-default-576b49df5c-j48jb"
Oct 31 01:06:07.659237 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:07.656970    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f8091219-d375-4a3c-9bea-5178857c8aeb-istio-podinfo\") pod \"e2e-praxis-b-3f25cd-openshift-default-576b49df5c-j48jb\" (UID: \"f8091219-d375-4a3c-9bea-5178857c8aeb\") " pod="openshift-ingress/e2e-praxis-b-3f25cd-openshift-default-576b49df5c-j48jb"
Oct 31 01:06:07.659237 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:07.656997    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f8091219-d375-4a3c-9bea-5178857c8aeb-workload-certs\") pod \"e2e-praxis-b-3f25cd-openshift-default-576b49df5c-j48jb\" (UID: \"f8091219-d375-4a3c-9bea-5178857c8aeb\") " pod="openshift-ingress/e2e-praxis-b-3f25cd-openshift-default-576b49df5c-j48jb"
Oct 31 01:06:07.659237 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:07.657041    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f8091219-d375-4a3c-9bea-5178857c8aeb-istio-envoy\") pod \"e2e-praxis-b-3f25cd-openshift-default-576b49df5c-j48jb\" (UID: \"f8091219-d375-4a3c-9bea-5178857c8aeb\") " pod="openshift-ingress/e2e-praxis-b-3f25cd-openshift-default-576b49df5c-j48jb"
Oct 31 01:06:07.659237 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:07.657104    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f8091219-d375-4a3c-9bea-5178857c8aeb-istio-data\") pod \"e2e-praxis-b-3f25cd-openshift-default-576b49df5c-j48jb\" (UID: \"f8091219-d375-4a3c-9bea-5178857c8aeb\") " pod="openshift-ingress/e2e-praxis-b-3f25cd-openshift-default-576b49df5c-j48jb"
Oct 31 01:06:07.659237 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:07.657147    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f8091219-d375-4a3c-9bea-5178857c8aeb-istio-token\") pod \"e2e-praxis-b-3f25cd-openshift-default-576b49df5c-j48jb\" (UID: \"f8091219-d375-4a3c-9bea-5178857c8aeb\") " pod="openshift-ingress/e2e-praxis-b-3f25cd-openshift-default-576b49df5c-j48jb"
Oct 31 01:06:07.659237 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:07.657822    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f8091219-d375-4a3c-9bea-5178857c8aeb-credential-socket\") pod \"e2e-praxis-b-3f25cd-openshift-default-576b49df5c-j48jb\" (UID: \"f8091219-d375-4a3c-9bea-5178857c8aeb\") " pod="openshift-ingress/e2e-praxis-b-3f25cd-openshift-default-576b49df5c-j48jb"
Oct 31 01:06:07.659237 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:07.658598    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f8091219-d375-4a3c-9bea-5178857c8aeb-istiod-ca-cert\") pod \"e2e-praxis-b-3f25cd-openshift-default-576b49df5c-j48jb\" (UID: \"f8091219-d375-4a3c-9bea-5178857c8aeb\") " pod="openshift-ingress/e2e-praxis-b-3f25cd-openshift-default-576b49df5c-j48jb"
Oct 31 01:06:07.659237 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:07.658794    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f8091219-d375-4a3c-9bea-5178857c8aeb-workload-socket\") pod \"e2e-praxis-b-3f25cd-openshift-default-576b49df5c-j48jb\" (UID: \"f8091219-d375-4a3c-9bea-5178857c8aeb\") " pod="openshift-ingress/e2e-praxis-b-3f25cd-openshift-default-576b49df5c-j48jb"
Oct 31 01:06:07.672539 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:07.672509    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f8091219-d375-4a3c-9bea-5178857c8aeb-istio-envoy\") pod \"e2e-praxis-b-3f25cd-openshift-default-576b49df5c-j48jb\" (UID: \"f8091219-d375-4a3c-9bea-5178857c8aeb\") " pod="openshift-ingress/e2e-praxis-b-3f25cd-openshift-default-576b49df5c-j48jb"
Oct 31 01:06:07.672948 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:07.672924    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f8091219-d375-4a3c-9bea-5178857c8aeb-workload-certs\") pod \"e2e-praxis-b-3f25cd-openshift-default-576b49df5c-j48jb\" (UID: \"f8091219-d375-4a3c-9bea-5178857c8aeb\") " pod="openshift-ingress/e2e-praxis-b-3f25cd-openshift-default-576b49df5c-j48jb"
Oct 31 01:06:07.673270 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:07.673252    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f8091219-d375-4a3c-9bea-5178857c8aeb-istio-data\") pod \"e2e-praxis-b-3f25cd-openshift-default-576b49df5c-j48jb\" (UID: \"f8091219-d375-4a3c-9bea-5178857c8aeb\") " pod="openshift-ingress/e2e-praxis-b-3f25cd-openshift-default-576b49df5c-j48jb"
Oct 31 01:06:07.684186 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:07.684158    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f8091219-d375-4a3c-9bea-5178857c8aeb-istio-podinfo\") pod \"e2e-praxis-b-3f25cd-openshift-default-576b49df5c-j48jb\" (UID: \"f8091219-d375-4a3c-9bea-5178857c8aeb\") " pod="openshift-ingress/e2e-praxis-b-3f25cd-openshift-default-576b49df5c-j48jb"
Oct 31 01:06:07.724730 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:07.724699    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f8091219-d375-4a3c-9bea-5178857c8aeb-istio-token\") pod \"e2e-praxis-b-3f25cd-openshift-default-576b49df5c-j48jb\" (UID: \"f8091219-d375-4a3c-9bea-5178857c8aeb\") " pod="openshift-ingress/e2e-praxis-b-3f25cd-openshift-default-576b49df5c-j48jb"
Oct 31 01:06:07.725673 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:07.725594    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hk8tm\" (UniqueName: \"kubernetes.io/projected/f8091219-d375-4a3c-9bea-5178857c8aeb-kube-api-access-hk8tm\") pod \"e2e-praxis-b-3f25cd-openshift-default-576b49df5c-j48jb\" (UID: \"f8091219-d375-4a3c-9bea-5178857c8aeb\") " pod="openshift-ingress/e2e-praxis-b-3f25cd-openshift-default-576b49df5c-j48jb"
Oct 31 01:06:07.741380 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:07.741340    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:06:08.016312 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:08.016198    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-praxis-b-3f25cd-openshift-default-576b49df5c-j48jb"
Oct 31 01:06:08.441286 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:08.441258    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-praxis-b-3f25cd-openshift-default-576b49df5c-j48jb"]
Oct 31 01:06:08.442235 ip-10-0-132-173 kubenswrapper[4284]: W1031 01:06:08.442194    4284 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf8091219_d375_4a3c_9bea_5178857c8aeb.slice/crio-7fa37467df5b70967318730a1ea4c09b8098850ef4977d9c8c2acf36f35881bf WatchSource:0}: Error finding container 7fa37467df5b70967318730a1ea4c09b8098850ef4977d9c8c2acf36f35881bf: Status 404 returned error can't find the container with id 7fa37467df5b70967318730a1ea4c09b8098850ef4977d9c8c2acf36f35881bf
Oct 31 01:06:08.444841 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:08.444810    4284 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 01:06:08.444927 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:08.444873    4284 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 01:06:08.444927 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:08.444901    4284 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 01:06:08.937745 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:08.937705    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-b-3f25cd-openshift-default-576b49df5c-j48jb" event={"ID":"f8091219-d375-4a3c-9bea-5178857c8aeb","Type":"ContainerStarted","Data":"8afeb0d90d1ab3373c5f42e8c8c1f6805667a9feb4fc80a449c18d332cdafa8b"}
Oct 31 01:06:08.937963 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:08.937795    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-b-3f25cd-openshift-default-576b49df5c-j48jb" event={"ID":"f8091219-d375-4a3c-9bea-5178857c8aeb","Type":"ContainerStarted","Data":"7fa37467df5b70967318730a1ea4c09b8098850ef4977d9c8c2acf36f35881bf"}
Oct 31 01:06:08.985697 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:08.985660    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:06:09.027526 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:09.027495    4284 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-praxis-b-3f25cd-openshift-default-576b49df5c-j48jb"
Oct 31 01:06:09.067138 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:09.067074    4284 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-praxis-b-3f25cd-openshift-default-576b49df5c-j48jb" podStartSLOduration=2.067053705 podStartE2EDuration="2.067053705s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 01:06:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 01:06:09.065565462 +0000 UTC m=+2283.210572412" watchObservedRunningTime="2026-10-31 01:06:09.067053705 +0000 UTC m=+2283.212060623"
Oct 31 01:06:09.067704 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:09.067675    4284 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-praxis-b-3f25cd-openshift-default-576b49df5c-j48jb"
Oct 31 01:06:09.943838 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:09.943804    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-praxis-b-3f25cd-openshift-default-576b49df5c-j48jb"
Oct 31 01:06:09.944641 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:09.944622    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-praxis-b-3f25cd-openshift-default-576b49df5c-j48jb"
Oct 31 01:06:12.092984 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:12.092953    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:06:12.291703 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:12.291670    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:06:14.140960 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:14.140925    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:06:15.173511 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:15.173475    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-praxis-a-7ea4b0-75f8f8786f-6cljj"]
Oct 31 01:06:15.277569 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:15.277540    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-praxis-a-7ea4b0-68cb74f9-rlhnf"]
Oct 31 01:06:15.594349 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:15.592824    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:06:15.755050 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:15.754191    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:06:17.022463 ip-10-0-132-173 kubenswrapper[4284]: E1031 01:06:17.022361    4284 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fd8sp" containerName="extractor"
Oct 31 01:06:18.884967 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:18.884929    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:06:19.079167 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:19.079129    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:06:19.321316 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:19.321280    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:06:20.174267 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:20.174191    4284 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-praxis-a-7ea4b0-75f8f8786f-6cljj" podUID="462d3195-7fa1-4a0c-a8aa-9d32f0e87b6e" containerName="payload-processing" containerID="cri-o://53ed149ce6c083eedfcfb6b1c8fb638e0200559f4abf3311a4503f17f5ab8db9" gracePeriod=25
Oct 31 01:06:20.278568 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:20.278507    4284 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-7ea4b0-68cb74f9-rlhnf" podUID="869411ac-8712-4f5c-8634-7b4cc2ab37f9" containerName="payload-pre-processing" containerID="cri-o://4c38cd91eea1ccbaf125481d2ed16e9c17cea3573ad412c147ff82a8f6f4b325" gracePeriod=25
Oct 31 01:06:20.458734 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:20.458708    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-praxis-a-7ea4b0-75f8f8786f-6cljj"
Oct 31 01:06:20.546632 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:20.546600    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/462d3195-7fa1-4a0c-a8aa-9d32f0e87b6e-kube-api-access-8xl2f\" (UniqueName: \"kubernetes.io/projected/462d3195-7fa1-4a0c-a8aa-9d32f0e87b6e-kube-api-access-8xl2f\") pod \"462d3195-7fa1-4a0c-a8aa-9d32f0e87b6e\" (UID: \"462d3195-7fa1-4a0c-a8aa-9d32f0e87b6e\") "
Oct 31 01:06:20.546853 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:20.546706    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/462d3195-7fa1-4a0c-a8aa-9d32f0e87b6e-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/462d3195-7fa1-4a0c-a8aa-9d32f0e87b6e-plugins-config-volume\") pod \"462d3195-7fa1-4a0c-a8aa-9d32f0e87b6e\" (UID: \"462d3195-7fa1-4a0c-a8aa-9d32f0e87b6e\") "
Oct 31 01:06:20.547121 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:20.547096    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/462d3195-7fa1-4a0c-a8aa-9d32f0e87b6e-plugins-config-volume" pod "462d3195-7fa1-4a0c-a8aa-9d32f0e87b6e" (UID: "462d3195-7fa1-4a0c-a8aa-9d32f0e87b6e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 01:06:20.548772 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:20.548745    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/462d3195-7fa1-4a0c-a8aa-9d32f0e87b6e-kube-api-access-8xl2f" pod "462d3195-7fa1-4a0c-a8aa-9d32f0e87b6e" (UID: "462d3195-7fa1-4a0c-a8aa-9d32f0e87b6e"). InnerVolumeSpecName "kube-api-access-8xl2f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:06:20.566740 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:20.566717    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-7ea4b0-68cb74f9-rlhnf"
Oct 31 01:06:20.647728 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:20.647695    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/869411ac-8712-4f5c-8634-7b4cc2ab37f9-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/869411ac-8712-4f5c-8634-7b4cc2ab37f9-plugins-config-volume\") pod \"869411ac-8712-4f5c-8634-7b4cc2ab37f9\" (UID: \"869411ac-8712-4f5c-8634-7b4cc2ab37f9\") "
Oct 31 01:06:20.647917 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:20.647783    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/869411ac-8712-4f5c-8634-7b4cc2ab37f9-kube-api-access-x9hzm\" (UniqueName: \"kubernetes.io/projected/869411ac-8712-4f5c-8634-7b4cc2ab37f9-kube-api-access-x9hzm\") pod \"869411ac-8712-4f5c-8634-7b4cc2ab37f9\" (UID: \"869411ac-8712-4f5c-8634-7b4cc2ab37f9\") "
Oct 31 01:06:20.648021 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:20.648009    4284 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8xl2f\" (UniqueName: \"kubernetes.io/projected/462d3195-7fa1-4a0c-a8aa-9d32f0e87b6e-kube-api-access-8xl2f\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:06:20.648069 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:20.648023    4284 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/462d3195-7fa1-4a0c-a8aa-9d32f0e87b6e-plugins-config-volume\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:06:20.648166 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:20.648140    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/869411ac-8712-4f5c-8634-7b4cc2ab37f9-plugins-config-volume" pod "869411ac-8712-4f5c-8634-7b4cc2ab37f9" (UID: "869411ac-8712-4f5c-8634-7b4cc2ab37f9"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 01:06:20.649984 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:20.649964    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/869411ac-8712-4f5c-8634-7b4cc2ab37f9-kube-api-access-x9hzm" pod "869411ac-8712-4f5c-8634-7b4cc2ab37f9" (UID: "869411ac-8712-4f5c-8634-7b4cc2ab37f9"). InnerVolumeSpecName "kube-api-access-x9hzm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:06:20.748920 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:20.748841    4284 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/869411ac-8712-4f5c-8634-7b4cc2ab37f9-plugins-config-volume\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:06:20.748920 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:20.748871    4284 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x9hzm\" (UniqueName: \"kubernetes.io/projected/869411ac-8712-4f5c-8634-7b4cc2ab37f9-kube-api-access-x9hzm\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:06:21.108151 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:21.108052    4284 generic.go:356] "Generic (PLEG): container finished" podID="869411ac-8712-4f5c-8634-7b4cc2ab37f9" containerID="4c38cd91eea1ccbaf125481d2ed16e9c17cea3573ad412c147ff82a8f6f4b325" exitCode=0
Oct 31 01:06:21.108151 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:21.108119    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-7ea4b0-68cb74f9-rlhnf"
Oct 31 01:06:21.108151 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:21.108126    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-7ea4b0-68cb74f9-rlhnf" event={"ID":"869411ac-8712-4f5c-8634-7b4cc2ab37f9","Type":"ContainerDied","Data":"4c38cd91eea1ccbaf125481d2ed16e9c17cea3573ad412c147ff82a8f6f4b325"}
Oct 31 01:06:21.108510 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:21.108204    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-7ea4b0-68cb74f9-rlhnf" event={"ID":"869411ac-8712-4f5c-8634-7b4cc2ab37f9","Type":"ContainerDied","Data":"837a72bee81e8b9300e9b905cd4873b2062164ef8da240b337d47aeae23e4ffe"}
Oct 31 01:06:21.108510 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:21.108270    4284 scope.go:122] "RemoveContainer" containerID="4c38cd91eea1ccbaf125481d2ed16e9c17cea3573ad412c147ff82a8f6f4b325"
Oct 31 01:06:21.109943 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:21.109912    4284 generic.go:356] "Generic (PLEG): container finished" podID="462d3195-7fa1-4a0c-a8aa-9d32f0e87b6e" containerID="53ed149ce6c083eedfcfb6b1c8fb638e0200559f4abf3311a4503f17f5ab8db9" exitCode=0
Oct 31 01:06:21.110066 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:21.110019    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-praxis-a-7ea4b0-75f8f8786f-6cljj" event={"ID":"462d3195-7fa1-4a0c-a8aa-9d32f0e87b6e","Type":"ContainerDied","Data":"53ed149ce6c083eedfcfb6b1c8fb638e0200559f4abf3311a4503f17f5ab8db9"}
Oct 31 01:06:21.110121 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:21.110074    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-praxis-a-7ea4b0-75f8f8786f-6cljj" event={"ID":"462d3195-7fa1-4a0c-a8aa-9d32f0e87b6e","Type":"ContainerDied","Data":"e4d625020ec2a65d0405c16c4f5e985b357da8f4c4a720cbf5f9e59429e154b9"}
Oct 31 01:06:21.110160 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:21.110129    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-praxis-a-7ea4b0-75f8f8786f-6cljj"
Oct 31 01:06:21.120963 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:21.120939    4284 scope.go:122] "RemoveContainer" containerID="4c38cd91eea1ccbaf125481d2ed16e9c17cea3573ad412c147ff82a8f6f4b325"
Oct 31 01:06:21.121311 ip-10-0-132-173 kubenswrapper[4284]: E1031 01:06:21.121281    4284 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4c38cd91eea1ccbaf125481d2ed16e9c17cea3573ad412c147ff82a8f6f4b325\": container with ID starting with 4c38cd91eea1ccbaf125481d2ed16e9c17cea3573ad412c147ff82a8f6f4b325 not found: ID does not exist" containerID="4c38cd91eea1ccbaf125481d2ed16e9c17cea3573ad412c147ff82a8f6f4b325"
Oct 31 01:06:21.121425 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:21.121315    4284 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4c38cd91eea1ccbaf125481d2ed16e9c17cea3573ad412c147ff82a8f6f4b325"} err="failed to get container status \"4c38cd91eea1ccbaf125481d2ed16e9c17cea3573ad412c147ff82a8f6f4b325\": rpc error: code = NotFound desc = could not find container \"4c38cd91eea1ccbaf125481d2ed16e9c17cea3573ad412c147ff82a8f6f4b325\": container with ID starting with 4c38cd91eea1ccbaf125481d2ed16e9c17cea3573ad412c147ff82a8f6f4b325 not found: ID does not exist"
Oct 31 01:06:21.121425 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:21.121335    4284 scope.go:122] "RemoveContainer" containerID="53ed149ce6c083eedfcfb6b1c8fb638e0200559f4abf3311a4503f17f5ab8db9"
Oct 31 01:06:21.130475 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:21.130449    4284 scope.go:122] "RemoveContainer" containerID="53ed149ce6c083eedfcfb6b1c8fb638e0200559f4abf3311a4503f17f5ab8db9"
Oct 31 01:06:21.130775 ip-10-0-132-173 kubenswrapper[4284]: E1031 01:06:21.130755    4284 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"53ed149ce6c083eedfcfb6b1c8fb638e0200559f4abf3311a4503f17f5ab8db9\": container with ID starting with 53ed149ce6c083eedfcfb6b1c8fb638e0200559f4abf3311a4503f17f5ab8db9 not found: ID does not exist" containerID="53ed149ce6c083eedfcfb6b1c8fb638e0200559f4abf3311a4503f17f5ab8db9"
Oct 31 01:06:21.130833 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:21.130784    4284 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"53ed149ce6c083eedfcfb6b1c8fb638e0200559f4abf3311a4503f17f5ab8db9"} err="failed to get container status \"53ed149ce6c083eedfcfb6b1c8fb638e0200559f4abf3311a4503f17f5ab8db9\": rpc error: code = NotFound desc = could not find container \"53ed149ce6c083eedfcfb6b1c8fb638e0200559f4abf3311a4503f17f5ab8db9\": container with ID starting with 53ed149ce6c083eedfcfb6b1c8fb638e0200559f4abf3311a4503f17f5ab8db9 not found: ID does not exist"
Oct 31 01:06:21.276144 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:21.276105    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-praxis-a-7ea4b0-75f8f8786f-6cljj"]
Oct 31 01:06:21.323978 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:21.323943    4284 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-praxis-a-7ea4b0-75f8f8786f-6cljj"]
Oct 31 01:06:21.385512 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:21.385427    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-praxis-a-7ea4b0-68cb74f9-rlhnf"]
Oct 31 01:06:21.389643 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:21.389612    4284 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-praxis-a-7ea4b0-68cb74f9-rlhnf"]
Oct 31 01:06:21.963757 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:21.963721    4284 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="462d3195-7fa1-4a0c-a8aa-9d32f0e87b6e" path="/var/lib/kubelet/pods/462d3195-7fa1-4a0c-a8aa-9d32f0e87b6e/volumes"
Oct 31 01:06:21.964140 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:21.964128    4284 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="869411ac-8712-4f5c-8634-7b4cc2ab37f9" path="/var/lib/kubelet/pods/869411ac-8712-4f5c-8634-7b4cc2ab37f9/volumes"
Oct 31 01:06:24.048944 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:24.048907    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:06:28.148656 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:28.148614    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-a-7ea4b0-55986d8fcc-tc4zl"]
Oct 31 01:06:28.149133 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:28.148847    4284 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-7ea4b0-55986d8fcc-tc4zl" podUID="9eb52777-7315-4581-972c-544042cc9c17" containerName="maas-api" containerID="cri-o://d07b2b560b5a379f6f946516295e7def971e2045bebc6ac9bfd8042e08ddeb76" gracePeriod=30
Oct 31 01:06:28.180147 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:28.180105    4284 prober.go:122] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-7ea4b0-55986d8fcc-tc4zl" podUID="9eb52777-7315-4581-972c-544042cc9c17" containerName="maas-api" probeResult="failure" output="Get \"https://10.134.0.133:8443/health\": dial tcp 10.134.0.133:8443: connect: connection refused"
Oct 31 01:06:28.243228 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:28.243152    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:06:28.420935 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:28.420908    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-7ea4b0-55986d8fcc-tc4zl"
Oct 31 01:06:28.569828 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:28.568624    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/9eb52777-7315-4581-972c-544042cc9c17-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9eb52777-7315-4581-972c-544042cc9c17-openshift-service-ca\") pod \"9eb52777-7315-4581-972c-544042cc9c17\" (UID: \"9eb52777-7315-4581-972c-544042cc9c17\") "
Oct 31 01:06:28.569828 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:28.568731    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/9eb52777-7315-4581-972c-544042cc9c17-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9eb52777-7315-4581-972c-544042cc9c17-maas-api-tls\") pod \"9eb52777-7315-4581-972c-544042cc9c17\" (UID: \"9eb52777-7315-4581-972c-544042cc9c17\") "
Oct 31 01:06:28.569828 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:28.568777    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/9eb52777-7315-4581-972c-544042cc9c17-metrics-certs\" (UniqueName: \"kubernetes.io/secret/9eb52777-7315-4581-972c-544042cc9c17-metrics-certs\") pod \"9eb52777-7315-4581-972c-544042cc9c17\" (UID: \"9eb52777-7315-4581-972c-544042cc9c17\") "
Oct 31 01:06:28.569828 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:28.568876    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9eb52777-7315-4581-972c-544042cc9c17-kube-api-access-56cf7\" (UniqueName: \"kubernetes.io/projected/9eb52777-7315-4581-972c-544042cc9c17-kube-api-access-56cf7\") pod \"9eb52777-7315-4581-972c-544042cc9c17\" (UID: \"9eb52777-7315-4581-972c-544042cc9c17\") "
Oct 31 01:06:28.572635 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:28.572586    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9eb52777-7315-4581-972c-544042cc9c17-kube-api-access-56cf7" pod "9eb52777-7315-4581-972c-544042cc9c17" (UID: "9eb52777-7315-4581-972c-544042cc9c17"). InnerVolumeSpecName "kube-api-access-56cf7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:06:28.586298 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:28.574381    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9eb52777-7315-4581-972c-544042cc9c17-openshift-service-ca" pod "9eb52777-7315-4581-972c-544042cc9c17" (UID: "9eb52777-7315-4581-972c-544042cc9c17"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 01:06:28.586298 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:28.584654    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9eb52777-7315-4581-972c-544042cc9c17-metrics-certs" pod "9eb52777-7315-4581-972c-544042cc9c17" (UID: "9eb52777-7315-4581-972c-544042cc9c17"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 01:06:28.586298 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:28.585116    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9eb52777-7315-4581-972c-544042cc9c17-maas-api-tls" pod "9eb52777-7315-4581-972c-544042cc9c17" (UID: "9eb52777-7315-4581-972c-544042cc9c17"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 01:06:28.690546 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:28.681788    4284 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9eb52777-7315-4581-972c-544042cc9c17-openshift-service-ca\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:06:28.690546 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:28.690517    4284 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9eb52777-7315-4581-972c-544042cc9c17-maas-api-tls\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:06:28.690546 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:28.690549    4284 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9eb52777-7315-4581-972c-544042cc9c17-metrics-certs\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:06:28.690546 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:28.690567    4284 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-56cf7\" (UniqueName: \"kubernetes.io/projected/9eb52777-7315-4581-972c-544042cc9c17-kube-api-access-56cf7\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:06:29.175913 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:29.175878    4284 generic.go:356] "Generic (PLEG): container finished" podID="9eb52777-7315-4581-972c-544042cc9c17" containerID="d07b2b560b5a379f6f946516295e7def971e2045bebc6ac9bfd8042e08ddeb76" exitCode=0
Oct 31 01:06:29.176382 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:29.175931    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-7ea4b0-55986d8fcc-tc4zl"
Oct 31 01:06:29.176382 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:29.175964    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-7ea4b0-55986d8fcc-tc4zl" event={"ID":"9eb52777-7315-4581-972c-544042cc9c17","Type":"ContainerDied","Data":"d07b2b560b5a379f6f946516295e7def971e2045bebc6ac9bfd8042e08ddeb76"}
Oct 31 01:06:29.176382 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:29.176025    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-7ea4b0-55986d8fcc-tc4zl" event={"ID":"9eb52777-7315-4581-972c-544042cc9c17","Type":"ContainerDied","Data":"0664daf1e211ee878603d786c77e686dba5984c57bb8fd16c77a43d2fb58ddd3"}
Oct 31 01:06:29.176382 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:29.176056    4284 scope.go:122] "RemoveContainer" containerID="d07b2b560b5a379f6f946516295e7def971e2045bebc6ac9bfd8042e08ddeb76"
Oct 31 01:06:29.186679 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:29.186659    4284 scope.go:122] "RemoveContainer" containerID="d07b2b560b5a379f6f946516295e7def971e2045bebc6ac9bfd8042e08ddeb76"
Oct 31 01:06:29.186997 ip-10-0-132-173 kubenswrapper[4284]: E1031 01:06:29.186978    4284 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d07b2b560b5a379f6f946516295e7def971e2045bebc6ac9bfd8042e08ddeb76\": container with ID starting with d07b2b560b5a379f6f946516295e7def971e2045bebc6ac9bfd8042e08ddeb76 not found: ID does not exist" containerID="d07b2b560b5a379f6f946516295e7def971e2045bebc6ac9bfd8042e08ddeb76"
Oct 31 01:06:29.187050 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:29.187011    4284 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d07b2b560b5a379f6f946516295e7def971e2045bebc6ac9bfd8042e08ddeb76"} err="failed to get container status \"d07b2b560b5a379f6f946516295e7def971e2045bebc6ac9bfd8042e08ddeb76\": rpc error: code = NotFound desc = could not find container \"d07b2b560b5a379f6f946516295e7def971e2045bebc6ac9bfd8042e08ddeb76\": container with ID starting with d07b2b560b5a379f6f946516295e7def971e2045bebc6ac9bfd8042e08ddeb76 not found: ID does not exist"
Oct 31 01:06:29.335061 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:29.335027    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-a-7ea4b0-55986d8fcc-tc4zl"]
Oct 31 01:06:29.394818 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:29.394779    4284 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-a-7ea4b0-55986d8fcc-tc4zl"]
Oct 31 01:06:29.963755 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:29.963711    4284 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="9eb52777-7315-4581-972c-544042cc9c17" path="/var/lib/kubelet/pods/9eb52777-7315-4581-972c-544042cc9c17/volumes"
Oct 31 01:06:30.045558 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:30.045513    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:06:30.206363 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:30.206325    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:06:30.426291 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:30.424249    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:06:31.042326 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:31.042291    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:06:35.241560 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:35.241527    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:06:37.161247 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:37.161184    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:06:37.304989 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:37.304950    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:06:37.533630 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:37.533537    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:06:37.534613 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:37.534591    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-praxis-a-7ea4b0-openshift-default-65c5776677-nw4cq"]
Oct 31 01:06:37.534822 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:37.534800    4284 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-praxis-a-7ea4b0-openshift-default-65c5776677-nw4cq" podUID="4b7ffc18-b208-499f-afed-cb71c96259e4" containerName="istio-proxy" containerID="cri-o://9944fd5115259a76279c2ea42ed37df527f8a384919631ff616c21cd0715148b" gracePeriod=30
Oct 31 01:06:40.365117 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:40.365076    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:06:40.957437 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:40.957406    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-praxis-a-8c08e7-5668d6bd7c-btcgk"]
Oct 31 01:06:41.300070 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:41.299983    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:06:41.492829 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:41.492796    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:06:42.882109 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:42.882083    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-praxis-a-7ea4b0-openshift-default-65c5776677-nw4cq"
Oct 31 01:06:43.039352 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:43.039306    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4b7ffc18-b208-499f-afed-cb71c96259e4-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4b7ffc18-b208-499f-afed-cb71c96259e4-workload-socket\") pod \"4b7ffc18-b208-499f-afed-cb71c96259e4\" (UID: \"4b7ffc18-b208-499f-afed-cb71c96259e4\") "
Oct 31 01:06:43.039352 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:43.039355    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4b7ffc18-b208-499f-afed-cb71c96259e4-istio-token\" (UniqueName: \"kubernetes.io/projected/4b7ffc18-b208-499f-afed-cb71c96259e4-istio-token\") pod \"4b7ffc18-b208-499f-afed-cb71c96259e4\" (UID: \"4b7ffc18-b208-499f-afed-cb71c96259e4\") "
Oct 31 01:06:43.039641 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:43.039397    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4b7ffc18-b208-499f-afed-cb71c96259e4-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4b7ffc18-b208-499f-afed-cb71c96259e4-credential-socket\") pod \"4b7ffc18-b208-499f-afed-cb71c96259e4\" (UID: \"4b7ffc18-b208-499f-afed-cb71c96259e4\") "
Oct 31 01:06:43.039641 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:43.039418    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4b7ffc18-b208-499f-afed-cb71c96259e4-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4b7ffc18-b208-499f-afed-cb71c96259e4-istio-envoy\") pod \"4b7ffc18-b208-499f-afed-cb71c96259e4\" (UID: \"4b7ffc18-b208-499f-afed-cb71c96259e4\") "
Oct 31 01:06:43.039641 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:43.039467    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4b7ffc18-b208-499f-afed-cb71c96259e4-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4b7ffc18-b208-499f-afed-cb71c96259e4-workload-certs\") pod \"4b7ffc18-b208-499f-afed-cb71c96259e4\" (UID: \"4b7ffc18-b208-499f-afed-cb71c96259e4\") "
Oct 31 01:06:43.039641 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:43.039515    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4b7ffc18-b208-499f-afed-cb71c96259e4-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4b7ffc18-b208-499f-afed-cb71c96259e4-istio-data\") pod \"4b7ffc18-b208-499f-afed-cb71c96259e4\" (UID: \"4b7ffc18-b208-499f-afed-cb71c96259e4\") "
Oct 31 01:06:43.039641 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:43.039619    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4b7ffc18-b208-499f-afed-cb71c96259e4-kube-api-access-6vkpb\" (UniqueName: \"kubernetes.io/projected/4b7ffc18-b208-499f-afed-cb71c96259e4-kube-api-access-6vkpb\") pod \"4b7ffc18-b208-499f-afed-cb71c96259e4\" (UID: \"4b7ffc18-b208-499f-afed-cb71c96259e4\") "
Oct 31 01:06:43.039886 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:43.039636    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4b7ffc18-b208-499f-afed-cb71c96259e4-workload-socket" pod "4b7ffc18-b208-499f-afed-cb71c96259e4" (UID: "4b7ffc18-b208-499f-afed-cb71c96259e4"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:06:43.039886 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:43.039666    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/4b7ffc18-b208-499f-afed-cb71c96259e4-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4b7ffc18-b208-499f-afed-cb71c96259e4-istiod-ca-cert\") pod \"4b7ffc18-b208-499f-afed-cb71c96259e4\" (UID: \"4b7ffc18-b208-499f-afed-cb71c96259e4\") "
Oct 31 01:06:43.039886 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:43.039766    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/4b7ffc18-b208-499f-afed-cb71c96259e4-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4b7ffc18-b208-499f-afed-cb71c96259e4-istio-podinfo\") pod \"4b7ffc18-b208-499f-afed-cb71c96259e4\" (UID: \"4b7ffc18-b208-499f-afed-cb71c96259e4\") "
Oct 31 01:06:43.040084 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:43.039910    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4b7ffc18-b208-499f-afed-cb71c96259e4-credential-socket" pod "4b7ffc18-b208-499f-afed-cb71c96259e4" (UID: "4b7ffc18-b208-499f-afed-cb71c96259e4"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:06:43.040084 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:43.040041    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4b7ffc18-b208-499f-afed-cb71c96259e4-istio-data" pod "4b7ffc18-b208-499f-afed-cb71c96259e4" (UID: "4b7ffc18-b208-499f-afed-cb71c96259e4"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:06:43.040203 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:43.040121    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4b7ffc18-b208-499f-afed-cb71c96259e4-workload-certs" pod "4b7ffc18-b208-499f-afed-cb71c96259e4" (UID: "4b7ffc18-b208-499f-afed-cb71c96259e4"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:06:43.040286 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:43.040256    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4b7ffc18-b208-499f-afed-cb71c96259e4-istiod-ca-cert" pod "4b7ffc18-b208-499f-afed-cb71c96259e4" (UID: "4b7ffc18-b208-499f-afed-cb71c96259e4"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 01:06:43.040416 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:43.040398    4284 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4b7ffc18-b208-499f-afed-cb71c96259e4-workload-certs\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:06:43.040480 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:43.040423    4284 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4b7ffc18-b208-499f-afed-cb71c96259e4-istio-data\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:06:43.040480 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:43.040437    4284 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4b7ffc18-b208-499f-afed-cb71c96259e4-istiod-ca-cert\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:06:43.040480 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:43.040452    4284 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4b7ffc18-b208-499f-afed-cb71c96259e4-workload-socket\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:06:43.040480 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:43.040466    4284 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4b7ffc18-b208-499f-afed-cb71c96259e4-credential-socket\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:06:43.041991 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:43.041957    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4b7ffc18-b208-499f-afed-cb71c96259e4-istio-envoy" pod "4b7ffc18-b208-499f-afed-cb71c96259e4" (UID: "4b7ffc18-b208-499f-afed-cb71c96259e4"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:06:43.042123 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:43.042074    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4b7ffc18-b208-499f-afed-cb71c96259e4-istio-token" pod "4b7ffc18-b208-499f-afed-cb71c96259e4" (UID: "4b7ffc18-b208-499f-afed-cb71c96259e4"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:06:43.042263 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:43.042237    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4b7ffc18-b208-499f-afed-cb71c96259e4-kube-api-access-6vkpb" pod "4b7ffc18-b208-499f-afed-cb71c96259e4" (UID: "4b7ffc18-b208-499f-afed-cb71c96259e4"). InnerVolumeSpecName "kube-api-access-6vkpb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:06:43.042418 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:43.042404    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/4b7ffc18-b208-499f-afed-cb71c96259e4-istio-podinfo" pod "4b7ffc18-b208-499f-afed-cb71c96259e4" (UID: "4b7ffc18-b208-499f-afed-cb71c96259e4"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 01:06:43.141853 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:43.141771    4284 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6vkpb\" (UniqueName: \"kubernetes.io/projected/4b7ffc18-b208-499f-afed-cb71c96259e4-kube-api-access-6vkpb\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:06:43.141853 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:43.141800    4284 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4b7ffc18-b208-499f-afed-cb71c96259e4-istio-podinfo\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:06:43.141853 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:43.141812    4284 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4b7ffc18-b208-499f-afed-cb71c96259e4-istio-token\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:06:43.141853 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:43.141821    4284 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4b7ffc18-b208-499f-afed-cb71c96259e4-istio-envoy\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:06:43.270851 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:43.270569    4284 generic.go:356] "Generic (PLEG): container finished" podID="4b7ffc18-b208-499f-afed-cb71c96259e4" containerID="9944fd5115259a76279c2ea42ed37df527f8a384919631ff616c21cd0715148b" exitCode=0
Oct 31 01:06:43.270851 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:43.270662    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-a-7ea4b0-openshift-default-65c5776677-nw4cq" event={"ID":"4b7ffc18-b208-499f-afed-cb71c96259e4","Type":"ContainerDied","Data":"9944fd5115259a76279c2ea42ed37df527f8a384919631ff616c21cd0715148b"}
Oct 31 01:06:43.270851 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:43.270712    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-a-7ea4b0-openshift-default-65c5776677-nw4cq" event={"ID":"4b7ffc18-b208-499f-afed-cb71c96259e4","Type":"ContainerDied","Data":"6065afcc90988b94cd819ccf9e0bf6c32830a041445b5ca782c08be38cb99e95"}
Oct 31 01:06:43.270851 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:43.270737    4284 scope.go:122] "RemoveContainer" containerID="9944fd5115259a76279c2ea42ed37df527f8a384919631ff616c21cd0715148b"
Oct 31 01:06:43.270851 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:43.270750    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-praxis-a-7ea4b0-openshift-default-65c5776677-nw4cq"
Oct 31 01:06:43.281510 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:43.281486    4284 scope.go:122] "RemoveContainer" containerID="1acc08efa1ca567c1520e2bce6f783ec39e030c89612915e86fa5ddc750b6eea"
Oct 31 01:06:43.292117 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:43.292093    4284 scope.go:122] "RemoveContainer" containerID="9944fd5115259a76279c2ea42ed37df527f8a384919631ff616c21cd0715148b"
Oct 31 01:06:43.292470 ip-10-0-132-173 kubenswrapper[4284]: E1031 01:06:43.292448    4284 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9944fd5115259a76279c2ea42ed37df527f8a384919631ff616c21cd0715148b\": container with ID starting with 9944fd5115259a76279c2ea42ed37df527f8a384919631ff616c21cd0715148b not found: ID does not exist" containerID="9944fd5115259a76279c2ea42ed37df527f8a384919631ff616c21cd0715148b"
Oct 31 01:06:43.292544 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:43.292478    4284 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9944fd5115259a76279c2ea42ed37df527f8a384919631ff616c21cd0715148b"} err="failed to get container status \"9944fd5115259a76279c2ea42ed37df527f8a384919631ff616c21cd0715148b\": rpc error: code = NotFound desc = could not find container \"9944fd5115259a76279c2ea42ed37df527f8a384919631ff616c21cd0715148b\": container with ID starting with 9944fd5115259a76279c2ea42ed37df527f8a384919631ff616c21cd0715148b not found: ID does not exist"
Oct 31 01:06:43.292544 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:43.292495    4284 scope.go:122] "RemoveContainer" containerID="1acc08efa1ca567c1520e2bce6f783ec39e030c89612915e86fa5ddc750b6eea"
Oct 31 01:06:43.292776 ip-10-0-132-173 kubenswrapper[4284]: E1031 01:06:43.292756    4284 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1acc08efa1ca567c1520e2bce6f783ec39e030c89612915e86fa5ddc750b6eea\": container with ID starting with 1acc08efa1ca567c1520e2bce6f783ec39e030c89612915e86fa5ddc750b6eea not found: ID does not exist" containerID="1acc08efa1ca567c1520e2bce6f783ec39e030c89612915e86fa5ddc750b6eea"
Oct 31 01:06:43.292820 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:43.292783    4284 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1acc08efa1ca567c1520e2bce6f783ec39e030c89612915e86fa5ddc750b6eea"} err="failed to get container status \"1acc08efa1ca567c1520e2bce6f783ec39e030c89612915e86fa5ddc750b6eea\": rpc error: code = NotFound desc = could not find container \"1acc08efa1ca567c1520e2bce6f783ec39e030c89612915e86fa5ddc750b6eea\": container with ID starting with 1acc08efa1ca567c1520e2bce6f783ec39e030c89612915e86fa5ddc750b6eea not found: ID does not exist"
Oct 31 01:06:43.430288 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:43.430254    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-praxis-a-7ea4b0-openshift-default-65c5776677-nw4cq"]
Oct 31 01:06:43.437147 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:43.437112    4284 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-praxis-a-7ea4b0-openshift-default-65c5776677-nw4cq"]
Oct 31 01:06:43.963400 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:43.963365    4284 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="4b7ffc18-b208-499f-afed-cb71c96259e4" path="/var/lib/kubelet/pods/4b7ffc18-b208-499f-afed-cb71c96259e4/volumes"
Oct 31 01:06:45.135011 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:45.134961    4284 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w1-b63c24-openshift-default-d8c64855f-wfrh9"]
Oct 31 01:06:45.135574 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:45.135558    4284 cpu_manager.go:414] "containerMap: removing container" podUID="9eb52777-7315-4581-972c-544042cc9c17" containerName="maas-api"
Oct 31 01:06:45.135637 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:45.135576    4284 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9eb52777-7315-4581-972c-544042cc9c17" containerName="maas-api"
Oct 31 01:06:45.135637 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:45.135601    4284 cpu_manager.go:414] "containerMap: removing container" podUID="462d3195-7fa1-4a0c-a8aa-9d32f0e87b6e" containerName="payload-processing"
Oct 31 01:06:45.135637 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:45.135607    4284 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="462d3195-7fa1-4a0c-a8aa-9d32f0e87b6e" containerName="payload-processing"
Oct 31 01:06:45.135637 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:45.135628    4284 cpu_manager.go:414] "containerMap: removing container" podUID="869411ac-8712-4f5c-8634-7b4cc2ab37f9" containerName="payload-pre-processing"
Oct 31 01:06:45.135783 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:45.135637    4284 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="869411ac-8712-4f5c-8634-7b4cc2ab37f9" containerName="payload-pre-processing"
Oct 31 01:06:45.135783 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:45.135652    4284 cpu_manager.go:414] "containerMap: removing container" podUID="4b7ffc18-b208-499f-afed-cb71c96259e4" containerName="istio-proxy"
Oct 31 01:06:45.135783 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:45.135659    4284 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4b7ffc18-b208-499f-afed-cb71c96259e4" containerName="istio-proxy"
Oct 31 01:06:45.135783 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:45.135668    4284 cpu_manager.go:414] "containerMap: removing container" podUID="4b7ffc18-b208-499f-afed-cb71c96259e4" containerName="istio-proxy"
Oct 31 01:06:45.135783 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:45.135673    4284 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4b7ffc18-b208-499f-afed-cb71c96259e4" containerName="istio-proxy"
Oct 31 01:06:45.135783 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:45.135746    4284 memory_manager.go:365] "RemoveStaleState removing state" podUID="869411ac-8712-4f5c-8634-7b4cc2ab37f9" containerName="payload-pre-processing"
Oct 31 01:06:45.135783 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:45.135755    4284 memory_manager.go:365] "RemoveStaleState removing state" podUID="4b7ffc18-b208-499f-afed-cb71c96259e4" containerName="istio-proxy"
Oct 31 01:06:45.135783 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:45.135760    4284 memory_manager.go:365] "RemoveStaleState removing state" podUID="4b7ffc18-b208-499f-afed-cb71c96259e4" containerName="istio-proxy"
Oct 31 01:06:45.135783 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:45.135766    4284 memory_manager.go:365] "RemoveStaleState removing state" podUID="462d3195-7fa1-4a0c-a8aa-9d32f0e87b6e" containerName="payload-processing"
Oct 31 01:06:45.135783 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:45.135772    4284 memory_manager.go:365] "RemoveStaleState removing state" podUID="9eb52777-7315-4581-972c-544042cc9c17" containerName="maas-api"
Oct 31 01:06:45.139967 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:45.139948    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-b63c24-openshift-default-d8c64855f-wfrh9"
Oct 31 01:06:45.266588 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:45.266559    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/11338a78-9c55-4df1-8562-360ce3c79c04-istiod-ca-cert\") pod \"e2e-worker-w1-b63c24-openshift-default-d8c64855f-wfrh9\" (UID: \"11338a78-9c55-4df1-8562-360ce3c79c04\") " pod="openshift-ingress/e2e-worker-w1-b63c24-openshift-default-d8c64855f-wfrh9"
Oct 31 01:06:45.266588 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:45.266600    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/11338a78-9c55-4df1-8562-360ce3c79c04-workload-certs\") pod \"e2e-worker-w1-b63c24-openshift-default-d8c64855f-wfrh9\" (UID: \"11338a78-9c55-4df1-8562-360ce3c79c04\") " pod="openshift-ingress/e2e-worker-w1-b63c24-openshift-default-d8c64855f-wfrh9"
Oct 31 01:06:45.266838 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:45.266732    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/11338a78-9c55-4df1-8562-360ce3c79c04-istio-envoy\") pod \"e2e-worker-w1-b63c24-openshift-default-d8c64855f-wfrh9\" (UID: \"11338a78-9c55-4df1-8562-360ce3c79c04\") " pod="openshift-ingress/e2e-worker-w1-b63c24-openshift-default-d8c64855f-wfrh9"
Oct 31 01:06:45.266838 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:45.266778    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/11338a78-9c55-4df1-8562-360ce3c79c04-credential-socket\") pod \"e2e-worker-w1-b63c24-openshift-default-d8c64855f-wfrh9\" (UID: \"11338a78-9c55-4df1-8562-360ce3c79c04\") " pod="openshift-ingress/e2e-worker-w1-b63c24-openshift-default-d8c64855f-wfrh9"
Oct 31 01:06:45.266838 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:45.266800    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/11338a78-9c55-4df1-8562-360ce3c79c04-workload-socket\") pod \"e2e-worker-w1-b63c24-openshift-default-d8c64855f-wfrh9\" (UID: \"11338a78-9c55-4df1-8562-360ce3c79c04\") " pod="openshift-ingress/e2e-worker-w1-b63c24-openshift-default-d8c64855f-wfrh9"
Oct 31 01:06:45.267005 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:45.266877    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/11338a78-9c55-4df1-8562-360ce3c79c04-istio-token\") pod \"e2e-worker-w1-b63c24-openshift-default-d8c64855f-wfrh9\" (UID: \"11338a78-9c55-4df1-8562-360ce3c79c04\") " pod="openshift-ingress/e2e-worker-w1-b63c24-openshift-default-d8c64855f-wfrh9"
Oct 31 01:06:45.267005 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:45.266918    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/11338a78-9c55-4df1-8562-360ce3c79c04-istio-podinfo\") pod \"e2e-worker-w1-b63c24-openshift-default-d8c64855f-wfrh9\" (UID: \"11338a78-9c55-4df1-8562-360ce3c79c04\") " pod="openshift-ingress/e2e-worker-w1-b63c24-openshift-default-d8c64855f-wfrh9"
Oct 31 01:06:45.267111 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:45.267006    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bgfdb\" (UniqueName: \"kubernetes.io/projected/11338a78-9c55-4df1-8562-360ce3c79c04-kube-api-access-bgfdb\") pod \"e2e-worker-w1-b63c24-openshift-default-d8c64855f-wfrh9\" (UID: \"11338a78-9c55-4df1-8562-360ce3c79c04\") " pod="openshift-ingress/e2e-worker-w1-b63c24-openshift-default-d8c64855f-wfrh9"
Oct 31 01:06:45.267111 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:45.267034    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/11338a78-9c55-4df1-8562-360ce3c79c04-istio-data\") pod \"e2e-worker-w1-b63c24-openshift-default-d8c64855f-wfrh9\" (UID: \"11338a78-9c55-4df1-8562-360ce3c79c04\") " pod="openshift-ingress/e2e-worker-w1-b63c24-openshift-default-d8c64855f-wfrh9"
Oct 31 01:06:45.345785 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:45.345067    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-b63c24-openshift-default-d8c64855f-wfrh9"]
Oct 31 01:06:45.368423 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:45.368390    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/11338a78-9c55-4df1-8562-360ce3c79c04-istio-envoy\") pod \"e2e-worker-w1-b63c24-openshift-default-d8c64855f-wfrh9\" (UID: \"11338a78-9c55-4df1-8562-360ce3c79c04\") " pod="openshift-ingress/e2e-worker-w1-b63c24-openshift-default-d8c64855f-wfrh9"
Oct 31 01:06:45.368629 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:45.368454    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/11338a78-9c55-4df1-8562-360ce3c79c04-credential-socket\") pod \"e2e-worker-w1-b63c24-openshift-default-d8c64855f-wfrh9\" (UID: \"11338a78-9c55-4df1-8562-360ce3c79c04\") " pod="openshift-ingress/e2e-worker-w1-b63c24-openshift-default-d8c64855f-wfrh9"
Oct 31 01:06:45.368629 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:45.368483    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/11338a78-9c55-4df1-8562-360ce3c79c04-workload-socket\") pod \"e2e-worker-w1-b63c24-openshift-default-d8c64855f-wfrh9\" (UID: \"11338a78-9c55-4df1-8562-360ce3c79c04\") " pod="openshift-ingress/e2e-worker-w1-b63c24-openshift-default-d8c64855f-wfrh9"
Oct 31 01:06:45.368629 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:45.368509    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/11338a78-9c55-4df1-8562-360ce3c79c04-istio-token\") pod \"e2e-worker-w1-b63c24-openshift-default-d8c64855f-wfrh9\" (UID: \"11338a78-9c55-4df1-8562-360ce3c79c04\") " pod="openshift-ingress/e2e-worker-w1-b63c24-openshift-default-d8c64855f-wfrh9"
Oct 31 01:06:45.368629 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:45.368537    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/11338a78-9c55-4df1-8562-360ce3c79c04-istio-podinfo\") pod \"e2e-worker-w1-b63c24-openshift-default-d8c64855f-wfrh9\" (UID: \"11338a78-9c55-4df1-8562-360ce3c79c04\") " pod="openshift-ingress/e2e-worker-w1-b63c24-openshift-default-d8c64855f-wfrh9"
Oct 31 01:06:45.368629 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:45.368577    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bgfdb\" (UniqueName: \"kubernetes.io/projected/11338a78-9c55-4df1-8562-360ce3c79c04-kube-api-access-bgfdb\") pod \"e2e-worker-w1-b63c24-openshift-default-d8c64855f-wfrh9\" (UID: \"11338a78-9c55-4df1-8562-360ce3c79c04\") " pod="openshift-ingress/e2e-worker-w1-b63c24-openshift-default-d8c64855f-wfrh9"
Oct 31 01:06:45.368629 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:45.368600    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/11338a78-9c55-4df1-8562-360ce3c79c04-istio-data\") pod \"e2e-worker-w1-b63c24-openshift-default-d8c64855f-wfrh9\" (UID: \"11338a78-9c55-4df1-8562-360ce3c79c04\") " pod="openshift-ingress/e2e-worker-w1-b63c24-openshift-default-d8c64855f-wfrh9"
Oct 31 01:06:45.368962 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:45.368736    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/11338a78-9c55-4df1-8562-360ce3c79c04-istiod-ca-cert\") pod \"e2e-worker-w1-b63c24-openshift-default-d8c64855f-wfrh9\" (UID: \"11338a78-9c55-4df1-8562-360ce3c79c04\") " pod="openshift-ingress/e2e-worker-w1-b63c24-openshift-default-d8c64855f-wfrh9"
Oct 31 01:06:45.368962 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:45.368777    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/11338a78-9c55-4df1-8562-360ce3c79c04-workload-certs\") pod \"e2e-worker-w1-b63c24-openshift-default-d8c64855f-wfrh9\" (UID: \"11338a78-9c55-4df1-8562-360ce3c79c04\") " pod="openshift-ingress/e2e-worker-w1-b63c24-openshift-default-d8c64855f-wfrh9"
Oct 31 01:06:45.369076 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:45.368950    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/11338a78-9c55-4df1-8562-360ce3c79c04-workload-socket\") pod \"e2e-worker-w1-b63c24-openshift-default-d8c64855f-wfrh9\" (UID: \"11338a78-9c55-4df1-8562-360ce3c79c04\") " pod="openshift-ingress/e2e-worker-w1-b63c24-openshift-default-d8c64855f-wfrh9"
Oct 31 01:06:45.369132 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:45.369102    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/11338a78-9c55-4df1-8562-360ce3c79c04-istio-data\") pod \"e2e-worker-w1-b63c24-openshift-default-d8c64855f-wfrh9\" (UID: \"11338a78-9c55-4df1-8562-360ce3c79c04\") " pod="openshift-ingress/e2e-worker-w1-b63c24-openshift-default-d8c64855f-wfrh9"
Oct 31 01:06:45.369201 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:45.369176    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/11338a78-9c55-4df1-8562-360ce3c79c04-workload-certs\") pod \"e2e-worker-w1-b63c24-openshift-default-d8c64855f-wfrh9\" (UID: \"11338a78-9c55-4df1-8562-360ce3c79c04\") " pod="openshift-ingress/e2e-worker-w1-b63c24-openshift-default-d8c64855f-wfrh9"
Oct 31 01:06:45.369697 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:45.369669    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/11338a78-9c55-4df1-8562-360ce3c79c04-istiod-ca-cert\") pod \"e2e-worker-w1-b63c24-openshift-default-d8c64855f-wfrh9\" (UID: \"11338a78-9c55-4df1-8562-360ce3c79c04\") " pod="openshift-ingress/e2e-worker-w1-b63c24-openshift-default-d8c64855f-wfrh9"
Oct 31 01:06:45.369828 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:45.369807    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/11338a78-9c55-4df1-8562-360ce3c79c04-credential-socket\") pod \"e2e-worker-w1-b63c24-openshift-default-d8c64855f-wfrh9\" (UID: \"11338a78-9c55-4df1-8562-360ce3c79c04\") " pod="openshift-ingress/e2e-worker-w1-b63c24-openshift-default-d8c64855f-wfrh9"
Oct 31 01:06:45.371529 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:45.371507    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/11338a78-9c55-4df1-8562-360ce3c79c04-istio-envoy\") pod \"e2e-worker-w1-b63c24-openshift-default-d8c64855f-wfrh9\" (UID: \"11338a78-9c55-4df1-8562-360ce3c79c04\") " pod="openshift-ingress/e2e-worker-w1-b63c24-openshift-default-d8c64855f-wfrh9"
Oct 31 01:06:45.371768 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:45.371749    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/11338a78-9c55-4df1-8562-360ce3c79c04-istio-podinfo\") pod \"e2e-worker-w1-b63c24-openshift-default-d8c64855f-wfrh9\" (UID: \"11338a78-9c55-4df1-8562-360ce3c79c04\") " pod="openshift-ingress/e2e-worker-w1-b63c24-openshift-default-d8c64855f-wfrh9"
Oct 31 01:06:45.427919 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:45.427879    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bgfdb\" (UniqueName: \"kubernetes.io/projected/11338a78-9c55-4df1-8562-360ce3c79c04-kube-api-access-bgfdb\") pod \"e2e-worker-w1-b63c24-openshift-default-d8c64855f-wfrh9\" (UID: \"11338a78-9c55-4df1-8562-360ce3c79c04\") " pod="openshift-ingress/e2e-worker-w1-b63c24-openshift-default-d8c64855f-wfrh9"
Oct 31 01:06:45.428078 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:45.427885    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/11338a78-9c55-4df1-8562-360ce3c79c04-istio-token\") pod \"e2e-worker-w1-b63c24-openshift-default-d8c64855f-wfrh9\" (UID: \"11338a78-9c55-4df1-8562-360ce3c79c04\") " pod="openshift-ingress/e2e-worker-w1-b63c24-openshift-default-d8c64855f-wfrh9"
Oct 31 01:06:45.452770 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:45.452735    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-b63c24-openshift-default-d8c64855f-wfrh9"
Oct 31 01:06:45.758525 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:45.758439    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:06:45.880226 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:45.880180    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-b63c24-openshift-default-d8c64855f-wfrh9"]
Oct 31 01:06:45.880649 ip-10-0-132-173 kubenswrapper[4284]: W1031 01:06:45.880627    4284 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod11338a78_9c55_4df1_8562_360ce3c79c04.slice/crio-5f2f3930971db7a7c7584df3491d76f888ee81e7106283391fee77c3f5234ff0 WatchSource:0}: Error finding container 5f2f3930971db7a7c7584df3491d76f888ee81e7106283391fee77c3f5234ff0: Status 404 returned error can't find the container with id 5f2f3930971db7a7c7584df3491d76f888ee81e7106283391fee77c3f5234ff0
Oct 31 01:06:45.883005 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:45.882974    4284 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 01:06:45.883067 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:45.883039    4284 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 01:06:45.883100 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:45.883070    4284 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 01:06:45.958638 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:45.958592    4284 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-praxis-a-8c08e7-5668d6bd7c-btcgk" podUID="c724bcaf-457c-4b0b-8f2c-7ed25c4c8fb5" containerName="payload-processing" containerID="cri-o://6c499c1f0b2818da050f41649ff3d918d0eb641a6ab009d06726dd643f2b3761" gracePeriod=25
Oct 31 01:06:46.292040 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:46.292006    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-b63c24-openshift-default-d8c64855f-wfrh9" event={"ID":"11338a78-9c55-4df1-8562-360ce3c79c04","Type":"ContainerStarted","Data":"0cb7e4f8dc40d7a3c46db72e795056052bfe8ad45080151dd5ad4e3b69ba4aa1"}
Oct 31 01:06:46.292492 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:46.292060    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-b63c24-openshift-default-d8c64855f-wfrh9" event={"ID":"11338a78-9c55-4df1-8562-360ce3c79c04","Type":"ContainerStarted","Data":"5f2f3930971db7a7c7584df3491d76f888ee81e7106283391fee77c3f5234ff0"}
Oct 31 01:06:46.301980 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:46.301944    4284 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-praxis-a-8c08e7-5668d6bd7c-btcgk" podUID="c724bcaf-457c-4b0b-8f2c-7ed25c4c8fb5" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 31 01:06:46.409966 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:46.409908    4284 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w1-b63c24-openshift-default-d8c64855f-wfrh9" podStartSLOduration=1.409892675 podStartE2EDuration="1.409892675s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 01:06:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 01:06:46.407850171 +0000 UTC m=+2320.552857087" watchObservedRunningTime="2026-10-31 01:06:46.409892675 +0000 UTC m=+2320.554899590"
Oct 31 01:06:46.453308 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:46.453273    4284 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w1-b63c24-openshift-default-d8c64855f-wfrh9"
Oct 31 01:06:46.454529 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:46.454504    4284 patch_prober.go:29] interesting pod/e2e-worker-w1-b63c24-openshift-default-d8c64855f-wfrh9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.144:15021/healthz/ready\": dial tcp 10.134.0.144:15021: connect: connection refused" start-of-body=
Oct 31 01:06:46.454661 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:46.454554    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-b63c24-openshift-default-d8c64855f-wfrh9" podUID="11338a78-9c55-4df1-8562-360ce3c79c04" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.144:15021/healthz/ready\": dial tcp 10.134.0.144:15021: connect: connection refused"
Oct 31 01:06:47.015708 ip-10-0-132-173 kubenswrapper[4284]: E1031 01:06:47.015667    4284 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fd8sp" containerName="extractor"
Oct 31 01:06:47.454140 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:47.454107    4284 patch_prober.go:29] interesting pod/e2e-worker-w1-b63c24-openshift-default-d8c64855f-wfrh9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.144:15021/healthz/ready\": dial tcp 10.134.0.144:15021: connect: connection refused" start-of-body=
Oct 31 01:06:47.454670 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:47.454177    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-b63c24-openshift-default-d8c64855f-wfrh9" podUID="11338a78-9c55-4df1-8562-360ce3c79c04" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.144:15021/healthz/ready\": dial tcp 10.134.0.144:15021: connect: connection refused"
Oct 31 01:06:48.453331 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:48.453290    4284 patch_prober.go:29] interesting pod/e2e-worker-w1-b63c24-openshift-default-d8c64855f-wfrh9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.144:15021/healthz/ready\": dial tcp 10.134.0.144:15021: connect: connection refused" start-of-body=
Oct 31 01:06:48.453498 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:48.453350    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-b63c24-openshift-default-d8c64855f-wfrh9" podUID="11338a78-9c55-4df1-8562-360ce3c79c04" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.144:15021/healthz/ready\": dial tcp 10.134.0.144:15021: connect: connection refused"
Oct 31 01:06:49.453510 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:49.453471    4284 patch_prober.go:29] interesting pod/e2e-worker-w1-b63c24-openshift-default-d8c64855f-wfrh9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.144:15021/healthz/ready\": dial tcp 10.134.0.144:15021: connect: connection refused" start-of-body=
Oct 31 01:06:49.453931 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:49.453529    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-b63c24-openshift-default-d8c64855f-wfrh9" podUID="11338a78-9c55-4df1-8562-360ce3c79c04" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.144:15021/healthz/ready\": dial tcp 10.134.0.144:15021: connect: connection refused"
Oct 31 01:06:49.836402 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:49.836318    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-praxis-b-4ab4aa-5d9969dd59-n4vgd"]
Oct 31 01:06:50.210542 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:50.210407    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:06:50.364432 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:50.364392    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:06:50.454024 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:50.453980    4284 patch_prober.go:29] interesting pod/e2e-worker-w1-b63c24-openshift-default-d8c64855f-wfrh9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.144:15021/healthz/ready\": dial tcp 10.134.0.144:15021: connect: connection refused" start-of-body=
Oct 31 01:06:50.454570 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:50.454053    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-b63c24-openshift-default-d8c64855f-wfrh9" podUID="11338a78-9c55-4df1-8562-360ce3c79c04" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.144:15021/healthz/ready\": dial tcp 10.134.0.144:15021: connect: connection refused"
Oct 31 01:06:50.607960 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:50.607868    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:06:51.459713 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:51.457605    4284 patch_prober.go:29] interesting pod/e2e-worker-w1-b63c24-openshift-default-d8c64855f-wfrh9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.144:15021/healthz/ready\": dial tcp 10.134.0.144:15021: connect: connection refused" start-of-body=
Oct 31 01:06:51.459713 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:51.457663    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-b63c24-openshift-default-d8c64855f-wfrh9" podUID="11338a78-9c55-4df1-8562-360ce3c79c04" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.144:15021/healthz/ready\": dial tcp 10.134.0.144:15021: connect: connection refused"
Oct 31 01:06:52.453597 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:52.453555    4284 patch_prober.go:29] interesting pod/e2e-worker-w1-b63c24-openshift-default-d8c64855f-wfrh9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.144:15021/healthz/ready\": dial tcp 10.134.0.144:15021: connect: connection refused" start-of-body=
Oct 31 01:06:52.453797 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:52.453629    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-b63c24-openshift-default-d8c64855f-wfrh9" podUID="11338a78-9c55-4df1-8562-360ce3c79c04" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.144:15021/healthz/ready\": dial tcp 10.134.0.144:15021: connect: connection refused"
Oct 31 01:06:53.453775 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:53.453731    4284 patch_prober.go:29] interesting pod/e2e-worker-w1-b63c24-openshift-default-d8c64855f-wfrh9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.144:15021/healthz/ready\": dial tcp 10.134.0.144:15021: connect: connection refused" start-of-body=
Oct 31 01:06:53.454199 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:53.453802    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-b63c24-openshift-default-d8c64855f-wfrh9" podUID="11338a78-9c55-4df1-8562-360ce3c79c04" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.144:15021/healthz/ready\": dial tcp 10.134.0.144:15021: connect: connection refused"
Oct 31 01:06:54.837550 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:54.837487    4284 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-praxis-b-4ab4aa-5d9969dd59-n4vgd" podUID="82bd1e41-59c3-4dc5-80ac-e1c426f7b0c9" containerName="payload-processing" containerID="cri-o://db8c20fc4fb95c2d48ef59268f25cb7533fb49394a6d9cb5ebaab4ddb335c646" gracePeriod=25
Oct 31 01:06:54.867272 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:54.867235    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:06:55.278508 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:55.278475    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-praxis-b-4ab4aa-5d9969dd59-n4vgd"
Oct 31 01:06:55.405392 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:55.405356    4284 generic.go:356] "Generic (PLEG): container finished" podID="82bd1e41-59c3-4dc5-80ac-e1c426f7b0c9" containerID="db8c20fc4fb95c2d48ef59268f25cb7533fb49394a6d9cb5ebaab4ddb335c646" exitCode=0
Oct 31 01:06:55.405825 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:55.405647    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-praxis-b-4ab4aa-5d9969dd59-n4vgd" event={"ID":"82bd1e41-59c3-4dc5-80ac-e1c426f7b0c9","Type":"ContainerDied","Data":"db8c20fc4fb95c2d48ef59268f25cb7533fb49394a6d9cb5ebaab4ddb335c646"}
Oct 31 01:06:55.406036 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:55.406015    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-praxis-b-4ab4aa-5d9969dd59-n4vgd" event={"ID":"82bd1e41-59c3-4dc5-80ac-e1c426f7b0c9","Type":"ContainerDied","Data":"913f8ba3fc35e0a23dc20d8ae9805670697ac6526faedcf0faa591794d48b782"}
Oct 31 01:06:55.406200 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:55.405668    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-praxis-b-4ab4aa-5d9969dd59-n4vgd"
Oct 31 01:06:55.406690 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:55.406151    4284 scope.go:122] "RemoveContainer" containerID="db8c20fc4fb95c2d48ef59268f25cb7533fb49394a6d9cb5ebaab4ddb335c646"
Oct 31 01:06:55.420884 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:55.420743    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/82bd1e41-59c3-4dc5-80ac-e1c426f7b0c9-kube-api-access-xdjm5\" (UniqueName: \"kubernetes.io/projected/82bd1e41-59c3-4dc5-80ac-e1c426f7b0c9-kube-api-access-xdjm5\") pod \"82bd1e41-59c3-4dc5-80ac-e1c426f7b0c9\" (UID: \"82bd1e41-59c3-4dc5-80ac-e1c426f7b0c9\") "
Oct 31 01:06:55.421094 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:55.421068    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/82bd1e41-59c3-4dc5-80ac-e1c426f7b0c9-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/82bd1e41-59c3-4dc5-80ac-e1c426f7b0c9-plugins-config-volume\") pod \"82bd1e41-59c3-4dc5-80ac-e1c426f7b0c9\" (UID: \"82bd1e41-59c3-4dc5-80ac-e1c426f7b0c9\") "
Oct 31 01:06:55.421560 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:55.421537    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/82bd1e41-59c3-4dc5-80ac-e1c426f7b0c9-plugins-config-volume" pod "82bd1e41-59c3-4dc5-80ac-e1c426f7b0c9" (UID: "82bd1e41-59c3-4dc5-80ac-e1c426f7b0c9"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 01:06:55.421803 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:55.421779    4284 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/82bd1e41-59c3-4dc5-80ac-e1c426f7b0c9-plugins-config-volume\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:06:55.431567 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:55.431539    4284 scope.go:122] "RemoveContainer" containerID="db8c20fc4fb95c2d48ef59268f25cb7533fb49394a6d9cb5ebaab4ddb335c646"
Oct 31 01:06:55.432018 ip-10-0-132-173 kubenswrapper[4284]: E1031 01:06:55.431977    4284 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"db8c20fc4fb95c2d48ef59268f25cb7533fb49394a6d9cb5ebaab4ddb335c646\": container with ID starting with db8c20fc4fb95c2d48ef59268f25cb7533fb49394a6d9cb5ebaab4ddb335c646 not found: ID does not exist" containerID="db8c20fc4fb95c2d48ef59268f25cb7533fb49394a6d9cb5ebaab4ddb335c646"
Oct 31 01:06:55.432140 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:55.432031    4284 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"db8c20fc4fb95c2d48ef59268f25cb7533fb49394a6d9cb5ebaab4ddb335c646"} err="failed to get container status \"db8c20fc4fb95c2d48ef59268f25cb7533fb49394a6d9cb5ebaab4ddb335c646\": rpc error: code = NotFound desc = could not find container \"db8c20fc4fb95c2d48ef59268f25cb7533fb49394a6d9cb5ebaab4ddb335c646\": container with ID starting with db8c20fc4fb95c2d48ef59268f25cb7533fb49394a6d9cb5ebaab4ddb335c646 not found: ID does not exist"
Oct 31 01:06:55.435949 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:55.435920    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/82bd1e41-59c3-4dc5-80ac-e1c426f7b0c9-kube-api-access-xdjm5" pod "82bd1e41-59c3-4dc5-80ac-e1c426f7b0c9" (UID: "82bd1e41-59c3-4dc5-80ac-e1c426f7b0c9"). InnerVolumeSpecName "kube-api-access-xdjm5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:06:55.454458 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:55.454332    4284 patch_prober.go:29] interesting pod/e2e-worker-w1-b63c24-openshift-default-d8c64855f-wfrh9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.144:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 31 01:06:55.454458 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:55.454394    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-b63c24-openshift-default-d8c64855f-wfrh9" podUID="11338a78-9c55-4df1-8562-360ce3c79c04" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.144:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 31 01:06:55.488782 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:55.488684    4284 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w1-b63c24-openshift-default-d8c64855f-wfrh9"
Oct 31 01:06:55.489288 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:55.489260    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-b63c24-openshift-default-d8c64855f-wfrh9"
Oct 31 01:06:55.490273 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:55.490245    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w1-b63c24-openshift-default-d8c64855f-wfrh9"
Oct 31 01:06:55.522628 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:55.522597    4284 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xdjm5\" (UniqueName: \"kubernetes.io/projected/82bd1e41-59c3-4dc5-80ac-e1c426f7b0c9-kube-api-access-xdjm5\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:06:55.873116 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:55.873087    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-praxis-b-4ab4aa-5d9969dd59-n4vgd"]
Oct 31 01:06:55.877534 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:55.877511    4284 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-praxis-b-4ab4aa-5d9969dd59-n4vgd"]
Oct 31 01:06:55.962961 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:55.962929    4284 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="82bd1e41-59c3-4dc5-80ac-e1c426f7b0c9" path="/var/lib/kubelet/pods/82bd1e41-59c3-4dc5-80ac-e1c426f7b0c9/volumes"
Oct 31 01:06:56.302083 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:56.302045    4284 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-praxis-a-8c08e7-5668d6bd7c-btcgk" podUID="c724bcaf-457c-4b0b-8f2c-7ed25c4c8fb5" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 31 01:06:56.382055 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:56.382014    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:06:56.593512 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:56.593432    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:06:56.734352 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:56.734314    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:06:57.321510 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:57.321464    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:06:57.478526 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:57.478487    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:06:57.626705 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:06:57.626612    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:07:01.605698 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:01.605664    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:07:01.991414 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:01.991374    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:07:03.710657 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:03.710620    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:07:03.906946 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:03.906907    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:07:04.071973 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:04.071876    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-31ae40-openshift-default-66bb6b84bc-8wrm6"]
Oct 31 01:07:04.072171 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:04.072120    4284 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-31ae40-openshift-default-66bb6b84bc-8wrm6" podUID="5005c539-d22a-4c99-9235-e08fe509575f" containerName="istio-proxy" containerID="cri-o://7bba6e781c33e10af0892869e6be00e88ffe3edcd856926ea3ec2cdc005a41a9" gracePeriod=30
Oct 31 01:07:04.072958 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:04.072928    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:07:05.077303 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:05.077252    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:07:05.591640 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:05.591600    4284 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-a-f1d081-openshift-default-56c48f6487-9v42v"]
Oct 31 01:07:05.592271 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:05.592253    4284 cpu_manager.go:414] "containerMap: removing container" podUID="82bd1e41-59c3-4dc5-80ac-e1c426f7b0c9" containerName="payload-processing"
Oct 31 01:07:05.592271 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:05.592271    4284 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="82bd1e41-59c3-4dc5-80ac-e1c426f7b0c9" containerName="payload-processing"
Oct 31 01:07:05.592435 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:05.592423    4284 memory_manager.go:365] "RemoveStaleState removing state" podUID="82bd1e41-59c3-4dc5-80ac-e1c426f7b0c9" containerName="payload-processing"
Oct 31 01:07:05.595837 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:05.595817    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-f1d081-openshift-default-56c48f6487-9v42v"
Oct 31 01:07:05.646261 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:05.646228    4284 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-a-f1d081-openshift-default-dockercfg-8l6gb\""
Oct 31 01:07:05.671284 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:05.671252    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/94cb0276-7866-47b5-aa67-1d50bc2250ae-istio-data\") pod \"e2e-ipp-a-f1d081-openshift-default-56c48f6487-9v42v\" (UID: \"94cb0276-7866-47b5-aa67-1d50bc2250ae\") " pod="openshift-ingress/e2e-ipp-a-f1d081-openshift-default-56c48f6487-9v42v"
Oct 31 01:07:05.671462 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:05.671427    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/94cb0276-7866-47b5-aa67-1d50bc2250ae-istio-envoy\") pod \"e2e-ipp-a-f1d081-openshift-default-56c48f6487-9v42v\" (UID: \"94cb0276-7866-47b5-aa67-1d50bc2250ae\") " pod="openshift-ingress/e2e-ipp-a-f1d081-openshift-default-56c48f6487-9v42v"
Oct 31 01:07:05.671559 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:05.671541    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/94cb0276-7866-47b5-aa67-1d50bc2250ae-istio-token\") pod \"e2e-ipp-a-f1d081-openshift-default-56c48f6487-9v42v\" (UID: \"94cb0276-7866-47b5-aa67-1d50bc2250ae\") " pod="openshift-ingress/e2e-ipp-a-f1d081-openshift-default-56c48f6487-9v42v"
Oct 31 01:07:05.671629 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:05.671588    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/94cb0276-7866-47b5-aa67-1d50bc2250ae-workload-socket\") pod \"e2e-ipp-a-f1d081-openshift-default-56c48f6487-9v42v\" (UID: \"94cb0276-7866-47b5-aa67-1d50bc2250ae\") " pod="openshift-ingress/e2e-ipp-a-f1d081-openshift-default-56c48f6487-9v42v"
Oct 31 01:07:05.671685 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:05.671665    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/94cb0276-7866-47b5-aa67-1d50bc2250ae-istio-podinfo\") pod \"e2e-ipp-a-f1d081-openshift-default-56c48f6487-9v42v\" (UID: \"94cb0276-7866-47b5-aa67-1d50bc2250ae\") " pod="openshift-ingress/e2e-ipp-a-f1d081-openshift-default-56c48f6487-9v42v"
Oct 31 01:07:05.671780 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:05.671739    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/94cb0276-7866-47b5-aa67-1d50bc2250ae-credential-socket\") pod \"e2e-ipp-a-f1d081-openshift-default-56c48f6487-9v42v\" (UID: \"94cb0276-7866-47b5-aa67-1d50bc2250ae\") " pod="openshift-ingress/e2e-ipp-a-f1d081-openshift-default-56c48f6487-9v42v"
Oct 31 01:07:05.671826 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:05.671803    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/94cb0276-7866-47b5-aa67-1d50bc2250ae-workload-certs\") pod \"e2e-ipp-a-f1d081-openshift-default-56c48f6487-9v42v\" (UID: \"94cb0276-7866-47b5-aa67-1d50bc2250ae\") " pod="openshift-ingress/e2e-ipp-a-f1d081-openshift-default-56c48f6487-9v42v"
Oct 31 01:07:05.671876 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:05.671858    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/94cb0276-7866-47b5-aa67-1d50bc2250ae-istiod-ca-cert\") pod \"e2e-ipp-a-f1d081-openshift-default-56c48f6487-9v42v\" (UID: \"94cb0276-7866-47b5-aa67-1d50bc2250ae\") " pod="openshift-ingress/e2e-ipp-a-f1d081-openshift-default-56c48f6487-9v42v"
Oct 31 01:07:05.671935 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:05.671919    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b7wqx\" (UniqueName: \"kubernetes.io/projected/94cb0276-7866-47b5-aa67-1d50bc2250ae-kube-api-access-b7wqx\") pod \"e2e-ipp-a-f1d081-openshift-default-56c48f6487-9v42v\" (UID: \"94cb0276-7866-47b5-aa67-1d50bc2250ae\") " pod="openshift-ingress/e2e-ipp-a-f1d081-openshift-default-56c48f6487-9v42v"
Oct 31 01:07:05.681133 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:05.681102    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:07:05.696458 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:05.696426    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-f1d081-openshift-default-56c48f6487-9v42v"]
Oct 31 01:07:05.775166 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:05.775102    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/94cb0276-7866-47b5-aa67-1d50bc2250ae-workload-socket\") pod \"e2e-ipp-a-f1d081-openshift-default-56c48f6487-9v42v\" (UID: \"94cb0276-7866-47b5-aa67-1d50bc2250ae\") " pod="openshift-ingress/e2e-ipp-a-f1d081-openshift-default-56c48f6487-9v42v"
Oct 31 01:07:05.775645 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:05.775605    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/94cb0276-7866-47b5-aa67-1d50bc2250ae-istio-podinfo\") pod \"e2e-ipp-a-f1d081-openshift-default-56c48f6487-9v42v\" (UID: \"94cb0276-7866-47b5-aa67-1d50bc2250ae\") " pod="openshift-ingress/e2e-ipp-a-f1d081-openshift-default-56c48f6487-9v42v"
Oct 31 01:07:05.775811 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:05.775793    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/94cb0276-7866-47b5-aa67-1d50bc2250ae-credential-socket\") pod \"e2e-ipp-a-f1d081-openshift-default-56c48f6487-9v42v\" (UID: \"94cb0276-7866-47b5-aa67-1d50bc2250ae\") " pod="openshift-ingress/e2e-ipp-a-f1d081-openshift-default-56c48f6487-9v42v"
Oct 31 01:07:05.775932 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:05.775919    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/94cb0276-7866-47b5-aa67-1d50bc2250ae-workload-certs\") pod \"e2e-ipp-a-f1d081-openshift-default-56c48f6487-9v42v\" (UID: \"94cb0276-7866-47b5-aa67-1d50bc2250ae\") " pod="openshift-ingress/e2e-ipp-a-f1d081-openshift-default-56c48f6487-9v42v"
Oct 31 01:07:05.776045 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:05.776031    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/94cb0276-7866-47b5-aa67-1d50bc2250ae-istiod-ca-cert\") pod \"e2e-ipp-a-f1d081-openshift-default-56c48f6487-9v42v\" (UID: \"94cb0276-7866-47b5-aa67-1d50bc2250ae\") " pod="openshift-ingress/e2e-ipp-a-f1d081-openshift-default-56c48f6487-9v42v"
Oct 31 01:07:05.776156 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:05.776143    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b7wqx\" (UniqueName: \"kubernetes.io/projected/94cb0276-7866-47b5-aa67-1d50bc2250ae-kube-api-access-b7wqx\") pod \"e2e-ipp-a-f1d081-openshift-default-56c48f6487-9v42v\" (UID: \"94cb0276-7866-47b5-aa67-1d50bc2250ae\") " pod="openshift-ingress/e2e-ipp-a-f1d081-openshift-default-56c48f6487-9v42v"
Oct 31 01:07:05.780719 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:05.776382    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/94cb0276-7866-47b5-aa67-1d50bc2250ae-istio-data\") pod \"e2e-ipp-a-f1d081-openshift-default-56c48f6487-9v42v\" (UID: \"94cb0276-7866-47b5-aa67-1d50bc2250ae\") " pod="openshift-ingress/e2e-ipp-a-f1d081-openshift-default-56c48f6487-9v42v"
Oct 31 01:07:05.780931 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:05.780904    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/94cb0276-7866-47b5-aa67-1d50bc2250ae-workload-socket\") pod \"e2e-ipp-a-f1d081-openshift-default-56c48f6487-9v42v\" (UID: \"94cb0276-7866-47b5-aa67-1d50bc2250ae\") " pod="openshift-ingress/e2e-ipp-a-f1d081-openshift-default-56c48f6487-9v42v"
Oct 31 01:07:05.783632 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:05.780998    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/94cb0276-7866-47b5-aa67-1d50bc2250ae-istio-envoy\") pod \"e2e-ipp-a-f1d081-openshift-default-56c48f6487-9v42v\" (UID: \"94cb0276-7866-47b5-aa67-1d50bc2250ae\") " pod="openshift-ingress/e2e-ipp-a-f1d081-openshift-default-56c48f6487-9v42v"
Oct 31 01:07:05.783877 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:05.782311    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/94cb0276-7866-47b5-aa67-1d50bc2250ae-workload-certs\") pod \"e2e-ipp-a-f1d081-openshift-default-56c48f6487-9v42v\" (UID: \"94cb0276-7866-47b5-aa67-1d50bc2250ae\") " pod="openshift-ingress/e2e-ipp-a-f1d081-openshift-default-56c48f6487-9v42v"
Oct 31 01:07:05.784016 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:05.782878    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/94cb0276-7866-47b5-aa67-1d50bc2250ae-istiod-ca-cert\") pod \"e2e-ipp-a-f1d081-openshift-default-56c48f6487-9v42v\" (UID: \"94cb0276-7866-47b5-aa67-1d50bc2250ae\") " pod="openshift-ingress/e2e-ipp-a-f1d081-openshift-default-56c48f6487-9v42v"
Oct 31 01:07:05.784159 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:05.783172    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/94cb0276-7866-47b5-aa67-1d50bc2250ae-istio-data\") pod \"e2e-ipp-a-f1d081-openshift-default-56c48f6487-9v42v\" (UID: \"94cb0276-7866-47b5-aa67-1d50bc2250ae\") " pod="openshift-ingress/e2e-ipp-a-f1d081-openshift-default-56c48f6487-9v42v"
Oct 31 01:07:05.788361 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:05.780673    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/94cb0276-7866-47b5-aa67-1d50bc2250ae-credential-socket\") pod \"e2e-ipp-a-f1d081-openshift-default-56c48f6487-9v42v\" (UID: \"94cb0276-7866-47b5-aa67-1d50bc2250ae\") " pod="openshift-ingress/e2e-ipp-a-f1d081-openshift-default-56c48f6487-9v42v"
Oct 31 01:07:05.788526 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:05.785059    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/94cb0276-7866-47b5-aa67-1d50bc2250ae-istio-envoy\") pod \"e2e-ipp-a-f1d081-openshift-default-56c48f6487-9v42v\" (UID: \"94cb0276-7866-47b5-aa67-1d50bc2250ae\") " pod="openshift-ingress/e2e-ipp-a-f1d081-openshift-default-56c48f6487-9v42v"
Oct 31 01:07:05.788757 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:05.788736    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/94cb0276-7866-47b5-aa67-1d50bc2250ae-istio-token\") pod \"e2e-ipp-a-f1d081-openshift-default-56c48f6487-9v42v\" (UID: \"94cb0276-7866-47b5-aa67-1d50bc2250ae\") " pod="openshift-ingress/e2e-ipp-a-f1d081-openshift-default-56c48f6487-9v42v"
Oct 31 01:07:05.816238 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:05.816184    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/94cb0276-7866-47b5-aa67-1d50bc2250ae-istio-podinfo\") pod \"e2e-ipp-a-f1d081-openshift-default-56c48f6487-9v42v\" (UID: \"94cb0276-7866-47b5-aa67-1d50bc2250ae\") " pod="openshift-ingress/e2e-ipp-a-f1d081-openshift-default-56c48f6487-9v42v"
Oct 31 01:07:05.853299 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:05.853199    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b7wqx\" (UniqueName: \"kubernetes.io/projected/94cb0276-7866-47b5-aa67-1d50bc2250ae-kube-api-access-b7wqx\") pod \"e2e-ipp-a-f1d081-openshift-default-56c48f6487-9v42v\" (UID: \"94cb0276-7866-47b5-aa67-1d50bc2250ae\") " pod="openshift-ingress/e2e-ipp-a-f1d081-openshift-default-56c48f6487-9v42v"
Oct 31 01:07:05.893240 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:05.887161    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/94cb0276-7866-47b5-aa67-1d50bc2250ae-istio-token\") pod \"e2e-ipp-a-f1d081-openshift-default-56c48f6487-9v42v\" (UID: \"94cb0276-7866-47b5-aa67-1d50bc2250ae\") " pod="openshift-ingress/e2e-ipp-a-f1d081-openshift-default-56c48f6487-9v42v"
Oct 31 01:07:05.908987 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:05.908954    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-f1d081-openshift-default-56c48f6487-9v42v"
Oct 31 01:07:05.915454 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:05.915427    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:07:06.206015 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:06.205984    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-praxis-a-8c08e7-5668d6bd7c-btcgk"
Oct 31 01:07:06.249069 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:06.249034    4284 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-b-3f25cd-587bfbf98-8mgpz"]
Oct 31 01:07:06.249583 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:06.249571    4284 cpu_manager.go:414] "containerMap: removing container" podUID="c724bcaf-457c-4b0b-8f2c-7ed25c4c8fb5" containerName="payload-processing"
Oct 31 01:07:06.249639 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:06.249584    4284 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c724bcaf-457c-4b0b-8f2c-7ed25c4c8fb5" containerName="payload-processing"
Oct 31 01:07:06.249714 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:06.249704    4284 memory_manager.go:365] "RemoveStaleState removing state" podUID="c724bcaf-457c-4b0b-8f2c-7ed25c4c8fb5" containerName="payload-processing"
Oct 31 01:07:06.252899 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:06.252877    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-3f25cd-587bfbf98-8mgpz"
Oct 31 01:07:06.305598 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:06.305361    4284 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-praxis-b-3f25cd-serving-cert\""
Oct 31 01:07:06.309244 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:06.309203    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-b-3f25cd-587bfbf98-8mgpz"]
Oct 31 01:07:06.312932 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:06.312898    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c724bcaf-457c-4b0b-8f2c-7ed25c4c8fb5-kube-api-access-t7qzq\" (UniqueName: \"kubernetes.io/projected/c724bcaf-457c-4b0b-8f2c-7ed25c4c8fb5-kube-api-access-t7qzq\") pod \"c724bcaf-457c-4b0b-8f2c-7ed25c4c8fb5\" (UID: \"c724bcaf-457c-4b0b-8f2c-7ed25c4c8fb5\") "
Oct 31 01:07:06.313095 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:06.313009    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c724bcaf-457c-4b0b-8f2c-7ed25c4c8fb5-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c724bcaf-457c-4b0b-8f2c-7ed25c4c8fb5-plugins-config-volume\") pod \"c724bcaf-457c-4b0b-8f2c-7ed25c4c8fb5\" (UID: \"c724bcaf-457c-4b0b-8f2c-7ed25c4c8fb5\") "
Oct 31 01:07:06.313280 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:06.313259    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f9b75bd3-399f-4631-8561-c09222dc8b6f-metrics-certs\") pod \"maas-api-e2e-praxis-b-3f25cd-587bfbf98-8mgpz\" (UID: \"f9b75bd3-399f-4631-8561-c09222dc8b6f\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-3f25cd-587bfbf98-8mgpz"
Oct 31 01:07:06.313365 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:06.313341    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t8l7w\" (UniqueName: \"kubernetes.io/projected/f9b75bd3-399f-4631-8561-c09222dc8b6f-kube-api-access-t8l7w\") pod \"maas-api-e2e-praxis-b-3f25cd-587bfbf98-8mgpz\" (UID: \"f9b75bd3-399f-4631-8561-c09222dc8b6f\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-3f25cd-587bfbf98-8mgpz"
Oct 31 01:07:06.313479 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:06.313451    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c724bcaf-457c-4b0b-8f2c-7ed25c4c8fb5-plugins-config-volume" pod "c724bcaf-457c-4b0b-8f2c-7ed25c4c8fb5" (UID: "c724bcaf-457c-4b0b-8f2c-7ed25c4c8fb5"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 01:07:06.313522 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:06.313492    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f9b75bd3-399f-4631-8561-c09222dc8b6f-maas-api-tls\") pod \"maas-api-e2e-praxis-b-3f25cd-587bfbf98-8mgpz\" (UID: \"f9b75bd3-399f-4631-8561-c09222dc8b6f\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-3f25cd-587bfbf98-8mgpz"
Oct 31 01:07:06.313582 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:06.313553    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f9b75bd3-399f-4631-8561-c09222dc8b6f-openshift-service-ca\") pod \"maas-api-e2e-praxis-b-3f25cd-587bfbf98-8mgpz\" (UID: \"f9b75bd3-399f-4631-8561-c09222dc8b6f\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-3f25cd-587bfbf98-8mgpz"
Oct 31 01:07:06.313773 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:06.313742    4284 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c724bcaf-457c-4b0b-8f2c-7ed25c4c8fb5-plugins-config-volume\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:07:06.315469 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:06.315440    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c724bcaf-457c-4b0b-8f2c-7ed25c4c8fb5-kube-api-access-t7qzq" pod "c724bcaf-457c-4b0b-8f2c-7ed25c4c8fb5" (UID: "c724bcaf-457c-4b0b-8f2c-7ed25c4c8fb5"). InnerVolumeSpecName "kube-api-access-t7qzq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:07:06.386107 ip-10-0-132-173 kubenswrapper[4284]: W1031 01:07:06.386070    4284 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod94cb0276_7866_47b5_aa67_1d50bc2250ae.slice/crio-c127fd92094bd4e3f14c6127c98ef4ddadfbb72ff721b884e748d7dd1b367f61 WatchSource:0}: Error finding container c127fd92094bd4e3f14c6127c98ef4ddadfbb72ff721b884e748d7dd1b367f61: Status 404 returned error can't find the container with id c127fd92094bd4e3f14c6127c98ef4ddadfbb72ff721b884e748d7dd1b367f61
Oct 31 01:07:06.386107 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:06.386074    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-f1d081-openshift-default-56c48f6487-9v42v"]
Oct 31 01:07:06.389249 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:06.388838    4284 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 01:07:06.389249 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:06.388926    4284 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 01:07:06.389249 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:06.388967    4284 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 01:07:06.414797 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:06.414767    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f9b75bd3-399f-4631-8561-c09222dc8b6f-metrics-certs\") pod \"maas-api-e2e-praxis-b-3f25cd-587bfbf98-8mgpz\" (UID: \"f9b75bd3-399f-4631-8561-c09222dc8b6f\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-3f25cd-587bfbf98-8mgpz"
Oct 31 01:07:06.414925 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:06.414819    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t8l7w\" (UniqueName: \"kubernetes.io/projected/f9b75bd3-399f-4631-8561-c09222dc8b6f-kube-api-access-t8l7w\") pod \"maas-api-e2e-praxis-b-3f25cd-587bfbf98-8mgpz\" (UID: \"f9b75bd3-399f-4631-8561-c09222dc8b6f\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-3f25cd-587bfbf98-8mgpz"
Oct 31 01:07:06.414925 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:06.414891    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f9b75bd3-399f-4631-8561-c09222dc8b6f-maas-api-tls\") pod \"maas-api-e2e-praxis-b-3f25cd-587bfbf98-8mgpz\" (UID: \"f9b75bd3-399f-4631-8561-c09222dc8b6f\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-3f25cd-587bfbf98-8mgpz"
Oct 31 01:07:06.415012 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:06.414928    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f9b75bd3-399f-4631-8561-c09222dc8b6f-openshift-service-ca\") pod \"maas-api-e2e-praxis-b-3f25cd-587bfbf98-8mgpz\" (UID: \"f9b75bd3-399f-4631-8561-c09222dc8b6f\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-3f25cd-587bfbf98-8mgpz"
Oct 31 01:07:06.415012 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:06.415006    4284 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t7qzq\" (UniqueName: \"kubernetes.io/projected/c724bcaf-457c-4b0b-8f2c-7ed25c4c8fb5-kube-api-access-t7qzq\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:07:06.415106 ip-10-0-132-173 kubenswrapper[4284]: E1031 01:07:06.415023    4284 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-praxis-b-3f25cd-serving-cert: secret "maas-api-e2e-praxis-b-3f25cd-serving-cert" not found
Oct 31 01:07:06.415154 ip-10-0-132-173 kubenswrapper[4284]: E1031 01:07:06.415103    4284 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f9b75bd3-399f-4631-8561-c09222dc8b6f-maas-api-tls podName:f9b75bd3-399f-4631-8561-c09222dc8b6f nodeName:}" failed. No retries permitted until 2026-10-31 01:07:06.915082239 +0000 UTC m=+2341.060089147 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/f9b75bd3-399f-4631-8561-c09222dc8b6f-maas-api-tls") pod "maas-api-e2e-praxis-b-3f25cd-587bfbf98-8mgpz" (UID: "f9b75bd3-399f-4631-8561-c09222dc8b6f") : secret "maas-api-e2e-praxis-b-3f25cd-serving-cert" not found
Oct 31 01:07:06.415660 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:06.415640    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f9b75bd3-399f-4631-8561-c09222dc8b6f-openshift-service-ca\") pod \"maas-api-e2e-praxis-b-3f25cd-587bfbf98-8mgpz\" (UID: \"f9b75bd3-399f-4631-8561-c09222dc8b6f\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-3f25cd-587bfbf98-8mgpz"
Oct 31 01:07:06.417409 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:06.417391    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f9b75bd3-399f-4631-8561-c09222dc8b6f-metrics-certs\") pod \"maas-api-e2e-praxis-b-3f25cd-587bfbf98-8mgpz\" (UID: \"f9b75bd3-399f-4631-8561-c09222dc8b6f\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-3f25cd-587bfbf98-8mgpz"
Oct 31 01:07:06.436270 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:06.436203    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:07:06.471797 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:06.471760    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-f1d081-openshift-default-56c48f6487-9v42v" event={"ID":"94cb0276-7866-47b5-aa67-1d50bc2250ae","Type":"ContainerStarted","Data":"c127fd92094bd4e3f14c6127c98ef4ddadfbb72ff721b884e748d7dd1b367f61"}
Oct 31 01:07:06.474156 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:06.474127    4284 generic.go:356] "Generic (PLEG): container finished" podID="c724bcaf-457c-4b0b-8f2c-7ed25c4c8fb5" containerID="6c499c1f0b2818da050f41649ff3d918d0eb641a6ab009d06726dd643f2b3761" exitCode=0
Oct 31 01:07:06.474565 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:06.474202    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-praxis-a-8c08e7-5668d6bd7c-btcgk" event={"ID":"c724bcaf-457c-4b0b-8f2c-7ed25c4c8fb5","Type":"ContainerDied","Data":"6c499c1f0b2818da050f41649ff3d918d0eb641a6ab009d06726dd643f2b3761"}
Oct 31 01:07:06.474565 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:06.474293    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-praxis-a-8c08e7-5668d6bd7c-btcgk" event={"ID":"c724bcaf-457c-4b0b-8f2c-7ed25c4c8fb5","Type":"ContainerDied","Data":"2ba3402c29094ebaeb0a8b23c505c27da47c47e472b9e1a101bfad09276e9f18"}
Oct 31 01:07:06.474565 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:06.474304    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-praxis-a-8c08e7-5668d6bd7c-btcgk"
Oct 31 01:07:06.474565 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:06.474320    4284 scope.go:122] "RemoveContainer" containerID="6c499c1f0b2818da050f41649ff3d918d0eb641a6ab009d06726dd643f2b3761"
Oct 31 01:07:06.474565 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:06.474325    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t8l7w\" (UniqueName: \"kubernetes.io/projected/f9b75bd3-399f-4631-8561-c09222dc8b6f-kube-api-access-t8l7w\") pod \"maas-api-e2e-praxis-b-3f25cd-587bfbf98-8mgpz\" (UID: \"f9b75bd3-399f-4631-8561-c09222dc8b6f\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-3f25cd-587bfbf98-8mgpz"
Oct 31 01:07:06.536463 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:06.534513    4284 scope.go:122] "RemoveContainer" containerID="6c499c1f0b2818da050f41649ff3d918d0eb641a6ab009d06726dd643f2b3761"
Oct 31 01:07:06.536463 ip-10-0-132-173 kubenswrapper[4284]: E1031 01:07:06.535684    4284 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6c499c1f0b2818da050f41649ff3d918d0eb641a6ab009d06726dd643f2b3761\": container with ID starting with 6c499c1f0b2818da050f41649ff3d918d0eb641a6ab009d06726dd643f2b3761 not found: ID does not exist" containerID="6c499c1f0b2818da050f41649ff3d918d0eb641a6ab009d06726dd643f2b3761"
Oct 31 01:07:06.536463 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:06.535754    4284 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6c499c1f0b2818da050f41649ff3d918d0eb641a6ab009d06726dd643f2b3761"} err="failed to get container status \"6c499c1f0b2818da050f41649ff3d918d0eb641a6ab009d06726dd643f2b3761\": rpc error: code = NotFound desc = could not find container \"6c499c1f0b2818da050f41649ff3d918d0eb641a6ab009d06726dd643f2b3761\": container with ID starting with 6c499c1f0b2818da050f41649ff3d918d0eb641a6ab009d06726dd643f2b3761 not found: ID does not exist"
Oct 31 01:07:06.628855 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:06.628819    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:07:06.640288 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:06.640198    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-praxis-a-8c08e7-5668d6bd7c-btcgk"]
Oct 31 01:07:06.647335 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:06.647307    4284 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-praxis-a-8c08e7-5668d6bd7c-btcgk"]
Oct 31 01:07:06.774024 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:06.773969    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:07:06.921807 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:06.921781    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f9b75bd3-399f-4631-8561-c09222dc8b6f-maas-api-tls\") pod \"maas-api-e2e-praxis-b-3f25cd-587bfbf98-8mgpz\" (UID: \"f9b75bd3-399f-4631-8561-c09222dc8b6f\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-3f25cd-587bfbf98-8mgpz"
Oct 31 01:07:06.924464 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:06.924429    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f9b75bd3-399f-4631-8561-c09222dc8b6f-maas-api-tls\") pod \"maas-api-e2e-praxis-b-3f25cd-587bfbf98-8mgpz\" (UID: \"f9b75bd3-399f-4631-8561-c09222dc8b6f\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-3f25cd-587bfbf98-8mgpz"
Oct 31 01:07:07.167342 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:07.167307    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-3f25cd-587bfbf98-8mgpz"
Oct 31 01:07:07.481824 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:07.481738    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-f1d081-openshift-default-56c48f6487-9v42v" event={"ID":"94cb0276-7866-47b5-aa67-1d50bc2250ae","Type":"ContainerStarted","Data":"b2af715ca7ffe62865aece4e7996799523c55d84f84beebc977f60f086a2333a"}
Oct 31 01:07:07.543877 ip-10-0-132-173 kubenswrapper[4284]: W1031 01:07:07.543829    4284 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf9b75bd3_399f_4631_8561_c09222dc8b6f.slice/crio-07463ed13d04a1cfeef8f6bb44d648f16cc7d11e51b9a8a93e4c29b8eeff5c49 WatchSource:0}: Error finding container 07463ed13d04a1cfeef8f6bb44d648f16cc7d11e51b9a8a93e4c29b8eeff5c49: Status 404 returned error can't find the container with id 07463ed13d04a1cfeef8f6bb44d648f16cc7d11e51b9a8a93e4c29b8eeff5c49
Oct 31 01:07:07.582240 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:07.582194    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-b-3f25cd-587bfbf98-8mgpz"]
Oct 31 01:07:07.647460 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:07.647415    4284 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-a-f1d081-openshift-default-56c48f6487-9v42v" podStartSLOduration=2.647400109 podStartE2EDuration="2.647400109s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 01:07:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 01:07:07.645102564 +0000 UTC m=+2341.790109481" watchObservedRunningTime="2026-10-31 01:07:07.647400109 +0000 UTC m=+2341.792407025"
Oct 31 01:07:07.909225 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:07.909177    4284 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-f1d081-openshift-default-56c48f6487-9v42v"
Oct 31 01:07:07.915799 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:07.915769    4284 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-f1d081-openshift-default-56c48f6487-9v42v"
Oct 31 01:07:07.963355 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:07.963319    4284 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c724bcaf-457c-4b0b-8f2c-7ed25c4c8fb5" path="/var/lib/kubelet/pods/c724bcaf-457c-4b0b-8f2c-7ed25c4c8fb5/volumes"
Oct 31 01:07:08.488936 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:08.488901    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-3f25cd-587bfbf98-8mgpz" event={"ID":"f9b75bd3-399f-4631-8561-c09222dc8b6f","Type":"ContainerStarted","Data":"07463ed13d04a1cfeef8f6bb44d648f16cc7d11e51b9a8a93e4c29b8eeff5c49"}
Oct 31 01:07:08.489546 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:08.489090    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-f1d081-openshift-default-56c48f6487-9v42v"
Oct 31 01:07:08.490166 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:08.490148    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-f1d081-openshift-default-56c48f6487-9v42v"
Oct 31 01:07:09.403742 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:09.403716    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-31ae40-openshift-default-66bb6b84bc-8wrm6"
Oct 31 01:07:09.462463 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:09.462424    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5005c539-d22a-4c99-9235-e08fe509575f-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5005c539-d22a-4c99-9235-e08fe509575f-credential-socket\") pod \"5005c539-d22a-4c99-9235-e08fe509575f\" (UID: \"5005c539-d22a-4c99-9235-e08fe509575f\") "
Oct 31 01:07:09.462463 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:09.462466    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5005c539-d22a-4c99-9235-e08fe509575f-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5005c539-d22a-4c99-9235-e08fe509575f-istio-envoy\") pod \"5005c539-d22a-4c99-9235-e08fe509575f\" (UID: \"5005c539-d22a-4c99-9235-e08fe509575f\") "
Oct 31 01:07:09.462739 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:09.462515    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5005c539-d22a-4c99-9235-e08fe509575f-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5005c539-d22a-4c99-9235-e08fe509575f-workload-socket\") pod \"5005c539-d22a-4c99-9235-e08fe509575f\" (UID: \"5005c539-d22a-4c99-9235-e08fe509575f\") "
Oct 31 01:07:09.462739 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:09.462536    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5005c539-d22a-4c99-9235-e08fe509575f-kube-api-access-9tm4x\" (UniqueName: \"kubernetes.io/projected/5005c539-d22a-4c99-9235-e08fe509575f-kube-api-access-9tm4x\") pod \"5005c539-d22a-4c99-9235-e08fe509575f\" (UID: \"5005c539-d22a-4c99-9235-e08fe509575f\") "
Oct 31 01:07:09.462739 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:09.462569    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/5005c539-d22a-4c99-9235-e08fe509575f-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5005c539-d22a-4c99-9235-e08fe509575f-istio-podinfo\") pod \"5005c539-d22a-4c99-9235-e08fe509575f\" (UID: \"5005c539-d22a-4c99-9235-e08fe509575f\") "
Oct 31 01:07:09.462739 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:09.462669    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5005c539-d22a-4c99-9235-e08fe509575f-istio-token\" (UniqueName: \"kubernetes.io/projected/5005c539-d22a-4c99-9235-e08fe509575f-istio-token\") pod \"5005c539-d22a-4c99-9235-e08fe509575f\" (UID: \"5005c539-d22a-4c99-9235-e08fe509575f\") "
Oct 31 01:07:09.462739 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:09.462700    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5005c539-d22a-4c99-9235-e08fe509575f-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5005c539-d22a-4c99-9235-e08fe509575f-workload-certs\") pod \"5005c539-d22a-4c99-9235-e08fe509575f\" (UID: \"5005c539-d22a-4c99-9235-e08fe509575f\") "
Oct 31 01:07:09.463030 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:09.462797    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/5005c539-d22a-4c99-9235-e08fe509575f-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5005c539-d22a-4c99-9235-e08fe509575f-istiod-ca-cert\") pod \"5005c539-d22a-4c99-9235-e08fe509575f\" (UID: \"5005c539-d22a-4c99-9235-e08fe509575f\") "
Oct 31 01:07:09.463030 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:09.462879    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5005c539-d22a-4c99-9235-e08fe509575f-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5005c539-d22a-4c99-9235-e08fe509575f-istio-data\") pod \"5005c539-d22a-4c99-9235-e08fe509575f\" (UID: \"5005c539-d22a-4c99-9235-e08fe509575f\") "
Oct 31 01:07:09.463146 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:09.463066    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5005c539-d22a-4c99-9235-e08fe509575f-credential-socket" pod "5005c539-d22a-4c99-9235-e08fe509575f" (UID: "5005c539-d22a-4c99-9235-e08fe509575f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:07:09.463204 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:09.463165    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5005c539-d22a-4c99-9235-e08fe509575f-workload-socket" pod "5005c539-d22a-4c99-9235-e08fe509575f" (UID: "5005c539-d22a-4c99-9235-e08fe509575f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:07:09.463775 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:09.463535    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5005c539-d22a-4c99-9235-e08fe509575f-workload-certs" pod "5005c539-d22a-4c99-9235-e08fe509575f" (UID: "5005c539-d22a-4c99-9235-e08fe509575f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:07:09.463775 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:09.463517    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5005c539-d22a-4c99-9235-e08fe509575f-istio-data" pod "5005c539-d22a-4c99-9235-e08fe509575f" (UID: "5005c539-d22a-4c99-9235-e08fe509575f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:07:09.463775 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:09.463605    4284 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5005c539-d22a-4c99-9235-e08fe509575f-credential-socket\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:07:09.463775 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:09.463626    4284 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5005c539-d22a-4c99-9235-e08fe509575f-workload-socket\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:07:09.463775 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:09.463734    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5005c539-d22a-4c99-9235-e08fe509575f-istiod-ca-cert" pod "5005c539-d22a-4c99-9235-e08fe509575f" (UID: "5005c539-d22a-4c99-9235-e08fe509575f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 01:07:09.465470 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:09.465427    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5005c539-d22a-4c99-9235-e08fe509575f-kube-api-access-9tm4x" pod "5005c539-d22a-4c99-9235-e08fe509575f" (UID: "5005c539-d22a-4c99-9235-e08fe509575f"). InnerVolumeSpecName "kube-api-access-9tm4x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:07:09.465617 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:09.465541    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5005c539-d22a-4c99-9235-e08fe509575f-istio-envoy" pod "5005c539-d22a-4c99-9235-e08fe509575f" (UID: "5005c539-d22a-4c99-9235-e08fe509575f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:07:09.465833 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:09.465805    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5005c539-d22a-4c99-9235-e08fe509575f-istio-token" pod "5005c539-d22a-4c99-9235-e08fe509575f" (UID: "5005c539-d22a-4c99-9235-e08fe509575f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:07:09.466182 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:09.466162    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/5005c539-d22a-4c99-9235-e08fe509575f-istio-podinfo" pod "5005c539-d22a-4c99-9235-e08fe509575f" (UID: "5005c539-d22a-4c99-9235-e08fe509575f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 01:07:09.496779 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:09.496746    4284 generic.go:356] "Generic (PLEG): container finished" podID="5005c539-d22a-4c99-9235-e08fe509575f" containerID="7bba6e781c33e10af0892869e6be00e88ffe3edcd856926ea3ec2cdc005a41a9" exitCode=0
Oct 31 01:07:09.497297 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:09.496814    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-31ae40-openshift-default-66bb6b84bc-8wrm6"
Oct 31 01:07:09.497297 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:09.496829    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-31ae40-openshift-default-66bb6b84bc-8wrm6" event={"ID":"5005c539-d22a-4c99-9235-e08fe509575f","Type":"ContainerDied","Data":"7bba6e781c33e10af0892869e6be00e88ffe3edcd856926ea3ec2cdc005a41a9"}
Oct 31 01:07:09.497297 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:09.496886    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-31ae40-openshift-default-66bb6b84bc-8wrm6" event={"ID":"5005c539-d22a-4c99-9235-e08fe509575f","Type":"ContainerDied","Data":"55942112b1f7c6d42f731a5b145fc6aed978a5943401048a459a4169b9359601"}
Oct 31 01:07:09.497297 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:09.496916    4284 scope.go:122] "RemoveContainer" containerID="7bba6e781c33e10af0892869e6be00e88ffe3edcd856926ea3ec2cdc005a41a9"
Oct 31 01:07:09.498744 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:09.498716    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-3f25cd-587bfbf98-8mgpz" event={"ID":"f9b75bd3-399f-4631-8561-c09222dc8b6f","Type":"ContainerStarted","Data":"2fcb392edcc529776efdfdbfe2e28025039db079c79d5c1b3e07c6d37e2dfe1a"}
Oct 31 01:07:09.498873 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:09.498786    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-3f25cd-587bfbf98-8mgpz"
Oct 31 01:07:09.507099 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:09.507081    4284 scope.go:122] "RemoveContainer" containerID="838a3937661be6950efee60ba7a12b02ce531b5b69885538b4a9d49fb3b4edc6"
Oct 31 01:07:09.516425 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:09.516395    4284 scope.go:122] "RemoveContainer" containerID="7bba6e781c33e10af0892869e6be00e88ffe3edcd856926ea3ec2cdc005a41a9"
Oct 31 01:07:09.516714 ip-10-0-132-173 kubenswrapper[4284]: E1031 01:07:09.516694    4284 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7bba6e781c33e10af0892869e6be00e88ffe3edcd856926ea3ec2cdc005a41a9\": container with ID starting with 7bba6e781c33e10af0892869e6be00e88ffe3edcd856926ea3ec2cdc005a41a9 not found: ID does not exist" containerID="7bba6e781c33e10af0892869e6be00e88ffe3edcd856926ea3ec2cdc005a41a9"
Oct 31 01:07:09.516752 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:09.516724    4284 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7bba6e781c33e10af0892869e6be00e88ffe3edcd856926ea3ec2cdc005a41a9"} err="failed to get container status \"7bba6e781c33e10af0892869e6be00e88ffe3edcd856926ea3ec2cdc005a41a9\": rpc error: code = NotFound desc = could not find container \"7bba6e781c33e10af0892869e6be00e88ffe3edcd856926ea3ec2cdc005a41a9\": container with ID starting with 7bba6e781c33e10af0892869e6be00e88ffe3edcd856926ea3ec2cdc005a41a9 not found: ID does not exist"
Oct 31 01:07:09.516752 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:09.516743    4284 scope.go:122] "RemoveContainer" containerID="838a3937661be6950efee60ba7a12b02ce531b5b69885538b4a9d49fb3b4edc6"
Oct 31 01:07:09.517028 ip-10-0-132-173 kubenswrapper[4284]: E1031 01:07:09.517011    4284 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"838a3937661be6950efee60ba7a12b02ce531b5b69885538b4a9d49fb3b4edc6\": container with ID starting with 838a3937661be6950efee60ba7a12b02ce531b5b69885538b4a9d49fb3b4edc6 not found: ID does not exist" containerID="838a3937661be6950efee60ba7a12b02ce531b5b69885538b4a9d49fb3b4edc6"
Oct 31 01:07:09.517066 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:09.517035    4284 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"838a3937661be6950efee60ba7a12b02ce531b5b69885538b4a9d49fb3b4edc6"} err="failed to get container status \"838a3937661be6950efee60ba7a12b02ce531b5b69885538b4a9d49fb3b4edc6\": rpc error: code = NotFound desc = could not find container \"838a3937661be6950efee60ba7a12b02ce531b5b69885538b4a9d49fb3b4edc6\": container with ID starting with 838a3937661be6950efee60ba7a12b02ce531b5b69885538b4a9d49fb3b4edc6 not found: ID does not exist"
Oct 31 01:07:09.565815 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:09.565775    4284 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5005c539-d22a-4c99-9235-e08fe509575f-istio-token\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:07:09.565815 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:09.565813    4284 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5005c539-d22a-4c99-9235-e08fe509575f-workload-certs\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:07:09.566072 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:09.565829    4284 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5005c539-d22a-4c99-9235-e08fe509575f-istiod-ca-cert\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:07:09.566072 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:09.565843    4284 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5005c539-d22a-4c99-9235-e08fe509575f-istio-data\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:07:09.566072 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:09.565857    4284 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5005c539-d22a-4c99-9235-e08fe509575f-istio-envoy\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:07:09.566072 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:09.565870    4284 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9tm4x\" (UniqueName: \"kubernetes.io/projected/5005c539-d22a-4c99-9235-e08fe509575f-kube-api-access-9tm4x\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:07:09.566072 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:09.565885    4284 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5005c539-d22a-4c99-9235-e08fe509575f-istio-podinfo\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:07:09.644323 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:09.644188    4284 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-3f25cd-587bfbf98-8mgpz" podStartSLOduration=2.529705307 podStartE2EDuration="3.644173975s" totalImagesPullingTime="1.114468668s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 01:07:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 01:07:09.643495547 +0000 UTC m=+2343.788502464" watchObservedRunningTime="2026-10-31 01:07:09.644173975 +0000 UTC m=+2343.789180890"
Oct 31 01:07:09.752757 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:09.752702    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-31ae40-openshift-default-66bb6b84bc-8wrm6"]
Oct 31 01:07:09.777253 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:09.774376    4284 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-31ae40-openshift-default-66bb6b84bc-8wrm6"]
Oct 31 01:07:09.963427 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:09.963395    4284 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5005c539-d22a-4c99-9235-e08fe509575f" path="/var/lib/kubelet/pods/5005c539-d22a-4c99-9235-e08fe509575f/volumes"
Oct 31 01:07:10.403796 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:10.403768    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:07:10.570979 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:10.570895    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:07:10.729990 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:10.729603    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:07:11.717161 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:11.717125    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:07:14.541108 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:14.541079    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:07:14.738360 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:14.738321    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:07:14.957577 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:14.957543    4284 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-f1d081-7b4d567df9-xm4lr"]
Oct 31 01:07:14.958041 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:14.958029    4284 cpu_manager.go:414] "containerMap: removing container" podUID="5005c539-d22a-4c99-9235-e08fe509575f" containerName="istio-proxy"
Oct 31 01:07:14.958090 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:14.958043    4284 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5005c539-d22a-4c99-9235-e08fe509575f" containerName="istio-proxy"
Oct 31 01:07:14.958090 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:14.958062    4284 cpu_manager.go:414] "containerMap: removing container" podUID="5005c539-d22a-4c99-9235-e08fe509575f" containerName="istio-proxy"
Oct 31 01:07:14.958090 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:14.958067    4284 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5005c539-d22a-4c99-9235-e08fe509575f" containerName="istio-proxy"
Oct 31 01:07:14.958192 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:14.958175    4284 memory_manager.go:365] "RemoveStaleState removing state" podUID="5005c539-d22a-4c99-9235-e08fe509575f" containerName="istio-proxy"
Oct 31 01:07:14.958192 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:14.958184    4284 memory_manager.go:365] "RemoveStaleState removing state" podUID="5005c539-d22a-4c99-9235-e08fe509575f" containerName="istio-proxy"
Oct 31 01:07:14.962909 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:14.962889    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-f1d081-7b4d567df9-xm4lr"
Oct 31 01:07:15.016519 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:15.016488    4284 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-a-f1d081-dockercfg-7vrt8\""
Oct 31 01:07:15.016709 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:15.016690    4284 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-a-f1d081\""
Oct 31 01:07:15.057176 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:15.055106    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n5crn\" (UniqueName: \"kubernetes.io/projected/269e4d26-9e30-4ae6-93b7-111f73f64a81-kube-api-access-n5crn\") pod \"payload-pre-processing-e2e-ipp-a-f1d081-7b4d567df9-xm4lr\" (UID: \"269e4d26-9e30-4ae6-93b7-111f73f64a81\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-f1d081-7b4d567df9-xm4lr"
Oct 31 01:07:15.057176 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:15.055162    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/269e4d26-9e30-4ae6-93b7-111f73f64a81-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-f1d081-7b4d567df9-xm4lr\" (UID: \"269e4d26-9e30-4ae6-93b7-111f73f64a81\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-f1d081-7b4d567df9-xm4lr"
Oct 31 01:07:15.130850 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:15.126997    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-f1d081-7b4d567df9-xm4lr"]
Oct 31 01:07:15.157118 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:15.156265    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n5crn\" (UniqueName: \"kubernetes.io/projected/269e4d26-9e30-4ae6-93b7-111f73f64a81-kube-api-access-n5crn\") pod \"payload-pre-processing-e2e-ipp-a-f1d081-7b4d567df9-xm4lr\" (UID: \"269e4d26-9e30-4ae6-93b7-111f73f64a81\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-f1d081-7b4d567df9-xm4lr"
Oct 31 01:07:15.157118 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:15.156345    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/269e4d26-9e30-4ae6-93b7-111f73f64a81-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-f1d081-7b4d567df9-xm4lr\" (UID: \"269e4d26-9e30-4ae6-93b7-111f73f64a81\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-f1d081-7b4d567df9-xm4lr"
Oct 31 01:07:15.157372 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:15.157336    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/269e4d26-9e30-4ae6-93b7-111f73f64a81-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-f1d081-7b4d567df9-xm4lr\" (UID: \"269e4d26-9e30-4ae6-93b7-111f73f64a81\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-f1d081-7b4d567df9-xm4lr"
Oct 31 01:07:15.162863 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:15.159718    4284 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-f1d081-7866949869-8hd89"]
Oct 31 01:07:15.164639 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:15.164620    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-f1d081-7866949869-8hd89"
Oct 31 01:07:15.192654 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:15.192621    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-f1d081-7866949869-8hd89"]
Oct 31 01:07:15.218459 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:15.218377    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n5crn\" (UniqueName: \"kubernetes.io/projected/269e4d26-9e30-4ae6-93b7-111f73f64a81-kube-api-access-n5crn\") pod \"payload-pre-processing-e2e-ipp-a-f1d081-7b4d567df9-xm4lr\" (UID: \"269e4d26-9e30-4ae6-93b7-111f73f64a81\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-f1d081-7b4d567df9-xm4lr"
Oct 31 01:07:15.257141 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:15.257102    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d200ff24-a6a7-4c25-85a7-184e24963fd2-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-f1d081-7866949869-8hd89\" (UID: \"d200ff24-a6a7-4c25-85a7-184e24963fd2\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-f1d081-7866949869-8hd89"
Oct 31 01:07:15.257371 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:15.257240    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t67f5\" (UniqueName: \"kubernetes.io/projected/d200ff24-a6a7-4c25-85a7-184e24963fd2-kube-api-access-t67f5\") pod \"payload-processing-e2e-ipp-a-f1d081-7866949869-8hd89\" (UID: \"d200ff24-a6a7-4c25-85a7-184e24963fd2\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-f1d081-7866949869-8hd89"
Oct 31 01:07:15.275611 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:15.275577    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-f1d081-7b4d567df9-xm4lr"
Oct 31 01:07:15.360172 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:15.358408    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t67f5\" (UniqueName: \"kubernetes.io/projected/d200ff24-a6a7-4c25-85a7-184e24963fd2-kube-api-access-t67f5\") pod \"payload-processing-e2e-ipp-a-f1d081-7866949869-8hd89\" (UID: \"d200ff24-a6a7-4c25-85a7-184e24963fd2\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-f1d081-7866949869-8hd89"
Oct 31 01:07:15.360172 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:15.358569    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d200ff24-a6a7-4c25-85a7-184e24963fd2-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-f1d081-7866949869-8hd89\" (UID: \"d200ff24-a6a7-4c25-85a7-184e24963fd2\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-f1d081-7866949869-8hd89"
Oct 31 01:07:15.360172 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:15.359455    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d200ff24-a6a7-4c25-85a7-184e24963fd2-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-f1d081-7866949869-8hd89\" (UID: \"d200ff24-a6a7-4c25-85a7-184e24963fd2\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-f1d081-7866949869-8hd89"
Oct 31 01:07:15.418338 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:15.418307    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t67f5\" (UniqueName: \"kubernetes.io/projected/d200ff24-a6a7-4c25-85a7-184e24963fd2-kube-api-access-t67f5\") pod \"payload-processing-e2e-ipp-a-f1d081-7866949869-8hd89\" (UID: \"d200ff24-a6a7-4c25-85a7-184e24963fd2\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-f1d081-7866949869-8hd89"
Oct 31 01:07:15.479928 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:15.479847    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-f1d081-7866949869-8hd89"
Oct 31 01:07:15.512556 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:15.512531    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-3f25cd-587bfbf98-8mgpz"
Oct 31 01:07:15.652751 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:15.650882    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-f1d081-7b4d567df9-xm4lr"]
Oct 31 01:07:15.653245 ip-10-0-132-173 kubenswrapper[4284]: W1031 01:07:15.652845    4284 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod269e4d26_9e30_4ae6_93b7_111f73f64a81.slice/crio-082b2cdfeb3c49fab0c417164305f2c9e444b4b55e90a9231b37eaba89d189cf WatchSource:0}: Error finding container 082b2cdfeb3c49fab0c417164305f2c9e444b4b55e90a9231b37eaba89d189cf: Status 404 returned error can't find the container with id 082b2cdfeb3c49fab0c417164305f2c9e444b4b55e90a9231b37eaba89d189cf
Oct 31 01:07:15.860865 ip-10-0-132-173 kubenswrapper[4284]: W1031 01:07:15.860839    4284 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd200ff24_a6a7_4c25_85a7_184e24963fd2.slice/crio-28322daaca2eb1c2a4b135da72b2c92ee9546c246d02612ab626862f78a5c640 WatchSource:0}: Error finding container 28322daaca2eb1c2a4b135da72b2c92ee9546c246d02612ab626862f78a5c640: Status 404 returned error can't find the container with id 28322daaca2eb1c2a4b135da72b2c92ee9546c246d02612ab626862f78a5c640
Oct 31 01:07:15.861360 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:15.861337    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-f1d081-7866949869-8hd89"]
Oct 31 01:07:16.556150 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:16.556110    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-f1d081-7866949869-8hd89" event={"ID":"d200ff24-a6a7-4c25-85a7-184e24963fd2","Type":"ContainerStarted","Data":"8de2fa6e33514da4668b19e7e0e460ade06b9f1e09687e6210be4a72ea2f671d"}
Oct 31 01:07:16.556399 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:16.556175    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-f1d081-7866949869-8hd89" event={"ID":"d200ff24-a6a7-4c25-85a7-184e24963fd2","Type":"ContainerStarted","Data":"28322daaca2eb1c2a4b135da72b2c92ee9546c246d02612ab626862f78a5c640"}
Oct 31 01:07:16.556399 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:16.556203    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-f1d081-7866949869-8hd89"
Oct 31 01:07:16.557750 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:16.557727    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-f1d081-7b4d567df9-xm4lr" event={"ID":"269e4d26-9e30-4ae6-93b7-111f73f64a81","Type":"ContainerStarted","Data":"82389a1fe4ada0a336784248f1633ab7200cea630deef0466cca2a2994b8044d"}
Oct 31 01:07:16.557863 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:16.557767    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-f1d081-7b4d567df9-xm4lr" event={"ID":"269e4d26-9e30-4ae6-93b7-111f73f64a81","Type":"ContainerStarted","Data":"082b2cdfeb3c49fab0c417164305f2c9e444b4b55e90a9231b37eaba89d189cf"}
Oct 31 01:07:16.557863 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:16.557805    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-f1d081-7b4d567df9-xm4lr"
Oct 31 01:07:16.669954 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:16.669886    4284 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-a-f1d081-7866949869-8hd89" podStartSLOduration=1.6698652539999999 podStartE2EDuration="1.669865254s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 01:07:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 01:07:16.668326704 +0000 UTC m=+2350.813333620" watchObservedRunningTime="2026-10-31 01:07:16.669865254 +0000 UTC m=+2350.814872173"
Oct 31 01:07:16.796237 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:16.790885    4284 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-f1d081-7b4d567df9-xm4lr" podStartSLOduration=2.790866402 podStartE2EDuration="2.790866402s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 01:07:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 01:07:16.790621012 +0000 UTC m=+2350.935627930" watchObservedRunningTime="2026-10-31 01:07:16.790866402 +0000 UTC m=+2350.935873319"
Oct 31 01:07:16.990504 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:16.990459    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:07:17.015397 ip-10-0-132-173 kubenswrapper[4284]: E1031 01:07:17.015362    4284 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fd8sp" containerName="extractor"
Oct 31 01:07:22.610975 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:22.610933    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:07:27.400106 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:27.400070    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:07:27.567923 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:27.567891    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-f1d081-7866949869-8hd89"
Oct 31 01:07:27.568089 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:27.567978    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-f1d081-7b4d567df9-xm4lr"
Oct 31 01:07:28.854100 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:28.854070    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:07:29.045624 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:29.045537    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:07:29.670555 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:29.670520    4284 generic.go:356] "Generic (PLEG): container finished" podID="f8091219-d375-4a3c-9bea-5178857c8aeb" containerID="8afeb0d90d1ab3373c5f42e8c8c1f6805667a9feb4fc80a449c18d332cdafa8b" exitCode=0
Oct 31 01:07:29.670749 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:29.670573    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-b-3f25cd-openshift-default-576b49df5c-j48jb" event={"ID":"f8091219-d375-4a3c-9bea-5178857c8aeb","Type":"ContainerDied","Data":"8afeb0d90d1ab3373c5f42e8c8c1f6805667a9feb4fc80a449c18d332cdafa8b"}
Oct 31 01:07:29.670969 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:29.670953    4284 scope.go:122] "RemoveContainer" containerID="8afeb0d90d1ab3373c5f42e8c8c1f6805667a9feb4fc80a449c18d332cdafa8b"
Oct 31 01:07:29.672394 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:29.672366    4284 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 01:07:29.672465 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:29.672434    4284 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 01:07:29.672504 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:29.672483    4284 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 01:07:30.678080 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:30.678046    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-b-3f25cd-openshift-default-576b49df5c-j48jb" event={"ID":"f8091219-d375-4a3c-9bea-5178857c8aeb","Type":"ContainerStarted","Data":"1cc87c9890541d3e71e42db728eb4eed630e21c8eaae733163e52aaa7ebcd128"}
Oct 31 01:07:31.017285 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:31.017161    4284 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-praxis-b-3f25cd-openshift-default-576b49df5c-j48jb"
Oct 31 01:07:31.017479 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:31.017397    4284 patch_prober.go:29] interesting pod/e2e-praxis-b-3f25cd-openshift-default-576b49df5c-j48jb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.143:15021/healthz/ready\": dial tcp 10.134.0.143:15021: connect: connection refused" start-of-body=
Oct 31 01:07:31.017479 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:31.017437    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-b-3f25cd-openshift-default-576b49df5c-j48jb" podUID="f8091219-d375-4a3c-9bea-5178857c8aeb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.143:15021/healthz/ready\": dial tcp 10.134.0.143:15021: connect: connection refused"
Oct 31 01:07:31.439379 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:31.439348    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-b-4ab4aa-7d9ff55bd6-g7vx5"]
Oct 31 01:07:31.439592 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:31.439569    4284 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-4ab4aa-7d9ff55bd6-g7vx5" podUID="1000e907-d373-47f0-ac62-db82b1d816b3" containerName="maas-api" containerID="cri-o://e4ecd2e35dc6e866147c71f439b1a6035e375ca4d77b8a98f47cfd48f8fc2d65" gracePeriod=30
Oct 31 01:07:31.543286 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:31.543250    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:07:31.733119 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:31.733032    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:07:31.737705 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:31.737676    4284 generic.go:356] "Generic (PLEG): container finished" podID="1000e907-d373-47f0-ac62-db82b1d816b3" containerID="e4ecd2e35dc6e866147c71f439b1a6035e375ca4d77b8a98f47cfd48f8fc2d65" exitCode=0
Oct 31 01:07:31.737705 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:31.737697    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-4ab4aa-7d9ff55bd6-g7vx5" event={"ID":"1000e907-d373-47f0-ac62-db82b1d816b3","Type":"ContainerDied","Data":"e4ecd2e35dc6e866147c71f439b1a6035e375ca4d77b8a98f47cfd48f8fc2d65"}
Oct 31 01:07:31.811400 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:31.811369    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-4ab4aa-7d9ff55bd6-g7vx5"
Oct 31 01:07:31.855638 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:31.855602    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1000e907-d373-47f0-ac62-db82b1d816b3-kube-api-access-bd86x\" (UniqueName: \"kubernetes.io/projected/1000e907-d373-47f0-ac62-db82b1d816b3-kube-api-access-bd86x\") pod \"1000e907-d373-47f0-ac62-db82b1d816b3\" (UID: \"1000e907-d373-47f0-ac62-db82b1d816b3\") "
Oct 31 01:07:31.857237 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:31.857185    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/1000e907-d373-47f0-ac62-db82b1d816b3-metrics-certs\" (UniqueName: \"kubernetes.io/secret/1000e907-d373-47f0-ac62-db82b1d816b3-metrics-certs\") pod \"1000e907-d373-47f0-ac62-db82b1d816b3\" (UID: \"1000e907-d373-47f0-ac62-db82b1d816b3\") "
Oct 31 01:07:31.857591 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:31.857555    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1000e907-d373-47f0-ac62-db82b1d816b3-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1000e907-d373-47f0-ac62-db82b1d816b3-openshift-service-ca\") pod \"1000e907-d373-47f0-ac62-db82b1d816b3\" (UID: \"1000e907-d373-47f0-ac62-db82b1d816b3\") "
Oct 31 01:07:31.872366 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:31.872334    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/1000e907-d373-47f0-ac62-db82b1d816b3-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1000e907-d373-47f0-ac62-db82b1d816b3-maas-api-tls\") pod \"1000e907-d373-47f0-ac62-db82b1d816b3\" (UID: \"1000e907-d373-47f0-ac62-db82b1d816b3\") "
Oct 31 01:07:31.873724 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:31.858629    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1000e907-d373-47f0-ac62-db82b1d816b3-openshift-service-ca" pod "1000e907-d373-47f0-ac62-db82b1d816b3" (UID: "1000e907-d373-47f0-ac62-db82b1d816b3"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 01:07:31.879397 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:31.869500    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1000e907-d373-47f0-ac62-db82b1d816b3-kube-api-access-bd86x" pod "1000e907-d373-47f0-ac62-db82b1d816b3" (UID: "1000e907-d373-47f0-ac62-db82b1d816b3"). InnerVolumeSpecName "kube-api-access-bd86x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:07:31.879553 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:31.878039    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1000e907-d373-47f0-ac62-db82b1d816b3-metrics-certs" pod "1000e907-d373-47f0-ac62-db82b1d816b3" (UID: "1000e907-d373-47f0-ac62-db82b1d816b3"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 01:07:31.896357 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:31.895792    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:07:31.919301 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:31.918711    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1000e907-d373-47f0-ac62-db82b1d816b3-maas-api-tls" pod "1000e907-d373-47f0-ac62-db82b1d816b3" (UID: "1000e907-d373-47f0-ac62-db82b1d816b3"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 01:07:31.974282 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:31.974255    4284 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1000e907-d373-47f0-ac62-db82b1d816b3-openshift-service-ca\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:07:31.974282 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:31.974284    4284 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1000e907-d373-47f0-ac62-db82b1d816b3-maas-api-tls\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:07:31.974467 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:31.974298    4284 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bd86x\" (UniqueName: \"kubernetes.io/projected/1000e907-d373-47f0-ac62-db82b1d816b3-kube-api-access-bd86x\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:07:31.974467 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:31.974312    4284 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1000e907-d373-47f0-ac62-db82b1d816b3-metrics-certs\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:07:32.016725 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:32.016638    4284 patch_prober.go:29] interesting pod/e2e-praxis-b-3f25cd-openshift-default-576b49df5c-j48jb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.143:15021/healthz/ready\": dial tcp 10.134.0.143:15021: connect: connection refused" start-of-body=
Oct 31 01:07:32.016894 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:32.016716    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-b-3f25cd-openshift-default-576b49df5c-j48jb" podUID="f8091219-d375-4a3c-9bea-5178857c8aeb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.143:15021/healthz/ready\": dial tcp 10.134.0.143:15021: connect: connection refused"
Oct 31 01:07:32.745245 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:32.745132    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-4ab4aa-7d9ff55bd6-g7vx5"
Oct 31 01:07:32.745245 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:32.745172    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-4ab4aa-7d9ff55bd6-g7vx5" event={"ID":"1000e907-d373-47f0-ac62-db82b1d816b3","Type":"ContainerDied","Data":"f8c8e8cb4e2ccb35c70ea6a9040c0bcce7d2c528805cb0e98fe3b09e4bb733ab"}
Oct 31 01:07:32.745835 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:32.745276    4284 scope.go:122] "RemoveContainer" containerID="e4ecd2e35dc6e866147c71f439b1a6035e375ca4d77b8a98f47cfd48f8fc2d65"
Oct 31 01:07:32.911849 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:32.911817    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-b-4ab4aa-7d9ff55bd6-g7vx5"]
Oct 31 01:07:32.962196 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:32.962159    4284 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-b-4ab4aa-7d9ff55bd6-g7vx5"]
Oct 31 01:07:33.017673 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:33.017583    4284 patch_prober.go:29] interesting pod/e2e-praxis-b-3f25cd-openshift-default-576b49df5c-j48jb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.143:15021/healthz/ready\": dial tcp 10.134.0.143:15021: connect: connection refused" start-of-body=
Oct 31 01:07:33.018001 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:33.017952    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-b-3f25cd-openshift-default-576b49df5c-j48jb" podUID="f8091219-d375-4a3c-9bea-5178857c8aeb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.143:15021/healthz/ready\": dial tcp 10.134.0.143:15021: connect: connection refused"
Oct 31 01:07:33.106862 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:33.106826    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:07:33.316541 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:33.316456    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:07:33.556356 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:33.556323    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:07:33.964343 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:33.964310    4284 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1000e907-d373-47f0-ac62-db82b1d816b3" path="/var/lib/kubelet/pods/1000e907-d373-47f0-ac62-db82b1d816b3/volumes"
Oct 31 01:07:34.017240 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:34.017178    4284 patch_prober.go:29] interesting pod/e2e-praxis-b-3f25cd-openshift-default-576b49df5c-j48jb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.143:15021/healthz/ready\": dial tcp 10.134.0.143:15021: connect: connection refused" start-of-body=
Oct 31 01:07:34.017432 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:34.017265    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-b-3f25cd-openshift-default-576b49df5c-j48jb" podUID="f8091219-d375-4a3c-9bea-5178857c8aeb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.143:15021/healthz/ready\": dial tcp 10.134.0.143:15021: connect: connection refused"
Oct 31 01:07:34.029082 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:34.029050    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:07:35.016695 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:35.016665    4284 patch_prober.go:29] interesting pod/e2e-praxis-b-3f25cd-openshift-default-576b49df5c-j48jb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.143:15021/healthz/ready\": dial tcp 10.134.0.143:15021: connect: connection refused" start-of-body=
Oct 31 01:07:35.017071 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:35.016715    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-b-3f25cd-openshift-default-576b49df5c-j48jb" podUID="f8091219-d375-4a3c-9bea-5178857c8aeb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.143:15021/healthz/ready\": dial tcp 10.134.0.143:15021: connect: connection refused"
Oct 31 01:07:36.021429 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:36.021402    4284 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-praxis-b-3f25cd-openshift-default-576b49df5c-j48jb"
Oct 31 01:07:36.021835 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:36.021815    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-praxis-b-3f25cd-openshift-default-576b49df5c-j48jb"
Oct 31 01:07:36.022452 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:36.022433    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-praxis-b-3f25cd-openshift-default-576b49df5c-j48jb"
Oct 31 01:07:38.387745 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:38.387704    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:07:40.660575 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:40.660535    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:07:40.820544 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:40.820506    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:07:41.022204 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:41.022116    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-praxis-b-4ab4aa-openshift-default-799d6fd5-pqg68"]
Oct 31 01:07:41.022428 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:41.022385    4284 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-praxis-b-4ab4aa-openshift-default-799d6fd5-pqg68" podUID="832a208d-8b57-444a-9ee2-fe30d01e1525" containerName="istio-proxy" containerID="cri-o://ae92d5f931e923073a29138a26dd8255d4b4e43aff31c99e60bcf2a52f8ef39e" gracePeriod=30
Oct 31 01:07:41.023966 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:41.023940    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:07:43.584435 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:43.584392    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:07:46.301055 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:46.300982    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-praxis-b-4ab4aa-openshift-default-799d6fd5-pqg68"
Oct 31 01:07:46.394852 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:46.394819    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/832a208d-8b57-444a-9ee2-fe30d01e1525-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/832a208d-8b57-444a-9ee2-fe30d01e1525-workload-certs\") pod \"832a208d-8b57-444a-9ee2-fe30d01e1525\" (UID: \"832a208d-8b57-444a-9ee2-fe30d01e1525\") "
Oct 31 01:07:46.395028 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:46.394871    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/832a208d-8b57-444a-9ee2-fe30d01e1525-istio-token\" (UniqueName: \"kubernetes.io/projected/832a208d-8b57-444a-9ee2-fe30d01e1525-istio-token\") pod \"832a208d-8b57-444a-9ee2-fe30d01e1525\" (UID: \"832a208d-8b57-444a-9ee2-fe30d01e1525\") "
Oct 31 01:07:46.395028 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:46.394924    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/832a208d-8b57-444a-9ee2-fe30d01e1525-kube-api-access-8pt59\" (UniqueName: \"kubernetes.io/projected/832a208d-8b57-444a-9ee2-fe30d01e1525-kube-api-access-8pt59\") pod \"832a208d-8b57-444a-9ee2-fe30d01e1525\" (UID: \"832a208d-8b57-444a-9ee2-fe30d01e1525\") "
Oct 31 01:07:46.395028 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:46.394980    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/832a208d-8b57-444a-9ee2-fe30d01e1525-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/832a208d-8b57-444a-9ee2-fe30d01e1525-istio-podinfo\") pod \"832a208d-8b57-444a-9ee2-fe30d01e1525\" (UID: \"832a208d-8b57-444a-9ee2-fe30d01e1525\") "
Oct 31 01:07:46.395028 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:46.395000    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/832a208d-8b57-444a-9ee2-fe30d01e1525-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/832a208d-8b57-444a-9ee2-fe30d01e1525-istio-envoy\") pod \"832a208d-8b57-444a-9ee2-fe30d01e1525\" (UID: \"832a208d-8b57-444a-9ee2-fe30d01e1525\") "
Oct 31 01:07:46.395328 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:46.395047    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/832a208d-8b57-444a-9ee2-fe30d01e1525-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/832a208d-8b57-444a-9ee2-fe30d01e1525-workload-socket\") pod \"832a208d-8b57-444a-9ee2-fe30d01e1525\" (UID: \"832a208d-8b57-444a-9ee2-fe30d01e1525\") "
Oct 31 01:07:46.395328 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:46.395096    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/832a208d-8b57-444a-9ee2-fe30d01e1525-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/832a208d-8b57-444a-9ee2-fe30d01e1525-istiod-ca-cert\") pod \"832a208d-8b57-444a-9ee2-fe30d01e1525\" (UID: \"832a208d-8b57-444a-9ee2-fe30d01e1525\") "
Oct 31 01:07:46.395328 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:46.395126    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/832a208d-8b57-444a-9ee2-fe30d01e1525-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/832a208d-8b57-444a-9ee2-fe30d01e1525-istio-data\") pod \"832a208d-8b57-444a-9ee2-fe30d01e1525\" (UID: \"832a208d-8b57-444a-9ee2-fe30d01e1525\") "
Oct 31 01:07:46.395328 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:46.395152    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/832a208d-8b57-444a-9ee2-fe30d01e1525-workload-certs" pod "832a208d-8b57-444a-9ee2-fe30d01e1525" (UID: "832a208d-8b57-444a-9ee2-fe30d01e1525"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:07:46.395328 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:46.395260    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/832a208d-8b57-444a-9ee2-fe30d01e1525-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/832a208d-8b57-444a-9ee2-fe30d01e1525-credential-socket\") pod \"832a208d-8b57-444a-9ee2-fe30d01e1525\" (UID: \"832a208d-8b57-444a-9ee2-fe30d01e1525\") "
Oct 31 01:07:46.395576 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:46.395442    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/832a208d-8b57-444a-9ee2-fe30d01e1525-istio-data" pod "832a208d-8b57-444a-9ee2-fe30d01e1525" (UID: "832a208d-8b57-444a-9ee2-fe30d01e1525"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:07:46.395576 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:46.395531    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/832a208d-8b57-444a-9ee2-fe30d01e1525-workload-socket" pod "832a208d-8b57-444a-9ee2-fe30d01e1525" (UID: "832a208d-8b57-444a-9ee2-fe30d01e1525"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:07:46.395680 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:46.395632    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/832a208d-8b57-444a-9ee2-fe30d01e1525-istiod-ca-cert" pod "832a208d-8b57-444a-9ee2-fe30d01e1525" (UID: "832a208d-8b57-444a-9ee2-fe30d01e1525"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 01:07:46.395939 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:46.395913    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/832a208d-8b57-444a-9ee2-fe30d01e1525-credential-socket" pod "832a208d-8b57-444a-9ee2-fe30d01e1525" (UID: "832a208d-8b57-444a-9ee2-fe30d01e1525"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:07:46.395939 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:46.395926    4284 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/832a208d-8b57-444a-9ee2-fe30d01e1525-workload-certs\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:07:46.396091 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:46.395962    4284 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/832a208d-8b57-444a-9ee2-fe30d01e1525-workload-socket\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:07:46.396091 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:46.395981    4284 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/832a208d-8b57-444a-9ee2-fe30d01e1525-istiod-ca-cert\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:07:46.396091 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:46.395995    4284 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/832a208d-8b57-444a-9ee2-fe30d01e1525-istio-data\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:07:46.397426 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:46.397398    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/832a208d-8b57-444a-9ee2-fe30d01e1525-istio-token" pod "832a208d-8b57-444a-9ee2-fe30d01e1525" (UID: "832a208d-8b57-444a-9ee2-fe30d01e1525"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:07:46.397756 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:46.397730    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/832a208d-8b57-444a-9ee2-fe30d01e1525-istio-podinfo" pod "832a208d-8b57-444a-9ee2-fe30d01e1525" (UID: "832a208d-8b57-444a-9ee2-fe30d01e1525"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 01:07:46.397860 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:46.397827    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/832a208d-8b57-444a-9ee2-fe30d01e1525-istio-envoy" pod "832a208d-8b57-444a-9ee2-fe30d01e1525" (UID: "832a208d-8b57-444a-9ee2-fe30d01e1525"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:07:46.397860 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:46.397851    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/832a208d-8b57-444a-9ee2-fe30d01e1525-kube-api-access-8pt59" pod "832a208d-8b57-444a-9ee2-fe30d01e1525" (UID: "832a208d-8b57-444a-9ee2-fe30d01e1525"). InnerVolumeSpecName "kube-api-access-8pt59". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:07:46.497382 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:46.497300    4284 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/832a208d-8b57-444a-9ee2-fe30d01e1525-credential-socket\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:07:46.497382 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:46.497342    4284 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/832a208d-8b57-444a-9ee2-fe30d01e1525-istio-token\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:07:46.497382 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:46.497357    4284 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8pt59\" (UniqueName: \"kubernetes.io/projected/832a208d-8b57-444a-9ee2-fe30d01e1525-kube-api-access-8pt59\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:07:46.497382 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:46.497374    4284 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/832a208d-8b57-444a-9ee2-fe30d01e1525-istio-podinfo\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:07:46.497665 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:46.497389    4284 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/832a208d-8b57-444a-9ee2-fe30d01e1525-istio-envoy\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:07:46.835025 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:46.834987    4284 generic.go:356] "Generic (PLEG): container finished" podID="832a208d-8b57-444a-9ee2-fe30d01e1525" containerID="ae92d5f931e923073a29138a26dd8255d4b4e43aff31c99e60bcf2a52f8ef39e" exitCode=0
Oct 31 01:07:46.835265 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:46.835041    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-b-4ab4aa-openshift-default-799d6fd5-pqg68" event={"ID":"832a208d-8b57-444a-9ee2-fe30d01e1525","Type":"ContainerDied","Data":"ae92d5f931e923073a29138a26dd8255d4b4e43aff31c99e60bcf2a52f8ef39e"}
Oct 31 01:07:46.835265 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:46.835094    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-b-4ab4aa-openshift-default-799d6fd5-pqg68" event={"ID":"832a208d-8b57-444a-9ee2-fe30d01e1525","Type":"ContainerDied","Data":"d192885d624693910fb1280edd90ffb1428a8584908fd8ca6da63d43bc9017a5"}
Oct 31 01:07:46.835265 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:46.835123    4284 scope.go:122] "RemoveContainer" containerID="ae92d5f931e923073a29138a26dd8255d4b4e43aff31c99e60bcf2a52f8ef39e"
Oct 31 01:07:46.835494 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:46.835320    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-praxis-b-4ab4aa-openshift-default-799d6fd5-pqg68"
Oct 31 01:07:46.851880 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:46.851856    4284 scope.go:122] "RemoveContainer" containerID="5aac0cb912889747d7872fb30066bf839447f7474f3ed90c219f6b2730e0abe1"
Oct 31 01:07:46.862370 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:46.862344    4284 scope.go:122] "RemoveContainer" containerID="ae92d5f931e923073a29138a26dd8255d4b4e43aff31c99e60bcf2a52f8ef39e"
Oct 31 01:07:46.862660 ip-10-0-132-173 kubenswrapper[4284]: E1031 01:07:46.862643    4284 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ae92d5f931e923073a29138a26dd8255d4b4e43aff31c99e60bcf2a52f8ef39e\": container with ID starting with ae92d5f931e923073a29138a26dd8255d4b4e43aff31c99e60bcf2a52f8ef39e not found: ID does not exist" containerID="ae92d5f931e923073a29138a26dd8255d4b4e43aff31c99e60bcf2a52f8ef39e"
Oct 31 01:07:46.862737 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:46.862670    4284 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ae92d5f931e923073a29138a26dd8255d4b4e43aff31c99e60bcf2a52f8ef39e"} err="failed to get container status \"ae92d5f931e923073a29138a26dd8255d4b4e43aff31c99e60bcf2a52f8ef39e\": rpc error: code = NotFound desc = could not find container \"ae92d5f931e923073a29138a26dd8255d4b4e43aff31c99e60bcf2a52f8ef39e\": container with ID starting with ae92d5f931e923073a29138a26dd8255d4b4e43aff31c99e60bcf2a52f8ef39e not found: ID does not exist"
Oct 31 01:07:46.862737 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:46.862689    4284 scope.go:122] "RemoveContainer" containerID="5aac0cb912889747d7872fb30066bf839447f7474f3ed90c219f6b2730e0abe1"
Oct 31 01:07:46.863979 ip-10-0-132-173 kubenswrapper[4284]: E1031 01:07:46.863962    4284 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5aac0cb912889747d7872fb30066bf839447f7474f3ed90c219f6b2730e0abe1\": container with ID starting with 5aac0cb912889747d7872fb30066bf839447f7474f3ed90c219f6b2730e0abe1 not found: ID does not exist" containerID="5aac0cb912889747d7872fb30066bf839447f7474f3ed90c219f6b2730e0abe1"
Oct 31 01:07:46.864060 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:46.863985    4284 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5aac0cb912889747d7872fb30066bf839447f7474f3ed90c219f6b2730e0abe1"} err="failed to get container status \"5aac0cb912889747d7872fb30066bf839447f7474f3ed90c219f6b2730e0abe1\": rpc error: code = NotFound desc = could not find container \"5aac0cb912889747d7872fb30066bf839447f7474f3ed90c219f6b2730e0abe1\": container with ID starting with 5aac0cb912889747d7872fb30066bf839447f7474f3ed90c219f6b2730e0abe1 not found: ID does not exist"
Oct 31 01:07:46.998906 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:46.998873    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-praxis-b-4ab4aa-openshift-default-799d6fd5-pqg68"]
Oct 31 01:07:47.014636 ip-10-0-132-173 kubenswrapper[4284]: E1031 01:07:47.014599    4284 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fd8sp" containerName="extractor"
Oct 31 01:07:47.045927 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:47.045896    4284 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-praxis-b-4ab4aa-openshift-default-799d6fd5-pqg68"]
Oct 31 01:07:47.979333 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:47.979184    4284 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="832a208d-8b57-444a-9ee2-fe30d01e1525" path="/var/lib/kubelet/pods/832a208d-8b57-444a-9ee2-fe30d01e1525/volumes"
Oct 31 01:07:48.453847 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:48.453760    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:07:53.200694 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:53.200655    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:07:57.741062 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:57.741027    4284 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-f1d081-5b8f78496b-xz9lw"]
Oct 31 01:07:57.741644 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:57.741628    4284 cpu_manager.go:414] "containerMap: removing container" podUID="832a208d-8b57-444a-9ee2-fe30d01e1525" containerName="istio-proxy"
Oct 31 01:07:57.741694 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:57.741645    4284 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="832a208d-8b57-444a-9ee2-fe30d01e1525" containerName="istio-proxy"
Oct 31 01:07:57.741694 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:57.741676    4284 cpu_manager.go:414] "containerMap: removing container" podUID="1000e907-d373-47f0-ac62-db82b1d816b3" containerName="maas-api"
Oct 31 01:07:57.741694 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:57.741681    4284 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1000e907-d373-47f0-ac62-db82b1d816b3" containerName="maas-api"
Oct 31 01:07:57.741791 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:57.741694    4284 cpu_manager.go:414] "containerMap: removing container" podUID="832a208d-8b57-444a-9ee2-fe30d01e1525" containerName="istio-proxy"
Oct 31 01:07:57.741791 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:57.741700    4284 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="832a208d-8b57-444a-9ee2-fe30d01e1525" containerName="istio-proxy"
Oct 31 01:07:57.741791 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:57.741773    4284 memory_manager.go:365] "RemoveStaleState removing state" podUID="1000e907-d373-47f0-ac62-db82b1d816b3" containerName="maas-api"
Oct 31 01:07:57.741791 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:57.741785    4284 memory_manager.go:365] "RemoveStaleState removing state" podUID="832a208d-8b57-444a-9ee2-fe30d01e1525" containerName="istio-proxy"
Oct 31 01:07:57.746515 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:57.746495    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-f1d081-5b8f78496b-xz9lw"
Oct 31 01:07:57.796691 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:57.796660    4284 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-a-f1d081-serving-cert\""
Oct 31 01:07:57.849159 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:57.849122    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-f1d081-5b8f78496b-xz9lw"]
Oct 31 01:07:57.879076 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:57.879045    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dda5d4ad-a454-43fb-96ff-664f19acc3b4-metrics-certs\") pod \"maas-api-e2e-ipp-a-f1d081-5b8f78496b-xz9lw\" (UID: \"dda5d4ad-a454-43fb-96ff-664f19acc3b4\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-f1d081-5b8f78496b-xz9lw"
Oct 31 01:07:57.879284 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:57.879082    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m76pv\" (UniqueName: \"kubernetes.io/projected/dda5d4ad-a454-43fb-96ff-664f19acc3b4-kube-api-access-m76pv\") pod \"maas-api-e2e-ipp-a-f1d081-5b8f78496b-xz9lw\" (UID: \"dda5d4ad-a454-43fb-96ff-664f19acc3b4\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-f1d081-5b8f78496b-xz9lw"
Oct 31 01:07:57.879284 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:57.879106    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/dda5d4ad-a454-43fb-96ff-664f19acc3b4-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-f1d081-5b8f78496b-xz9lw\" (UID: \"dda5d4ad-a454-43fb-96ff-664f19acc3b4\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-f1d081-5b8f78496b-xz9lw"
Oct 31 01:07:57.879384 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:57.879314    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dda5d4ad-a454-43fb-96ff-664f19acc3b4-maas-api-tls\") pod \"maas-api-e2e-ipp-a-f1d081-5b8f78496b-xz9lw\" (UID: \"dda5d4ad-a454-43fb-96ff-664f19acc3b4\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-f1d081-5b8f78496b-xz9lw"
Oct 31 01:07:57.943876 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:57.943842    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:07:57.992256 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:57.989535    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dda5d4ad-a454-43fb-96ff-664f19acc3b4-metrics-certs\") pod \"maas-api-e2e-ipp-a-f1d081-5b8f78496b-xz9lw\" (UID: \"dda5d4ad-a454-43fb-96ff-664f19acc3b4\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-f1d081-5b8f78496b-xz9lw"
Oct 31 01:07:57.992699 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:57.992576    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m76pv\" (UniqueName: \"kubernetes.io/projected/dda5d4ad-a454-43fb-96ff-664f19acc3b4-kube-api-access-m76pv\") pod \"maas-api-e2e-ipp-a-f1d081-5b8f78496b-xz9lw\" (UID: \"dda5d4ad-a454-43fb-96ff-664f19acc3b4\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-f1d081-5b8f78496b-xz9lw"
Oct 31 01:07:57.993031 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:57.992998    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/dda5d4ad-a454-43fb-96ff-664f19acc3b4-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-f1d081-5b8f78496b-xz9lw\" (UID: \"dda5d4ad-a454-43fb-96ff-664f19acc3b4\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-f1d081-5b8f78496b-xz9lw"
Oct 31 01:07:57.993193 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:57.993164    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dda5d4ad-a454-43fb-96ff-664f19acc3b4-maas-api-tls\") pod \"maas-api-e2e-ipp-a-f1d081-5b8f78496b-xz9lw\" (UID: \"dda5d4ad-a454-43fb-96ff-664f19acc3b4\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-f1d081-5b8f78496b-xz9lw"
Oct 31 01:07:57.993503 ip-10-0-132-173 kubenswrapper[4284]: E1031 01:07:57.993472    4284 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-a-f1d081-serving-cert: secret "maas-api-e2e-ipp-a-f1d081-serving-cert" not found
Oct 31 01:07:57.993630 ip-10-0-132-173 kubenswrapper[4284]: E1031 01:07:57.993552    4284 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dda5d4ad-a454-43fb-96ff-664f19acc3b4-maas-api-tls podName:dda5d4ad-a454-43fb-96ff-664f19acc3b4 nodeName:}" failed. No retries permitted until 2026-10-31 01:07:58.493531477 +0000 UTC m=+2392.638538385 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/dda5d4ad-a454-43fb-96ff-664f19acc3b4-maas-api-tls") pod "maas-api-e2e-ipp-a-f1d081-5b8f78496b-xz9lw" (UID: "dda5d4ad-a454-43fb-96ff-664f19acc3b4") : secret "maas-api-e2e-ipp-a-f1d081-serving-cert" not found
Oct 31 01:07:57.994279 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:57.994254    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/dda5d4ad-a454-43fb-96ff-664f19acc3b4-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-f1d081-5b8f78496b-xz9lw\" (UID: \"dda5d4ad-a454-43fb-96ff-664f19acc3b4\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-f1d081-5b8f78496b-xz9lw"
Oct 31 01:07:57.995565 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:57.995475    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dda5d4ad-a454-43fb-96ff-664f19acc3b4-metrics-certs\") pod \"maas-api-e2e-ipp-a-f1d081-5b8f78496b-xz9lw\" (UID: \"dda5d4ad-a454-43fb-96ff-664f19acc3b4\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-f1d081-5b8f78496b-xz9lw"
Oct 31 01:07:58.052284 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:58.052250    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m76pv\" (UniqueName: \"kubernetes.io/projected/dda5d4ad-a454-43fb-96ff-664f19acc3b4-kube-api-access-m76pv\") pod \"maas-api-e2e-ipp-a-f1d081-5b8f78496b-xz9lw\" (UID: \"dda5d4ad-a454-43fb-96ff-664f19acc3b4\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-f1d081-5b8f78496b-xz9lw"
Oct 31 01:07:58.142546 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:58.142505    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:07:58.307035 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:58.306950    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:07:58.499443 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:58.499411    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dda5d4ad-a454-43fb-96ff-664f19acc3b4-maas-api-tls\") pod \"maas-api-e2e-ipp-a-f1d081-5b8f78496b-xz9lw\" (UID: \"dda5d4ad-a454-43fb-96ff-664f19acc3b4\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-f1d081-5b8f78496b-xz9lw"
Oct 31 01:07:58.501934 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:58.501903    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dda5d4ad-a454-43fb-96ff-664f19acc3b4-maas-api-tls\") pod \"maas-api-e2e-ipp-a-f1d081-5b8f78496b-xz9lw\" (UID: \"dda5d4ad-a454-43fb-96ff-664f19acc3b4\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-f1d081-5b8f78496b-xz9lw"
Oct 31 01:07:58.664418 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:58.664383    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-f1d081-5b8f78496b-xz9lw"
Oct 31 01:07:59.040087 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:59.040048    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-f1d081-5b8f78496b-xz9lw"]
Oct 31 01:07:59.041131 ip-10-0-132-173 kubenswrapper[4284]: W1031 01:07:59.041106    4284 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddda5d4ad_a454_43fb_96ff_664f19acc3b4.slice/crio-0eab682be83e63c0c5a61dbacc6d581ba0fe586ddf4251eadcb3adbddd11b490 WatchSource:0}: Error finding container 0eab682be83e63c0c5a61dbacc6d581ba0fe586ddf4251eadcb3adbddd11b490: Status 404 returned error can't find the container with id 0eab682be83e63c0c5a61dbacc6d581ba0fe586ddf4251eadcb3adbddd11b490
Oct 31 01:07:59.042646 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:59.042628    4284 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 31 01:07:59.449746 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:59.449716    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:07:59.918897 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:07:59.918865    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-f1d081-5b8f78496b-xz9lw" event={"ID":"dda5d4ad-a454-43fb-96ff-664f19acc3b4","Type":"ContainerStarted","Data":"0eab682be83e63c0c5a61dbacc6d581ba0fe586ddf4251eadcb3adbddd11b490"}
Oct 31 01:08:00.926197 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:08:00.926165    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-f1d081-5b8f78496b-xz9lw" event={"ID":"dda5d4ad-a454-43fb-96ff-664f19acc3b4","Type":"ContainerStarted","Data":"797debadf5ad6801c24fdc7dda62a61e235faf745451f7499c35a815f593a599"}
Oct 31 01:08:00.926655 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:08:00.926257    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-f1d081-5b8f78496b-xz9lw"
Oct 31 01:08:01.041871 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:08:01.041820    4284 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-f1d081-5b8f78496b-xz9lw" podStartSLOduration=3.117376709 podStartE2EDuration="4.041800877s" totalImagesPullingTime="924.424168ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 01:07:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 01:08:01.039158681 +0000 UTC m=+2395.184165600" watchObservedRunningTime="2026-10-31 01:08:01.041800877 +0000 UTC m=+2395.186807840"
Oct 31 01:08:02.330397 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:08:02.330360    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:08:03.273818 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:08:03.273782    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:08:06.939127 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:08:06.939099    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-f1d081-5b8f78496b-xz9lw"
Oct 31 01:08:08.726980 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:08:08.726927    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:08:13.872679 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:08:13.872638    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:08:14.053765 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:08:14.053731    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-praxis-a-add5bf-564b7f99c5-dhn9p"]
Oct 31 01:08:14.416013 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:08:14.415969    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:08:14.574725 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:08:14.574153    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:08:17.015971 ip-10-0-132-173 kubenswrapper[4284]: E1031 01:08:17.015935    4284 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fd8sp" containerName="extractor"
Oct 31 01:08:18.488763 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:08:18.488726    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:08:19.054645 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:08:19.054607    4284 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-praxis-a-add5bf-564b7f99c5-dhn9p" podUID="5f50d8f5-28a3-4289-a54d-79aa99424929" containerName="payload-processing" containerID="cri-o://92a3f7096b4983e643717cf0924e49173f48b7052411ad9cbfbc5285e44e0192" gracePeriod=25
Oct 31 01:08:19.319492 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:08:19.319428    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-praxis-a-add5bf-564b7f99c5-dhn9p"
Oct 31 01:08:19.422331 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:08:19.421367    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/5f50d8f5-28a3-4289-a54d-79aa99424929-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5f50d8f5-28a3-4289-a54d-79aa99424929-plugins-config-volume\") pod \"5f50d8f5-28a3-4289-a54d-79aa99424929\" (UID: \"5f50d8f5-28a3-4289-a54d-79aa99424929\") "
Oct 31 01:08:19.422331 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:08:19.421664    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5f50d8f5-28a3-4289-a54d-79aa99424929-kube-api-access-v5g7g\" (UniqueName: \"kubernetes.io/projected/5f50d8f5-28a3-4289-a54d-79aa99424929-kube-api-access-v5g7g\") pod \"5f50d8f5-28a3-4289-a54d-79aa99424929\" (UID: \"5f50d8f5-28a3-4289-a54d-79aa99424929\") "
Oct 31 01:08:19.422736 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:08:19.422708    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5f50d8f5-28a3-4289-a54d-79aa99424929-plugins-config-volume" pod "5f50d8f5-28a3-4289-a54d-79aa99424929" (UID: "5f50d8f5-28a3-4289-a54d-79aa99424929"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 01:08:19.426419 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:08:19.426384    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5f50d8f5-28a3-4289-a54d-79aa99424929-kube-api-access-v5g7g" pod "5f50d8f5-28a3-4289-a54d-79aa99424929" (UID: "5f50d8f5-28a3-4289-a54d-79aa99424929"). InnerVolumeSpecName "kube-api-access-v5g7g". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:08:19.523141 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:08:19.523104    4284 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v5g7g\" (UniqueName: \"kubernetes.io/projected/5f50d8f5-28a3-4289-a54d-79aa99424929-kube-api-access-v5g7g\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:08:19.523141 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:08:19.523140    4284 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5f50d8f5-28a3-4289-a54d-79aa99424929-plugins-config-volume\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:08:20.049439 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:08:20.049405    4284 generic.go:356] "Generic (PLEG): container finished" podID="5f50d8f5-28a3-4289-a54d-79aa99424929" containerID="92a3f7096b4983e643717cf0924e49173f48b7052411ad9cbfbc5285e44e0192" exitCode=0
Oct 31 01:08:20.049637 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:08:20.049534    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-praxis-a-add5bf-564b7f99c5-dhn9p"
Oct 31 01:08:20.049637 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:08:20.049537    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-praxis-a-add5bf-564b7f99c5-dhn9p" event={"ID":"5f50d8f5-28a3-4289-a54d-79aa99424929","Type":"ContainerDied","Data":"92a3f7096b4983e643717cf0924e49173f48b7052411ad9cbfbc5285e44e0192"}
Oct 31 01:08:20.049637 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:08:20.049600    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-praxis-a-add5bf-564b7f99c5-dhn9p" event={"ID":"5f50d8f5-28a3-4289-a54d-79aa99424929","Type":"ContainerDied","Data":"d4c964ebd8700423fe4baa39eb451abd939af5cf8d5a96ad22d562787b4c6ede"}
Oct 31 01:08:20.049637 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:08:20.049631    4284 scope.go:122] "RemoveContainer" containerID="92a3f7096b4983e643717cf0924e49173f48b7052411ad9cbfbc5285e44e0192"
Oct 31 01:08:20.059533 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:08:20.059410    4284 scope.go:122] "RemoveContainer" containerID="92a3f7096b4983e643717cf0924e49173f48b7052411ad9cbfbc5285e44e0192"
Oct 31 01:08:20.059796 ip-10-0-132-173 kubenswrapper[4284]: E1031 01:08:20.059776    4284 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"92a3f7096b4983e643717cf0924e49173f48b7052411ad9cbfbc5285e44e0192\": container with ID starting with 92a3f7096b4983e643717cf0924e49173f48b7052411ad9cbfbc5285e44e0192 not found: ID does not exist" containerID="92a3f7096b4983e643717cf0924e49173f48b7052411ad9cbfbc5285e44e0192"
Oct 31 01:08:20.059854 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:08:20.059805    4284 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"92a3f7096b4983e643717cf0924e49173f48b7052411ad9cbfbc5285e44e0192"} err="failed to get container status \"92a3f7096b4983e643717cf0924e49173f48b7052411ad9cbfbc5285e44e0192\": rpc error: code = NotFound desc = could not find container \"92a3f7096b4983e643717cf0924e49173f48b7052411ad9cbfbc5285e44e0192\": container with ID starting with 92a3f7096b4983e643717cf0924e49173f48b7052411ad9cbfbc5285e44e0192 not found: ID does not exist"
Oct 31 01:08:20.107531 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:08:20.107494    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:08:20.208439 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:08:20.208316    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-praxis-a-add5bf-564b7f99c5-dhn9p"]
Oct 31 01:08:20.256207 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:08:20.256115    4284 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-praxis-a-add5bf-564b7f99c5-dhn9p"]
Oct 31 01:08:20.367926 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:08:20.367878    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:08:20.511641 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:08:20.511559    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:08:21.108882 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:08:21.108849    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:08:21.963025 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:08:21.962994    4284 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5f50d8f5-28a3-4289-a54d-79aa99424929" path="/var/lib/kubelet/pods/5f50d8f5-28a3-4289-a54d-79aa99424929/volumes"
Oct 31 01:08:25.427772 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:08:25.427726    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:08:26.065479 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:08:26.065443    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-praxis-a-8c08e7/praxis-route-8c08e7-kserve-7966879b6c-bpw2m"]
Oct 31 01:08:26.209919 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:08:26.209889    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:08:29.040361 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:08:29.040326    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:08:29.507789 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:08:29.507750    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-a-add5bf-5f65cc9f84-2cwpd"]
Oct 31 01:08:29.507978 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:08:29.507945    4284 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-add5bf-5f65cc9f84-2cwpd" podUID="1761233b-1619-498b-a762-edbbcb0e967a" containerName="maas-api" containerID="cri-o://160379e20885f721958b10bf6e5e06891a75930b483698bca4b1717354333bb5" gracePeriod=30
Oct 31 01:08:29.606387 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:08:29.606342    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:08:29.806346 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:08:29.806318    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-add5bf-5f65cc9f84-2cwpd"
Oct 31 01:08:29.945067 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:08:29.945004    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1761233b-1619-498b-a762-edbbcb0e967a-kube-api-access-2gq5c\" (UniqueName: \"kubernetes.io/projected/1761233b-1619-498b-a762-edbbcb0e967a-kube-api-access-2gq5c\") pod \"1761233b-1619-498b-a762-edbbcb0e967a\" (UID: \"1761233b-1619-498b-a762-edbbcb0e967a\") "
Oct 31 01:08:29.945290 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:08:29.945098    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1761233b-1619-498b-a762-edbbcb0e967a-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1761233b-1619-498b-a762-edbbcb0e967a-openshift-service-ca\") pod \"1761233b-1619-498b-a762-edbbcb0e967a\" (UID: \"1761233b-1619-498b-a762-edbbcb0e967a\") "
Oct 31 01:08:29.945290 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:08:29.945275    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/1761233b-1619-498b-a762-edbbcb0e967a-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1761233b-1619-498b-a762-edbbcb0e967a-maas-api-tls\") pod \"1761233b-1619-498b-a762-edbbcb0e967a\" (UID: \"1761233b-1619-498b-a762-edbbcb0e967a\") "
Oct 31 01:08:29.945423 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:08:29.945308    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/1761233b-1619-498b-a762-edbbcb0e967a-metrics-certs\" (UniqueName: \"kubernetes.io/secret/1761233b-1619-498b-a762-edbbcb0e967a-metrics-certs\") pod \"1761233b-1619-498b-a762-edbbcb0e967a\" (UID: \"1761233b-1619-498b-a762-edbbcb0e967a\") "
Oct 31 01:08:29.949710 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:08:29.949662    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1761233b-1619-498b-a762-edbbcb0e967a-openshift-service-ca" pod "1761233b-1619-498b-a762-edbbcb0e967a" (UID: "1761233b-1619-498b-a762-edbbcb0e967a"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 01:08:29.953456 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:08:29.953419    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1761233b-1619-498b-a762-edbbcb0e967a-kube-api-access-2gq5c" pod "1761233b-1619-498b-a762-edbbcb0e967a" (UID: "1761233b-1619-498b-a762-edbbcb0e967a"). InnerVolumeSpecName "kube-api-access-2gq5c". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:08:29.953649 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:08:29.953626    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1761233b-1619-498b-a762-edbbcb0e967a-maas-api-tls" pod "1761233b-1619-498b-a762-edbbcb0e967a" (UID: "1761233b-1619-498b-a762-edbbcb0e967a"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 01:08:29.955413 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:08:29.955340    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1761233b-1619-498b-a762-edbbcb0e967a-metrics-certs" pod "1761233b-1619-498b-a762-edbbcb0e967a" (UID: "1761233b-1619-498b-a762-edbbcb0e967a"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 01:08:30.068700 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:08:30.068170    4284 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1761233b-1619-498b-a762-edbbcb0e967a-openshift-service-ca\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:08:30.068700 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:08:30.068237    4284 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1761233b-1619-498b-a762-edbbcb0e967a-maas-api-tls\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:08:30.068700 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:08:30.068253    4284 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1761233b-1619-498b-a762-edbbcb0e967a-metrics-certs\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:08:30.068700 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:08:30.068269    4284 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2gq5c\" (UniqueName: \"kubernetes.io/projected/1761233b-1619-498b-a762-edbbcb0e967a-kube-api-access-2gq5c\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:08:30.109461 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:08:30.109426    4284 generic.go:356] "Generic (PLEG): container finished" podID="1761233b-1619-498b-a762-edbbcb0e967a" containerID="160379e20885f721958b10bf6e5e06891a75930b483698bca4b1717354333bb5" exitCode=0
Oct 31 01:08:30.109647 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:08:30.109474    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-add5bf-5f65cc9f84-2cwpd"
Oct 31 01:08:30.109647 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:08:30.109493    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-add5bf-5f65cc9f84-2cwpd" event={"ID":"1761233b-1619-498b-a762-edbbcb0e967a","Type":"ContainerDied","Data":"160379e20885f721958b10bf6e5e06891a75930b483698bca4b1717354333bb5"}
Oct 31 01:08:30.109647 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:08:30.109535    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-add5bf-5f65cc9f84-2cwpd" event={"ID":"1761233b-1619-498b-a762-edbbcb0e967a","Type":"ContainerDied","Data":"60339042f50779e0c5ca81573a9b30a39bc01a56af23f658439f53fa0b167be3"}
Oct 31 01:08:30.109647 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:08:30.109555    4284 scope.go:122] "RemoveContainer" containerID="160379e20885f721958b10bf6e5e06891a75930b483698bca4b1717354333bb5"
Oct 31 01:08:30.119297 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:08:30.119277    4284 scope.go:122] "RemoveContainer" containerID="160379e20885f721958b10bf6e5e06891a75930b483698bca4b1717354333bb5"
Oct 31 01:08:30.119585 ip-10-0-132-173 kubenswrapper[4284]: E1031 01:08:30.119568    4284 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"160379e20885f721958b10bf6e5e06891a75930b483698bca4b1717354333bb5\": container with ID starting with 160379e20885f721958b10bf6e5e06891a75930b483698bca4b1717354333bb5 not found: ID does not exist" containerID="160379e20885f721958b10bf6e5e06891a75930b483698bca4b1717354333bb5"
Oct 31 01:08:30.119665 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:08:30.119594    4284 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"160379e20885f721958b10bf6e5e06891a75930b483698bca4b1717354333bb5"} err="failed to get container status \"160379e20885f721958b10bf6e5e06891a75930b483698bca4b1717354333bb5\": rpc error: code = NotFound desc = could not find container \"160379e20885f721958b10bf6e5e06891a75930b483698bca4b1717354333bb5\": container with ID starting with 160379e20885f721958b10bf6e5e06891a75930b483698bca4b1717354333bb5 not found: ID does not exist"
Oct 31 01:08:30.267892 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:08:30.267855    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-a-add5bf-5f65cc9f84-2cwpd"]
Oct 31 01:08:30.272816 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:08:30.272789    4284 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-a-add5bf-5f65cc9f84-2cwpd"]
Oct 31 01:08:31.105711 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:08:31.105677    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:08:31.309886 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:08:31.309846    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:08:31.472238 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:08:31.465099    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:08:31.962836 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:08:31.962796    4284 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1761233b-1619-498b-a762-edbbcb0e967a" path="/var/lib/kubelet/pods/1761233b-1619-498b-a762-edbbcb0e967a/volumes"
Oct 31 01:08:32.045707 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:08:32.045136    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:08:33.060763 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:08:33.060725    4284 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-7c849"]
Oct 31 01:08:33.061312 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:08:33.061297    4284 cpu_manager.go:414] "containerMap: removing container" podUID="1761233b-1619-498b-a762-edbbcb0e967a" containerName="maas-api"
Oct 31 01:08:33.061371 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:08:33.061313    4284 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1761233b-1619-498b-a762-edbbcb0e967a" containerName="maas-api"
Oct 31 01:08:33.061371 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:08:33.061336    4284 cpu_manager.go:414] "containerMap: removing container" podUID="5f50d8f5-28a3-4289-a54d-79aa99424929" containerName="payload-processing"
Oct 31 01:08:33.061371 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:08:33.061341    4284 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5f50d8f5-28a3-4289-a54d-79aa99424929" containerName="payload-processing"
Oct 31 01:08:33.061476 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:08:33.061431    4284 memory_manager.go:365] "RemoveStaleState removing state" podUID="832a208d-8b57-444a-9ee2-fe30d01e1525" containerName="istio-proxy"
Oct 31 01:08:33.061476 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:08:33.061440    4284 memory_manager.go:365] "RemoveStaleState removing state" podUID="5f50d8f5-28a3-4289-a54d-79aa99424929" containerName="payload-processing"
Oct 31 01:08:33.061476 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:08:33.061449    4284 memory_manager.go:365] "RemoveStaleState removing state" podUID="1761233b-1619-498b-a762-edbbcb0e967a" containerName="maas-api"
Oct 31 01:08:33.064930 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:08:33.064913    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-7c849"
Oct 31 01:08:33.115261 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:08:33.115229    4284 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w4-ef2572-openshift-default-dockercfg-fvb72\""
Oct 31 01:08:33.118728 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:08:33.118704    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/28346ae6-eb40-4e6f-a0b2-71fe57c02119-istio-data\") pod \"e2e-worker-w4-ef2572-openshift-default-7649dd87-7c849\" (UID: \"28346ae6-eb40-4e6f-a0b2-71fe57c02119\") " pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-7c849"
Oct 31 01:08:33.118830 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:08:33.118769    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/28346ae6-eb40-4e6f-a0b2-71fe57c02119-workload-socket\") pod \"e2e-worker-w4-ef2572-openshift-default-7649dd87-7c849\" (UID: \"28346ae6-eb40-4e6f-a0b2-71fe57c02119\") " pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-7c849"
Oct 31 01:08:33.118909 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:08:33.118887    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/28346ae6-eb40-4e6f-a0b2-71fe57c02119-istio-token\") pod \"e2e-worker-w4-ef2572-openshift-default-7649dd87-7c849\" (UID: \"28346ae6-eb40-4e6f-a0b2-71fe57c02119\") " pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-7c849"
Oct 31 01:08:33.118974 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:08:33.118935    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/28346ae6-eb40-4e6f-a0b2-71fe57c02119-istiod-ca-cert\") pod \"e2e-worker-w4-ef2572-openshift-default-7649dd87-7c849\" (UID: \"28346ae6-eb40-4e6f-a0b2-71fe57c02119\") " pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-7c849"
Oct 31 01:08:33.119041 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:08:33.119029    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cczxw\" (UniqueName: \"kubernetes.io/projected/28346ae6-eb40-4e6f-a0b2-71fe57c02119-kube-api-access-cczxw\") pod \"e2e-worker-w4-ef2572-openshift-default-7649dd87-7c849\" (UID: \"28346ae6-eb40-4e6f-a0b2-71fe57c02119\") " pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-7c849"
Oct 31 01:08:33.119101 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:08:33.119059    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/28346ae6-eb40-4e6f-a0b2-71fe57c02119-workload-certs\") pod \"e2e-worker-w4-ef2572-openshift-default-7649dd87-7c849\" (UID: \"28346ae6-eb40-4e6f-a0b2-71fe57c02119\") " pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-7c849"
Oct 31 01:08:33.119193 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:08:33.119172    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/28346ae6-eb40-4e6f-a0b2-71fe57c02119-istio-podinfo\") pod \"e2e-worker-w4-ef2572-openshift-default-7649dd87-7c849\" (UID: \"28346ae6-eb40-4e6f-a0b2-71fe57c02119\") " pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-7c849"
Oct 31 01:08:33.119270 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:08:33.119246    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/28346ae6-eb40-4e6f-a0b2-71fe57c02119-credential-socket\") pod \"e2e-worker-w4-ef2572-openshift-default-7649dd87-7c849\" (UID: \"28346ae6-eb40-4e6f-a0b2-71fe57c02119\") " pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-7c849"
Oct 31 01:08:33.119309 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:08:33.119290    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/28346ae6-eb40-4e6f-a0b2-71fe57c02119-istio-envoy\") pod \"e2e-worker-w4-ef2572-openshift-default-7649dd87-7c849\" (UID: \"28346ae6-eb40-4e6f-a0b2-71fe57c02119\") " pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-7c849"
Oct 31 01:08:33.166643 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:08:33.166608    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-7c849"]
Oct 31 01:08:33.220784 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:08:33.220748    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/28346ae6-eb40-4e6f-a0b2-71fe57c02119-istio-token\") pod \"e2e-worker-w4-ef2572-openshift-default-7649dd87-7c849\" (UID: \"28346ae6-eb40-4e6f-a0b2-71fe57c02119\") " pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-7c849"
Oct 31 01:08:33.220996 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:08:33.220794    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/28346ae6-eb40-4e6f-a0b2-71fe57c02119-istiod-ca-cert\") pod \"e2e-worker-w4-ef2572-openshift-default-7649dd87-7c849\" (UID: \"28346ae6-eb40-4e6f-a0b2-71fe57c02119\") " pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-7c849"
Oct 31 01:08:33.220996 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:08:33.220825    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cczxw\" (UniqueName: \"kubernetes.io/projected/28346ae6-eb40-4e6f-a0b2-71fe57c02119-kube-api-access-cczxw\") pod \"e2e-worker-w4-ef2572-openshift-default-7649dd87-7c849\" (UID: \"28346ae6-eb40-4e6f-a0b2-71fe57c02119\") " pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-7c849"
Oct 31 01:08:33.220996 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:08:33.220848    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/28346ae6-eb40-4e6f-a0b2-71fe57c02119-workload-certs\") pod \"e2e-worker-w4-ef2572-openshift-default-7649dd87-7c849\" (UID: \"28346ae6-eb40-4e6f-a0b2-71fe57c02119\") " pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-7c849"
Oct 31 01:08:33.220996 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:08:33.220922    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/28346ae6-eb40-4e6f-a0b2-71fe57c02119-istio-podinfo\") pod \"e2e-worker-w4-ef2572-openshift-default-7649dd87-7c849\" (UID: \"28346ae6-eb40-4e6f-a0b2-71fe57c02119\") " pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-7c849"
Oct 31 01:08:33.220996 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:08:33.220975    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/28346ae6-eb40-4e6f-a0b2-71fe57c02119-credential-socket\") pod \"e2e-worker-w4-ef2572-openshift-default-7649dd87-7c849\" (UID: \"28346ae6-eb40-4e6f-a0b2-71fe57c02119\") " pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-7c849"
Oct 31 01:08:33.221329 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:08:33.221028    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/28346ae6-eb40-4e6f-a0b2-71fe57c02119-istio-envoy\") pod \"e2e-worker-w4-ef2572-openshift-default-7649dd87-7c849\" (UID: \"28346ae6-eb40-4e6f-a0b2-71fe57c02119\") " pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-7c849"
Oct 31 01:08:33.221329 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:08:33.221107    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/28346ae6-eb40-4e6f-a0b2-71fe57c02119-istio-data\") pod \"e2e-worker-w4-ef2572-openshift-default-7649dd87-7c849\" (UID: \"28346ae6-eb40-4e6f-a0b2-71fe57c02119\") " pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-7c849"
Oct 31 01:08:33.221329 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:08:33.221201    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/28346ae6-eb40-4e6f-a0b2-71fe57c02119-workload-socket\") pod \"e2e-worker-w4-ef2572-openshift-default-7649dd87-7c849\" (UID: \"28346ae6-eb40-4e6f-a0b2-71fe57c02119\") " pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-7c849"
Oct 31 01:08:33.221713 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:08:33.221685    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/28346ae6-eb40-4e6f-a0b2-71fe57c02119-istio-data\") pod \"e2e-worker-w4-ef2572-openshift-default-7649dd87-7c849\" (UID: \"28346ae6-eb40-4e6f-a0b2-71fe57c02119\") " pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-7c849"
Oct 31 01:08:33.221830 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:08:33.221764    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/28346ae6-eb40-4e6f-a0b2-71fe57c02119-workload-socket\") pod \"e2e-worker-w4-ef2572-openshift-default-7649dd87-7c849\" (UID: \"28346ae6-eb40-4e6f-a0b2-71fe57c02119\") " pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-7c849"
Oct 31 01:08:33.222058 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:08:33.222029    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/28346ae6-eb40-4e6f-a0b2-71fe57c02119-credential-socket\") pod \"e2e-worker-w4-ef2572-openshift-default-7649dd87-7c849\" (UID: \"28346ae6-eb40-4e6f-a0b2-71fe57c02119\") " pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-7c849"
Oct 31 01:08:33.222172 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:08:33.222154    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/28346ae6-eb40-4e6f-a0b2-71fe57c02119-workload-certs\") pod \"e2e-worker-w4-ef2572-openshift-default-7649dd87-7c849\" (UID: \"28346ae6-eb40-4e6f-a0b2-71fe57c02119\") " pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-7c849"
Oct 31 01:08:33.222339 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:08:33.222317    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/28346ae6-eb40-4e6f-a0b2-71fe57c02119-istiod-ca-cert\") pod \"e2e-worker-w4-ef2572-openshift-default-7649dd87-7c849\" (UID: \"28346ae6-eb40-4e6f-a0b2-71fe57c02119\") " pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-7c849"
Oct 31 01:08:33.224954 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:08:33.224930    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/28346ae6-eb40-4e6f-a0b2-71fe57c02119-istio-podinfo\") pod \"e2e-worker-w4-ef2572-openshift-default-7649dd87-7c849\" (UID: \"28346ae6-eb40-4e6f-a0b2-71fe57c02119\") " pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-7c849"
Oct 31 01:08:33.226169 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:08:33.226145    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/28346ae6-eb40-4e6f-a0b2-71fe57c02119-istio-envoy\") pod \"e2e-worker-w4-ef2572-openshift-default-7649dd87-7c849\" (UID: \"28346ae6-eb40-4e6f-a0b2-71fe57c02119\") " pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-7c849"
Oct 31 01:08:33.312331 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:08:33.312248    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/28346ae6-eb40-4e6f-a0b2-71fe57c02119-istio-token\") pod \"e2e-worker-w4-ef2572-openshift-default-7649dd87-7c849\" (UID: \"28346ae6-eb40-4e6f-a0b2-71fe57c02119\") " pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-7c849"
Oct 31 01:08:33.312514 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:08:33.312491    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cczxw\" (UniqueName: \"kubernetes.io/projected/28346ae6-eb40-4e6f-a0b2-71fe57c02119-kube-api-access-cczxw\") pod \"e2e-worker-w4-ef2572-openshift-default-7649dd87-7c849\" (UID: \"28346ae6-eb40-4e6f-a0b2-71fe57c02119\") " pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-7c849"
Oct 31 01:08:33.378416 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:08:33.378377    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-7c849"
Oct 31 01:08:33.828450 ip-10-0-132-173 kubenswrapper[4284]: W1031 01:08:33.828413    4284 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod28346ae6_eb40_4e6f_a0b2_71fe57c02119.slice/crio-d95e129d0e4544e645ebf7358ebcdc2beac50b2b278d5f6c6221648e927a95cf WatchSource:0}: Error finding container d95e129d0e4544e645ebf7358ebcdc2beac50b2b278d5f6c6221648e927a95cf: Status 404 returned error can't find the container with id d95e129d0e4544e645ebf7358ebcdc2beac50b2b278d5f6c6221648e927a95cf
Oct 31 01:08:33.828450 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:08:33.828438    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-7c849"]
Oct 31 01:08:33.831341 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:08:33.831308    4284 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 01:08:33.831444 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:08:33.831373    4284 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 01:08:33.831444 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:08:33.831405    4284 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 01:08:34.136414 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:08:34.136320    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-7c849" event={"ID":"28346ae6-eb40-4e6f-a0b2-71fe57c02119","Type":"ContainerStarted","Data":"7ae25bd0ad3d46d71424cce5798ae47bda06e3a920c80ba595dd7a6e547bf70b"}
Oct 31 01:08:34.136414 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:08:34.136384    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-7c849" event={"ID":"28346ae6-eb40-4e6f-a0b2-71fe57c02119","Type":"ContainerStarted","Data":"d95e129d0e4544e645ebf7358ebcdc2beac50b2b278d5f6c6221648e927a95cf"}
Oct 31 01:08:34.378888 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:08:34.378856    4284 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-7c849"
Oct 31 01:08:34.380466 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:08:34.380443    4284 patch_prober.go:29] interesting pod/e2e-worker-w4-ef2572-openshift-default-7649dd87-7c849 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.150:15021/healthz/ready\": dial tcp 10.134.0.150:15021: connect: connection refused" start-of-body=
Oct 31 01:08:34.380596 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:08:34.380501    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-7c849" podUID="28346ae6-eb40-4e6f-a0b2-71fe57c02119" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.150:15021/healthz/ready\": dial tcp 10.134.0.150:15021: connect: connection refused"
Oct 31 01:08:35.379823 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:08:35.379785    4284 patch_prober.go:29] interesting pod/e2e-worker-w4-ef2572-openshift-default-7649dd87-7c849 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.150:15021/healthz/ready\": dial tcp 10.134.0.150:15021: connect: connection refused" start-of-body=
Oct 31 01:08:35.380362 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:08:35.379858    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-7c849" podUID="28346ae6-eb40-4e6f-a0b2-71fe57c02119" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.150:15021/healthz/ready\": dial tcp 10.134.0.150:15021: connect: connection refused"
Oct 31 01:08:36.379738 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:08:36.379694    4284 patch_prober.go:29] interesting pod/e2e-worker-w4-ef2572-openshift-default-7649dd87-7c849 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.150:15021/healthz/ready\": dial tcp 10.134.0.150:15021: connect: connection refused" start-of-body=
Oct 31 01:08:36.379936 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:08:36.379791    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-7c849" podUID="28346ae6-eb40-4e6f-a0b2-71fe57c02119" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.150:15021/healthz/ready\": dial tcp 10.134.0.150:15021: connect: connection refused"
Oct 31 01:08:36.710235 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:08:36.708000    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:08:37.379798 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:08:37.379755    4284 patch_prober.go:29] interesting pod/e2e-worker-w4-ef2572-openshift-default-7649dd87-7c849 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.150:15021/healthz/ready\": dial tcp 10.134.0.150:15021: connect: connection refused" start-of-body=
Oct 31 01:08:37.379977 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:08:37.379813    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-7c849" podUID="28346ae6-eb40-4e6f-a0b2-71fe57c02119" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.150:15021/healthz/ready\": dial tcp 10.134.0.150:15021: connect: connection refused"
Oct 31 01:08:38.265385 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:08:38.265331    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:08:38.379457 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:08:38.379406    4284 patch_prober.go:29] interesting pod/e2e-worker-w4-ef2572-openshift-default-7649dd87-7c849 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.150:15021/healthz/ready\": dial tcp 10.134.0.150:15021: connect: connection refused" start-of-body=
Oct 31 01:08:38.379641 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:08:38.379481    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-7c849" podUID="28346ae6-eb40-4e6f-a0b2-71fe57c02119" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.150:15021/healthz/ready\": dial tcp 10.134.0.150:15021: connect: connection refused"
Oct 31 01:08:38.428102 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:08:38.428062    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:08:38.658112 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:08:38.658078    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:08:39.379352 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:08:39.379307    4284 patch_prober.go:29] interesting pod/e2e-worker-w4-ef2572-openshift-default-7649dd87-7c849 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.150:15021/healthz/ready\": dial tcp 10.134.0.150:15021: connect: connection refused" start-of-body=
Oct 31 01:08:39.379570 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:08:39.379375    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-7c849" podUID="28346ae6-eb40-4e6f-a0b2-71fe57c02119" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.150:15021/healthz/ready\": dial tcp 10.134.0.150:15021: connect: connection refused"
Oct 31 01:08:39.550426 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:08:39.550383    4284 patch_prober.go:29] interesting pod/e2e-ipp-a-f1d081-openshift-default-56c48f6487-9v42v container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.145:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 31 01:08:39.551015 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:08:39.550974    4284 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-a-f1d081-openshift-default-56c48f6487-9v42v" podUID="94cb0276-7866-47b5-aa67-1d50bc2250ae" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.145:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 31 01:08:40.379812 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:08:40.379760    4284 patch_prober.go:29] interesting pod/e2e-worker-w4-ef2572-openshift-default-7649dd87-7c849 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.150:15021/healthz/ready\": dial tcp 10.134.0.150:15021: connect: connection refused" start-of-body=
Oct 31 01:08:40.380004 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:08:40.379833    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-7c849" podUID="28346ae6-eb40-4e6f-a0b2-71fe57c02119" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.150:15021/healthz/ready\": dial tcp 10.134.0.150:15021: connect: connection refused"
Oct 31 01:08:41.086760 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:08:41.086709    4284 kuberuntime_container.go:898] "Killing container with a grace period" pod="ai-tenant-e2e-praxis-a-8c08e7/praxis-route-8c08e7-kserve-7966879b6c-bpw2m" podUID="f6f3f062-5d9e-4051-9c1a-9cf20b93ba0a" containerName="main" containerID="cri-o://ce30606a226b452e005599060768efa00a35cbd3f569aff22c4a48db86695359" gracePeriod=45
Oct 31 01:08:41.212092 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:08:41.212052    4284 prober.go:122] "Probe failed" probeType="Readiness" pod="ai-tenant-e2e-praxis-a-8c08e7/praxis-route-8c08e7-kserve-7966879b6c-bpw2m" podUID="f6f3f062-5d9e-4051-9c1a-9cf20b93ba0a" containerName="main" probeResult="failure" output="Get \"https://10.134.0.139:8000/ready\": dial tcp 10.134.0.139:8000: connect: connection refused"
Oct 31 01:08:41.383612 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:08:41.383573    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-praxis-a-8c08e7/praxis-route-8c08e7-kserve-7966879b6c-bpw2m"
Oct 31 01:08:41.383797 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:08:41.383743    4284 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-7c849"
Oct 31 01:08:41.384241 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:08:41.384205    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-7c849"
Oct 31 01:08:41.385166 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:08:41.385145    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-7c849"
Oct 31 01:08:41.467682 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:08:41.467642    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f6f3f062-5d9e-4051-9c1a-9cf20b93ba0a-home\" (UniqueName: \"kubernetes.io/empty-dir/f6f3f062-5d9e-4051-9c1a-9cf20b93ba0a-home\") pod \"f6f3f062-5d9e-4051-9c1a-9cf20b93ba0a\" (UID: \"f6f3f062-5d9e-4051-9c1a-9cf20b93ba0a\") "
Oct 31 01:08:41.467879 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:08:41.467754    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f6f3f062-5d9e-4051-9c1a-9cf20b93ba0a-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f6f3f062-5d9e-4051-9c1a-9cf20b93ba0a-model-cache\") pod \"f6f3f062-5d9e-4051-9c1a-9cf20b93ba0a\" (UID: \"f6f3f062-5d9e-4051-9c1a-9cf20b93ba0a\") "
Oct 31 01:08:41.467879 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:08:41.467807    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f6f3f062-5d9e-4051-9c1a-9cf20b93ba0a-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f6f3f062-5d9e-4051-9c1a-9cf20b93ba0a-tmp-dir\") pod \"f6f3f062-5d9e-4051-9c1a-9cf20b93ba0a\" (UID: \"f6f3f062-5d9e-4051-9c1a-9cf20b93ba0a\") "
Oct 31 01:08:41.467879 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:08:41.467848    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f6f3f062-5d9e-4051-9c1a-9cf20b93ba0a-kube-api-access-qvrb6\" (UniqueName: \"kubernetes.io/projected/f6f3f062-5d9e-4051-9c1a-9cf20b93ba0a-kube-api-access-qvrb6\") pod \"f6f3f062-5d9e-4051-9c1a-9cf20b93ba0a\" (UID: \"f6f3f062-5d9e-4051-9c1a-9cf20b93ba0a\") "
Oct 31 01:08:41.467879 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:08:41.467876    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/f6f3f062-5d9e-4051-9c1a-9cf20b93ba0a-tls-certs\" (UniqueName: \"kubernetes.io/secret/f6f3f062-5d9e-4051-9c1a-9cf20b93ba0a-tls-certs\") pod \"f6f3f062-5d9e-4051-9c1a-9cf20b93ba0a\" (UID: \"f6f3f062-5d9e-4051-9c1a-9cf20b93ba0a\") "
Oct 31 01:08:41.468097 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:08:41.467948    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f6f3f062-5d9e-4051-9c1a-9cf20b93ba0a-dshm\" (UniqueName: \"kubernetes.io/empty-dir/f6f3f062-5d9e-4051-9c1a-9cf20b93ba0a-dshm\") pod \"f6f3f062-5d9e-4051-9c1a-9cf20b93ba0a\" (UID: \"f6f3f062-5d9e-4051-9c1a-9cf20b93ba0a\") "
Oct 31 01:08:41.468097 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:08:41.467965    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f6f3f062-5d9e-4051-9c1a-9cf20b93ba0a-home" pod "f6f3f062-5d9e-4051-9c1a-9cf20b93ba0a" (UID: "f6f3f062-5d9e-4051-9c1a-9cf20b93ba0a"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:08:41.468097 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:08:41.468010    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f6f3f062-5d9e-4051-9c1a-9cf20b93ba0a-model-cache" pod "f6f3f062-5d9e-4051-9c1a-9cf20b93ba0a" (UID: "f6f3f062-5d9e-4051-9c1a-9cf20b93ba0a"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:08:41.468316 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:08:41.468113    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f6f3f062-5d9e-4051-9c1a-9cf20b93ba0a-tmp-dir" pod "f6f3f062-5d9e-4051-9c1a-9cf20b93ba0a" (UID: "f6f3f062-5d9e-4051-9c1a-9cf20b93ba0a"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:08:41.468465 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:08:41.468447    4284 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f6f3f062-5d9e-4051-9c1a-9cf20b93ba0a-home\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:08:41.468555 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:08:41.468471    4284 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f6f3f062-5d9e-4051-9c1a-9cf20b93ba0a-model-cache\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:08:41.468555 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:08:41.468484    4284 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f6f3f062-5d9e-4051-9c1a-9cf20b93ba0a-tmp-dir\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:08:41.470475 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:08:41.470452    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f6f3f062-5d9e-4051-9c1a-9cf20b93ba0a-tls-certs" pod "f6f3f062-5d9e-4051-9c1a-9cf20b93ba0a" (UID: "f6f3f062-5d9e-4051-9c1a-9cf20b93ba0a"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 01:08:41.470475 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:08:41.470461    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f6f3f062-5d9e-4051-9c1a-9cf20b93ba0a-dshm" pod "f6f3f062-5d9e-4051-9c1a-9cf20b93ba0a" (UID: "f6f3f062-5d9e-4051-9c1a-9cf20b93ba0a"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:08:41.470599 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:08:41.470502    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f6f3f062-5d9e-4051-9c1a-9cf20b93ba0a-kube-api-access-qvrb6" pod "f6f3f062-5d9e-4051-9c1a-9cf20b93ba0a" (UID: "f6f3f062-5d9e-4051-9c1a-9cf20b93ba0a"). InnerVolumeSpecName "kube-api-access-qvrb6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:08:41.499131 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:08:41.499079    4284 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-7c849" podStartSLOduration=8.499064381 podStartE2EDuration="8.499064381s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 01:08:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 01:08:34.261709242 +0000 UTC m=+2428.406716176" watchObservedRunningTime="2026-10-31 01:08:41.499064381 +0000 UTC m=+2435.644071296"
Oct 31 01:08:41.569750 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:08:41.569707    4284 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qvrb6\" (UniqueName: \"kubernetes.io/projected/f6f3f062-5d9e-4051-9c1a-9cf20b93ba0a-kube-api-access-qvrb6\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:08:41.569750 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:08:41.569745    4284 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f6f3f062-5d9e-4051-9c1a-9cf20b93ba0a-tls-certs\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:08:41.569936 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:08:41.569762    4284 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f6f3f062-5d9e-4051-9c1a-9cf20b93ba0a-dshm\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:08:41.996657 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:08:41.996617    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:08:42.191252 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:08:42.188995    4284 generic.go:356] "Generic (PLEG): container finished" podID="f6f3f062-5d9e-4051-9c1a-9cf20b93ba0a" containerID="ce30606a226b452e005599060768efa00a35cbd3f569aff22c4a48db86695359" exitCode=0
Oct 31 01:08:42.191252 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:08:42.189041    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-praxis-a-8c08e7/praxis-route-8c08e7-kserve-7966879b6c-bpw2m"
Oct 31 01:08:42.191252 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:08:42.189074    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-praxis-a-8c08e7/praxis-route-8c08e7-kserve-7966879b6c-bpw2m" event={"ID":"f6f3f062-5d9e-4051-9c1a-9cf20b93ba0a","Type":"ContainerDied","Data":"ce30606a226b452e005599060768efa00a35cbd3f569aff22c4a48db86695359"}
Oct 31 01:08:42.191252 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:08:42.189120    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-praxis-a-8c08e7/praxis-route-8c08e7-kserve-7966879b6c-bpw2m" event={"ID":"f6f3f062-5d9e-4051-9c1a-9cf20b93ba0a","Type":"ContainerDied","Data":"f18ff32598c388332c67c4e7e436e0f96477c419b1b244717fd1fe1e6cc10a38"}
Oct 31 01:08:42.191252 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:08:42.189148    4284 scope.go:122] "RemoveContainer" containerID="ce30606a226b452e005599060768efa00a35cbd3f569aff22c4a48db86695359"
Oct 31 01:08:42.221449 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:08:42.199895    4284 scope.go:122] "RemoveContainer" containerID="ce30606a226b452e005599060768efa00a35cbd3f569aff22c4a48db86695359"
Oct 31 01:08:42.221449 ip-10-0-132-173 kubenswrapper[4284]: E1031 01:08:42.200194    4284 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ce30606a226b452e005599060768efa00a35cbd3f569aff22c4a48db86695359\": container with ID starting with ce30606a226b452e005599060768efa00a35cbd3f569aff22c4a48db86695359 not found: ID does not exist" containerID="ce30606a226b452e005599060768efa00a35cbd3f569aff22c4a48db86695359"
Oct 31 01:08:42.221449 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:08:42.200235    4284 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ce30606a226b452e005599060768efa00a35cbd3f569aff22c4a48db86695359"} err="failed to get container status \"ce30606a226b452e005599060768efa00a35cbd3f569aff22c4a48db86695359\": rpc error: code = NotFound desc = could not find container \"ce30606a226b452e005599060768efa00a35cbd3f569aff22c4a48db86695359\": container with ID starting with ce30606a226b452e005599060768efa00a35cbd3f569aff22c4a48db86695359 not found: ID does not exist"
Oct 31 01:08:42.349946 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:08:42.349914    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-praxis-a-8c08e7/praxis-route-8c08e7-kserve-7966879b6c-bpw2m"]
Oct 31 01:08:42.352483 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:08:42.352464    4284 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-praxis-a-8c08e7/praxis-route-8c08e7-kserve-7966879b6c-bpw2m"]
Oct 31 01:08:43.197392 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:08:43.197362    4284 generic.go:356] "Generic (PLEG): container finished" podID="94cb0276-7866-47b5-aa67-1d50bc2250ae" containerID="b2af715ca7ffe62865aece4e7996799523c55d84f84beebc977f60f086a2333a" exitCode=0
Oct 31 01:08:43.197864 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:08:43.197444    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-f1d081-openshift-default-56c48f6487-9v42v" event={"ID":"94cb0276-7866-47b5-aa67-1d50bc2250ae","Type":"ContainerDied","Data":"b2af715ca7ffe62865aece4e7996799523c55d84f84beebc977f60f086a2333a"}
Oct 31 01:08:43.197944 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:08:43.197897    4284 scope.go:122] "RemoveContainer" containerID="b2af715ca7ffe62865aece4e7996799523c55d84f84beebc977f60f086a2333a"
Oct 31 01:08:43.199737 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:08:43.199704    4284 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 01:08:43.199865 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:08:43.199789    4284 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 01:08:43.199865 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:08:43.199838    4284 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 01:08:43.963184 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:08:43.963150    4284 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f6f3f062-5d9e-4051-9c1a-9cf20b93ba0a" path="/var/lib/kubelet/pods/f6f3f062-5d9e-4051-9c1a-9cf20b93ba0a/volumes"
Oct 31 01:08:44.216018 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:08:44.215933    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-f1d081-openshift-default-56c48f6487-9v42v" event={"ID":"94cb0276-7866-47b5-aa67-1d50bc2250ae","Type":"ContainerStarted","Data":"ff6967fc6c9d453f8da6d968a270ae6f08dfb8c3cfb5277b9288e2d345d40bfe"}
Oct 31 01:08:44.910240 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:08:44.910120    4284 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-f1d081-openshift-default-56c48f6487-9v42v"
Oct 31 01:08:44.910468 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:08:44.910442    4284 patch_prober.go:29] interesting pod/e2e-ipp-a-f1d081-openshift-default-56c48f6487-9v42v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.145:15021/healthz/ready\": dial tcp 10.134.0.145:15021: connect: connection refused" start-of-body=
Oct 31 01:08:44.910539 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:08:44.910502    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-f1d081-openshift-default-56c48f6487-9v42v" podUID="94cb0276-7866-47b5-aa67-1d50bc2250ae" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.145:15021/healthz/ready\": dial tcp 10.134.0.145:15021: connect: connection refused"
Oct 31 01:08:45.909718 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:08:45.909676    4284 patch_prober.go:29] interesting pod/e2e-ipp-a-f1d081-openshift-default-56c48f6487-9v42v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.145:15021/healthz/ready\": dial tcp 10.134.0.145:15021: connect: connection refused" start-of-body=
Oct 31 01:08:45.910129 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:08:45.909741    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-f1d081-openshift-default-56c48f6487-9v42v" podUID="94cb0276-7866-47b5-aa67-1d50bc2250ae" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.145:15021/healthz/ready\": dial tcp 10.134.0.145:15021: connect: connection refused"
Oct 31 01:08:46.909632 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:08:46.909589    4284 patch_prober.go:29] interesting pod/e2e-ipp-a-f1d081-openshift-default-56c48f6487-9v42v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.145:15021/healthz/ready\": dial tcp 10.134.0.145:15021: connect: connection refused" start-of-body=
Oct 31 01:08:46.909825 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:08:46.909676    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-f1d081-openshift-default-56c48f6487-9v42v" podUID="94cb0276-7866-47b5-aa67-1d50bc2250ae" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.145:15021/healthz/ready\": dial tcp 10.134.0.145:15021: connect: connection refused"
Oct 31 01:08:47.036061 ip-10-0-132-173 kubenswrapper[4284]: E1031 01:08:47.036021    4284 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fd8sp" containerName="extractor"
Oct 31 01:08:47.404291 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:08:47.403835    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:08:47.909707 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:08:47.909662    4284 patch_prober.go:29] interesting pod/e2e-ipp-a-f1d081-openshift-default-56c48f6487-9v42v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.145:15021/healthz/ready\": dial tcp 10.134.0.145:15021: connect: connection refused" start-of-body=
Oct 31 01:08:47.909906 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:08:47.909738    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-f1d081-openshift-default-56c48f6487-9v42v" podUID="94cb0276-7866-47b5-aa67-1d50bc2250ae" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.145:15021/healthz/ready\": dial tcp 10.134.0.145:15021: connect: connection refused"
Oct 31 01:08:48.909702 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:08:48.909667    4284 patch_prober.go:29] interesting pod/e2e-ipp-a-f1d081-openshift-default-56c48f6487-9v42v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.145:15021/healthz/ready\": dial tcp 10.134.0.145:15021: connect: connection refused" start-of-body=
Oct 31 01:08:48.909962 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:08:48.909938    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-f1d081-openshift-default-56c48f6487-9v42v" podUID="94cb0276-7866-47b5-aa67-1d50bc2250ae" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.145:15021/healthz/ready\": dial tcp 10.134.0.145:15021: connect: connection refused"
Oct 31 01:08:49.914255 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:08:49.914194    4284 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-f1d081-openshift-default-56c48f6487-9v42v"
Oct 31 01:08:49.914640 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:08:49.914505    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-f1d081-openshift-default-56c48f6487-9v42v"
Oct 31 01:08:49.915315 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:08:49.915295    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-f1d081-openshift-default-56c48f6487-9v42v"
Oct 31 01:08:51.347117 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:08:51.347070    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:08:51.555993 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:08:51.555953    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:08:52.533573 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:08:52.533538    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:08:57.624606 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:08:57.624561    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:08:58.082000 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:08:58.081954    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:08:58.224743 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:08:58.224675    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-f1d081-7866949869-8hd89"]
Oct 31 01:08:58.224950 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:08:58.224762    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-f1d081-7b4d567df9-xm4lr"]
Oct 31 01:08:58.497373 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:08:58.497337    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:08:58.726240 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:08:58.717601    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:09:02.597712 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:02.597665    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:09:02.747581 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:02.747541    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:09:03.226003 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:03.225946    4284 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-f1d081-7b4d567df9-xm4lr" podUID="269e4d26-9e30-4ae6-93b7-111f73f64a81" containerName="payload-pre-processing" containerID="cri-o://82389a1fe4ada0a336784248f1633ab7200cea630deef0466cca2a2994b8044d" gracePeriod=25
Oct 31 01:09:03.226367 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:03.225963    4284 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-a-f1d081-7866949869-8hd89" podUID="d200ff24-a6a7-4c25-85a7-184e24963fd2" containerName="payload-processing" containerID="cri-o://8de2fa6e33514da4668b19e7e0e460ade06b9f1e09687e6210be4a72ea2f671d" gracePeriod=25
Oct 31 01:09:03.550189 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:03.550164    4284 generic.go:356] "Generic (PLEG): container finished" podID="d200ff24-a6a7-4c25-85a7-184e24963fd2" containerID="8de2fa6e33514da4668b19e7e0e460ade06b9f1e09687e6210be4a72ea2f671d" exitCode=0
Oct 31 01:09:03.550359 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:03.550262    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-f1d081-7866949869-8hd89" event={"ID":"d200ff24-a6a7-4c25-85a7-184e24963fd2","Type":"ContainerDied","Data":"8de2fa6e33514da4668b19e7e0e460ade06b9f1e09687e6210be4a72ea2f671d"}
Oct 31 01:09:03.550359 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:03.550321    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-f1d081-7866949869-8hd89" event={"ID":"d200ff24-a6a7-4c25-85a7-184e24963fd2","Type":"ContainerDied","Data":"28322daaca2eb1c2a4b135da72b2c92ee9546c246d02612ab626862f78a5c640"}
Oct 31 01:09:03.550359 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:03.550346    4284 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="28322daaca2eb1c2a4b135da72b2c92ee9546c246d02612ab626862f78a5c640"
Oct 31 01:09:03.551973 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:03.551948    4284 generic.go:356] "Generic (PLEG): container finished" podID="269e4d26-9e30-4ae6-93b7-111f73f64a81" containerID="82389a1fe4ada0a336784248f1633ab7200cea630deef0466cca2a2994b8044d" exitCode=0
Oct 31 01:09:03.552110 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:03.552037    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-f1d081-7b4d567df9-xm4lr" event={"ID":"269e4d26-9e30-4ae6-93b7-111f73f64a81","Type":"ContainerDied","Data":"82389a1fe4ada0a336784248f1633ab7200cea630deef0466cca2a2994b8044d"}
Oct 31 01:09:03.560007 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:03.559983    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-f1d081-7866949869-8hd89"
Oct 31 01:09:03.573831 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:03.573798    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:09:03.612700 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:03.612669    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d200ff24-a6a7-4c25-85a7-184e24963fd2-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d200ff24-a6a7-4c25-85a7-184e24963fd2-plugins-config-volume\") pod \"d200ff24-a6a7-4c25-85a7-184e24963fd2\" (UID: \"d200ff24-a6a7-4c25-85a7-184e24963fd2\") "
Oct 31 01:09:03.613051 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:03.612748    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d200ff24-a6a7-4c25-85a7-184e24963fd2-kube-api-access-t67f5\" (UniqueName: \"kubernetes.io/projected/d200ff24-a6a7-4c25-85a7-184e24963fd2-kube-api-access-t67f5\") pod \"d200ff24-a6a7-4c25-85a7-184e24963fd2\" (UID: \"d200ff24-a6a7-4c25-85a7-184e24963fd2\") "
Oct 31 01:09:03.613098 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:03.613053    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d200ff24-a6a7-4c25-85a7-184e24963fd2-plugins-config-volume" pod "d200ff24-a6a7-4c25-85a7-184e24963fd2" (UID: "d200ff24-a6a7-4c25-85a7-184e24963fd2"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 01:09:03.613418 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:03.613403    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-f1d081-7b4d567df9-xm4lr"
Oct 31 01:09:03.614908 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:03.614890    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d200ff24-a6a7-4c25-85a7-184e24963fd2-kube-api-access-t67f5" pod "d200ff24-a6a7-4c25-85a7-184e24963fd2" (UID: "d200ff24-a6a7-4c25-85a7-184e24963fd2"). InnerVolumeSpecName "kube-api-access-t67f5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:09:03.713986 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:03.713938    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/269e4d26-9e30-4ae6-93b7-111f73f64a81-kube-api-access-n5crn\" (UniqueName: \"kubernetes.io/projected/269e4d26-9e30-4ae6-93b7-111f73f64a81-kube-api-access-n5crn\") pod \"269e4d26-9e30-4ae6-93b7-111f73f64a81\" (UID: \"269e4d26-9e30-4ae6-93b7-111f73f64a81\") "
Oct 31 01:09:03.714197 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:03.714096    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/269e4d26-9e30-4ae6-93b7-111f73f64a81-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/269e4d26-9e30-4ae6-93b7-111f73f64a81-plugins-config-volume\") pod \"269e4d26-9e30-4ae6-93b7-111f73f64a81\" (UID: \"269e4d26-9e30-4ae6-93b7-111f73f64a81\") "
Oct 31 01:09:03.714391 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:03.714373    4284 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d200ff24-a6a7-4c25-85a7-184e24963fd2-plugins-config-volume\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:09:03.714465 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:03.714391    4284 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t67f5\" (UniqueName: \"kubernetes.io/projected/d200ff24-a6a7-4c25-85a7-184e24963fd2-kube-api-access-t67f5\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:09:03.714523 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:03.714491    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/269e4d26-9e30-4ae6-93b7-111f73f64a81-plugins-config-volume" pod "269e4d26-9e30-4ae6-93b7-111f73f64a81" (UID: "269e4d26-9e30-4ae6-93b7-111f73f64a81"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 01:09:03.716250 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:03.716203    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/269e4d26-9e30-4ae6-93b7-111f73f64a81-kube-api-access-n5crn" pod "269e4d26-9e30-4ae6-93b7-111f73f64a81" (UID: "269e4d26-9e30-4ae6-93b7-111f73f64a81"). InnerVolumeSpecName "kube-api-access-n5crn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:09:03.815983 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:03.815885    4284 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n5crn\" (UniqueName: \"kubernetes.io/projected/269e4d26-9e30-4ae6-93b7-111f73f64a81-kube-api-access-n5crn\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:09:03.815983 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:03.815925    4284 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/269e4d26-9e30-4ae6-93b7-111f73f64a81-plugins-config-volume\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:09:04.559746 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:04.559662    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-f1d081-7b4d567df9-xm4lr"
Oct 31 01:09:04.559746 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:04.559706    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-f1d081-7b4d567df9-xm4lr" event={"ID":"269e4d26-9e30-4ae6-93b7-111f73f64a81","Type":"ContainerDied","Data":"082b2cdfeb3c49fab0c417164305f2c9e444b4b55e90a9231b37eaba89d189cf"}
Oct 31 01:09:04.559946 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:04.559767    4284 scope.go:122] "RemoveContainer" containerID="82389a1fe4ada0a336784248f1633ab7200cea630deef0466cca2a2994b8044d"
Oct 31 01:09:04.559946 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:04.559891    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-f1d081-7866949869-8hd89"
Oct 31 01:09:04.718010 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:04.717980    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-f1d081-7866949869-8hd89"]
Oct 31 01:09:04.765542 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:04.765510    4284 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-f1d081-7866949869-8hd89"]
Oct 31 01:09:04.828892 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:04.828809    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-f1d081-7b4d567df9-xm4lr"]
Oct 31 01:09:04.834676 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:04.834649    4284 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-f1d081-7b4d567df9-xm4lr"]
Oct 31 01:09:05.963115 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:05.963083    4284 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="269e4d26-9e30-4ae6-93b7-111f73f64a81" path="/var/lib/kubelet/pods/269e4d26-9e30-4ae6-93b7-111f73f64a81/volumes"
Oct 31 01:09:05.963521 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:05.963484    4284 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d200ff24-a6a7-4c25-85a7-184e24963fd2" path="/var/lib/kubelet/pods/d200ff24-a6a7-4c25-85a7-184e24963fd2/volumes"
Oct 31 01:09:07.498319 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:07.498283    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:09:09.729238 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:09.729167    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:09:09.940447 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:09.940395    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:09:10.091618 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:10.091523    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-praxis-a-8c08e7-openshift-default-5d56d76c65-qnh6x"]
Oct 31 01:09:10.091804 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:10.091772    4284 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-praxis-a-8c08e7-openshift-default-5d56d76c65-qnh6x" podUID="16131784-3cb6-4d01-a27f-6ce599437f3c" containerName="istio-proxy" containerID="cri-o://94ac5923a1bf6d888ab9c36887d11735840e2818943bb261763441c8eac4b4db" gracePeriod=30
Oct 31 01:09:10.108925 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:10.108890    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:09:10.393264 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:10.393147    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:09:10.886556 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:10.886520    4284 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-w4-8fb7f0-openshift-default-77b6794b69-bqwdf"]
Oct 31 01:09:10.887112 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:10.887095    4284 cpu_manager.go:414] "containerMap: removing container" podUID="f6f3f062-5d9e-4051-9c1a-9cf20b93ba0a" containerName="main"
Oct 31 01:09:10.887157 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:10.887115    4284 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f6f3f062-5d9e-4051-9c1a-9cf20b93ba0a" containerName="main"
Oct 31 01:09:10.887157 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:10.887134    4284 cpu_manager.go:414] "containerMap: removing container" podUID="269e4d26-9e30-4ae6-93b7-111f73f64a81" containerName="payload-pre-processing"
Oct 31 01:09:10.887157 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:10.887140    4284 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="269e4d26-9e30-4ae6-93b7-111f73f64a81" containerName="payload-pre-processing"
Oct 31 01:09:10.887157 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:10.887151    4284 cpu_manager.go:414] "containerMap: removing container" podUID="d200ff24-a6a7-4c25-85a7-184e24963fd2" containerName="payload-processing"
Oct 31 01:09:10.887157 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:10.887156    4284 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d200ff24-a6a7-4c25-85a7-184e24963fd2" containerName="payload-processing"
Oct 31 01:09:10.887341 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:10.887278    4284 memory_manager.go:365] "RemoveStaleState removing state" podUID="d200ff24-a6a7-4c25-85a7-184e24963fd2" containerName="payload-processing"
Oct 31 01:09:10.887341 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:10.887297    4284 memory_manager.go:365] "RemoveStaleState removing state" podUID="f6f3f062-5d9e-4051-9c1a-9cf20b93ba0a" containerName="main"
Oct 31 01:09:10.887341 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:10.887303    4284 memory_manager.go:365] "RemoveStaleState removing state" podUID="269e4d26-9e30-4ae6-93b7-111f73f64a81" containerName="payload-pre-processing"
Oct 31 01:09:10.892310 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:10.892285    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w4-8fb7f0-openshift-default-77b6794b69-bqwdf"
Oct 31 01:09:10.971959 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:10.971911    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w4-8fb7f0-openshift-default-77b6794b69-bqwdf"]
Oct 31 01:09:10.996389 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:10.996353    4284 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-w4-8fb7f0-openshift-default-dockercfg-c4hbz\""
Oct 31 01:09:11.008455 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:11.008425    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8af02a59-8229-4932-a556-bfd6ea4e99e0-credential-socket\") pod \"e2e-shared-a-w4-8fb7f0-openshift-default-77b6794b69-bqwdf\" (UID: \"8af02a59-8229-4932-a556-bfd6ea4e99e0\") " pod="openshift-ingress/e2e-shared-a-w4-8fb7f0-openshift-default-77b6794b69-bqwdf"
Oct 31 01:09:11.008643 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:11.008477    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xx4qm\" (UniqueName: \"kubernetes.io/projected/8af02a59-8229-4932-a556-bfd6ea4e99e0-kube-api-access-xx4qm\") pod \"e2e-shared-a-w4-8fb7f0-openshift-default-77b6794b69-bqwdf\" (UID: \"8af02a59-8229-4932-a556-bfd6ea4e99e0\") " pod="openshift-ingress/e2e-shared-a-w4-8fb7f0-openshift-default-77b6794b69-bqwdf"
Oct 31 01:09:11.008739 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:11.008689    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8af02a59-8229-4932-a556-bfd6ea4e99e0-istio-data\") pod \"e2e-shared-a-w4-8fb7f0-openshift-default-77b6794b69-bqwdf\" (UID: \"8af02a59-8229-4932-a556-bfd6ea4e99e0\") " pod="openshift-ingress/e2e-shared-a-w4-8fb7f0-openshift-default-77b6794b69-bqwdf"
Oct 31 01:09:11.008830 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:11.008809    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8af02a59-8229-4932-a556-bfd6ea4e99e0-istiod-ca-cert\") pod \"e2e-shared-a-w4-8fb7f0-openshift-default-77b6794b69-bqwdf\" (UID: \"8af02a59-8229-4932-a556-bfd6ea4e99e0\") " pod="openshift-ingress/e2e-shared-a-w4-8fb7f0-openshift-default-77b6794b69-bqwdf"
Oct 31 01:09:11.009044 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:11.009017    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8af02a59-8229-4932-a556-bfd6ea4e99e0-workload-socket\") pod \"e2e-shared-a-w4-8fb7f0-openshift-default-77b6794b69-bqwdf\" (UID: \"8af02a59-8229-4932-a556-bfd6ea4e99e0\") " pod="openshift-ingress/e2e-shared-a-w4-8fb7f0-openshift-default-77b6794b69-bqwdf"
Oct 31 01:09:11.009137 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:11.009121    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8af02a59-8229-4932-a556-bfd6ea4e99e0-istio-token\") pod \"e2e-shared-a-w4-8fb7f0-openshift-default-77b6794b69-bqwdf\" (UID: \"8af02a59-8229-4932-a556-bfd6ea4e99e0\") " pod="openshift-ingress/e2e-shared-a-w4-8fb7f0-openshift-default-77b6794b69-bqwdf"
Oct 31 01:09:11.009237 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:11.009198    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8af02a59-8229-4932-a556-bfd6ea4e99e0-istio-envoy\") pod \"e2e-shared-a-w4-8fb7f0-openshift-default-77b6794b69-bqwdf\" (UID: \"8af02a59-8229-4932-a556-bfd6ea4e99e0\") " pod="openshift-ingress/e2e-shared-a-w4-8fb7f0-openshift-default-77b6794b69-bqwdf"
Oct 31 01:09:11.009383 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:11.009281    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8af02a59-8229-4932-a556-bfd6ea4e99e0-istio-podinfo\") pod \"e2e-shared-a-w4-8fb7f0-openshift-default-77b6794b69-bqwdf\" (UID: \"8af02a59-8229-4932-a556-bfd6ea4e99e0\") " pod="openshift-ingress/e2e-shared-a-w4-8fb7f0-openshift-default-77b6794b69-bqwdf"
Oct 31 01:09:11.009383 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:11.009354    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8af02a59-8229-4932-a556-bfd6ea4e99e0-workload-certs\") pod \"e2e-shared-a-w4-8fb7f0-openshift-default-77b6794b69-bqwdf\" (UID: \"8af02a59-8229-4932-a556-bfd6ea4e99e0\") " pod="openshift-ingress/e2e-shared-a-w4-8fb7f0-openshift-default-77b6794b69-bqwdf"
Oct 31 01:09:11.010547 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:11.010525    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:09:11.111745 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:11.111705    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8af02a59-8229-4932-a556-bfd6ea4e99e0-credential-socket\") pod \"e2e-shared-a-w4-8fb7f0-openshift-default-77b6794b69-bqwdf\" (UID: \"8af02a59-8229-4932-a556-bfd6ea4e99e0\") " pod="openshift-ingress/e2e-shared-a-w4-8fb7f0-openshift-default-77b6794b69-bqwdf"
Oct 31 01:09:11.112081 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:11.112059    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xx4qm\" (UniqueName: \"kubernetes.io/projected/8af02a59-8229-4932-a556-bfd6ea4e99e0-kube-api-access-xx4qm\") pod \"e2e-shared-a-w4-8fb7f0-openshift-default-77b6794b69-bqwdf\" (UID: \"8af02a59-8229-4932-a556-bfd6ea4e99e0\") " pod="openshift-ingress/e2e-shared-a-w4-8fb7f0-openshift-default-77b6794b69-bqwdf"
Oct 31 01:09:11.112593 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:11.112570    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8af02a59-8229-4932-a556-bfd6ea4e99e0-istio-data\") pod \"e2e-shared-a-w4-8fb7f0-openshift-default-77b6794b69-bqwdf\" (UID: \"8af02a59-8229-4932-a556-bfd6ea4e99e0\") " pod="openshift-ingress/e2e-shared-a-w4-8fb7f0-openshift-default-77b6794b69-bqwdf"
Oct 31 01:09:11.112743 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:11.112729    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8af02a59-8229-4932-a556-bfd6ea4e99e0-istiod-ca-cert\") pod \"e2e-shared-a-w4-8fb7f0-openshift-default-77b6794b69-bqwdf\" (UID: \"8af02a59-8229-4932-a556-bfd6ea4e99e0\") " pod="openshift-ingress/e2e-shared-a-w4-8fb7f0-openshift-default-77b6794b69-bqwdf"
Oct 31 01:09:11.114254 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:11.114230    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8af02a59-8229-4932-a556-bfd6ea4e99e0-workload-socket\") pod \"e2e-shared-a-w4-8fb7f0-openshift-default-77b6794b69-bqwdf\" (UID: \"8af02a59-8229-4932-a556-bfd6ea4e99e0\") " pod="openshift-ingress/e2e-shared-a-w4-8fb7f0-openshift-default-77b6794b69-bqwdf"
Oct 31 01:09:11.114459 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:11.114444    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8af02a59-8229-4932-a556-bfd6ea4e99e0-istio-token\") pod \"e2e-shared-a-w4-8fb7f0-openshift-default-77b6794b69-bqwdf\" (UID: \"8af02a59-8229-4932-a556-bfd6ea4e99e0\") " pod="openshift-ingress/e2e-shared-a-w4-8fb7f0-openshift-default-77b6794b69-bqwdf"
Oct 31 01:09:11.115787 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:11.115753    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8af02a59-8229-4932-a556-bfd6ea4e99e0-istio-envoy\") pod \"e2e-shared-a-w4-8fb7f0-openshift-default-77b6794b69-bqwdf\" (UID: \"8af02a59-8229-4932-a556-bfd6ea4e99e0\") " pod="openshift-ingress/e2e-shared-a-w4-8fb7f0-openshift-default-77b6794b69-bqwdf"
Oct 31 01:09:11.115891 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:11.115826    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8af02a59-8229-4932-a556-bfd6ea4e99e0-istio-podinfo\") pod \"e2e-shared-a-w4-8fb7f0-openshift-default-77b6794b69-bqwdf\" (UID: \"8af02a59-8229-4932-a556-bfd6ea4e99e0\") " pod="openshift-ingress/e2e-shared-a-w4-8fb7f0-openshift-default-77b6794b69-bqwdf"
Oct 31 01:09:11.115891 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:11.115881    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8af02a59-8229-4932-a556-bfd6ea4e99e0-workload-certs\") pod \"e2e-shared-a-w4-8fb7f0-openshift-default-77b6794b69-bqwdf\" (UID: \"8af02a59-8229-4932-a556-bfd6ea4e99e0\") " pod="openshift-ingress/e2e-shared-a-w4-8fb7f0-openshift-default-77b6794b69-bqwdf"
Oct 31 01:09:11.116018 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:11.115762    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8af02a59-8229-4932-a556-bfd6ea4e99e0-istiod-ca-cert\") pod \"e2e-shared-a-w4-8fb7f0-openshift-default-77b6794b69-bqwdf\" (UID: \"8af02a59-8229-4932-a556-bfd6ea4e99e0\") " pod="openshift-ingress/e2e-shared-a-w4-8fb7f0-openshift-default-77b6794b69-bqwdf"
Oct 31 01:09:11.116074 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:11.114831    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8af02a59-8229-4932-a556-bfd6ea4e99e0-workload-socket\") pod \"e2e-shared-a-w4-8fb7f0-openshift-default-77b6794b69-bqwdf\" (UID: \"8af02a59-8229-4932-a556-bfd6ea4e99e0\") " pod="openshift-ingress/e2e-shared-a-w4-8fb7f0-openshift-default-77b6794b69-bqwdf"
Oct 31 01:09:11.116074 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:11.112347    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8af02a59-8229-4932-a556-bfd6ea4e99e0-credential-socket\") pod \"e2e-shared-a-w4-8fb7f0-openshift-default-77b6794b69-bqwdf\" (UID: \"8af02a59-8229-4932-a556-bfd6ea4e99e0\") " pod="openshift-ingress/e2e-shared-a-w4-8fb7f0-openshift-default-77b6794b69-bqwdf"
Oct 31 01:09:11.116074 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:11.114870    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8af02a59-8229-4932-a556-bfd6ea4e99e0-istio-data\") pod \"e2e-shared-a-w4-8fb7f0-openshift-default-77b6794b69-bqwdf\" (UID: \"8af02a59-8229-4932-a556-bfd6ea4e99e0\") " pod="openshift-ingress/e2e-shared-a-w4-8fb7f0-openshift-default-77b6794b69-bqwdf"
Oct 31 01:09:11.116541 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:11.116337    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8af02a59-8229-4932-a556-bfd6ea4e99e0-workload-certs\") pod \"e2e-shared-a-w4-8fb7f0-openshift-default-77b6794b69-bqwdf\" (UID: \"8af02a59-8229-4932-a556-bfd6ea4e99e0\") " pod="openshift-ingress/e2e-shared-a-w4-8fb7f0-openshift-default-77b6794b69-bqwdf"
Oct 31 01:09:11.123774 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:11.122018    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8af02a59-8229-4932-a556-bfd6ea4e99e0-istio-podinfo\") pod \"e2e-shared-a-w4-8fb7f0-openshift-default-77b6794b69-bqwdf\" (UID: \"8af02a59-8229-4932-a556-bfd6ea4e99e0\") " pod="openshift-ingress/e2e-shared-a-w4-8fb7f0-openshift-default-77b6794b69-bqwdf"
Oct 31 01:09:11.127972 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:11.127948    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8af02a59-8229-4932-a556-bfd6ea4e99e0-istio-envoy\") pod \"e2e-shared-a-w4-8fb7f0-openshift-default-77b6794b69-bqwdf\" (UID: \"8af02a59-8229-4932-a556-bfd6ea4e99e0\") " pod="openshift-ingress/e2e-shared-a-w4-8fb7f0-openshift-default-77b6794b69-bqwdf"
Oct 31 01:09:11.171246 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:11.171186    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xx4qm\" (UniqueName: \"kubernetes.io/projected/8af02a59-8229-4932-a556-bfd6ea4e99e0-kube-api-access-xx4qm\") pod \"e2e-shared-a-w4-8fb7f0-openshift-default-77b6794b69-bqwdf\" (UID: \"8af02a59-8229-4932-a556-bfd6ea4e99e0\") " pod="openshift-ingress/e2e-shared-a-w4-8fb7f0-openshift-default-77b6794b69-bqwdf"
Oct 31 01:09:11.172982 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:11.172931    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8af02a59-8229-4932-a556-bfd6ea4e99e0-istio-token\") pod \"e2e-shared-a-w4-8fb7f0-openshift-default-77b6794b69-bqwdf\" (UID: \"8af02a59-8229-4932-a556-bfd6ea4e99e0\") " pod="openshift-ingress/e2e-shared-a-w4-8fb7f0-openshift-default-77b6794b69-bqwdf"
Oct 31 01:09:11.206053 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:11.206014    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w4-8fb7f0-openshift-default-77b6794b69-bqwdf"
Oct 31 01:09:11.366111 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:11.366072    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:09:11.626809 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:11.626785    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w4-8fb7f0-openshift-default-77b6794b69-bqwdf"]
Oct 31 01:09:11.627511 ip-10-0-132-173 kubenswrapper[4284]: W1031 01:09:11.627478    4284 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8af02a59_8229_4932_a556_bfd6ea4e99e0.slice/crio-4c3c32900870b927cc1656503b09f5daac4ad5a7fc4f3c76226d5fc4526ceaae WatchSource:0}: Error finding container 4c3c32900870b927cc1656503b09f5daac4ad5a7fc4f3c76226d5fc4526ceaae: Status 404 returned error can't find the container with id 4c3c32900870b927cc1656503b09f5daac4ad5a7fc4f3c76226d5fc4526ceaae
Oct 31 01:09:11.630161 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:11.630121    4284 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 01:09:11.630268 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:11.630243    4284 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 01:09:11.630311 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:11.630292    4284 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 01:09:12.232720 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:12.232689    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:09:12.608865 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:12.608775    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w4-8fb7f0-openshift-default-77b6794b69-bqwdf" event={"ID":"8af02a59-8229-4932-a556-bfd6ea4e99e0","Type":"ContainerStarted","Data":"f7aad86c2c59b80bdf1c3ebaeb3464cc25b23ac0e09f031cbae0f32e5a09549a"}
Oct 31 01:09:12.608865 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:12.608825    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w4-8fb7f0-openshift-default-77b6794b69-bqwdf" event={"ID":"8af02a59-8229-4932-a556-bfd6ea4e99e0","Type":"ContainerStarted","Data":"4c3c32900870b927cc1656503b09f5daac4ad5a7fc4f3c76226d5fc4526ceaae"}
Oct 31 01:09:12.726676 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:12.726628    4284 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-w4-8fb7f0-openshift-default-77b6794b69-bqwdf" podStartSLOduration=2.726615877 podStartE2EDuration="2.726615877s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 01:09:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 01:09:12.72580112 +0000 UTC m=+2466.870808036" watchObservedRunningTime="2026-10-31 01:09:12.726615877 +0000 UTC m=+2466.871622793"
Oct 31 01:09:13.206407 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:13.206374    4284 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w4-8fb7f0-openshift-default-77b6794b69-bqwdf"
Oct 31 01:09:13.211820 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:13.211795    4284 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w4-8fb7f0-openshift-default-77b6794b69-bqwdf"
Oct 31 01:09:13.615411 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:13.615324    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w4-8fb7f0-openshift-default-77b6794b69-bqwdf"
Oct 31 01:09:13.616387 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:13.616361    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w4-8fb7f0-openshift-default-77b6794b69-bqwdf"
Oct 31 01:09:15.701044 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:15.700340    4284 generic.go:356] "Generic (PLEG): container finished" podID="16131784-3cb6-4d01-a27f-6ce599437f3c" containerID="94ac5923a1bf6d888ab9c36887d11735840e2818943bb261763441c8eac4b4db" exitCode=0
Oct 31 01:09:15.702397 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:15.702368    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-a-8c08e7-openshift-default-5d56d76c65-qnh6x" event={"ID":"16131784-3cb6-4d01-a27f-6ce599437f3c","Type":"ContainerDied","Data":"94ac5923a1bf6d888ab9c36887d11735840e2818943bb261763441c8eac4b4db"}
Oct 31 01:09:15.720164 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:15.720124    4284 scope.go:122] "RemoveContainer" containerID="bae8086e21fa68bb91f5084ddbb3811c359f00373ecd02892c41bad22b4625dc"
Oct 31 01:09:15.760522 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:15.757313    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:09:15.764875 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:15.764803    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-praxis-a-8c08e7-openshift-default-5d56d76c65-qnh6x"
Oct 31 01:09:15.796446 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:15.796360    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/16131784-3cb6-4d01-a27f-6ce599437f3c-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/16131784-3cb6-4d01-a27f-6ce599437f3c-istio-data\") pod \"16131784-3cb6-4d01-a27f-6ce599437f3c\" (UID: \"16131784-3cb6-4d01-a27f-6ce599437f3c\") "
Oct 31 01:09:15.797187 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:15.797164    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/16131784-3cb6-4d01-a27f-6ce599437f3c-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/16131784-3cb6-4d01-a27f-6ce599437f3c-credential-socket\") pod \"16131784-3cb6-4d01-a27f-6ce599437f3c\" (UID: \"16131784-3cb6-4d01-a27f-6ce599437f3c\") "
Oct 31 01:09:15.797651 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:15.797632    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/16131784-3cb6-4d01-a27f-6ce599437f3c-istio-token\" (UniqueName: \"kubernetes.io/projected/16131784-3cb6-4d01-a27f-6ce599437f3c-istio-token\") pod \"16131784-3cb6-4d01-a27f-6ce599437f3c\" (UID: \"16131784-3cb6-4d01-a27f-6ce599437f3c\") "
Oct 31 01:09:15.798185 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:15.798165    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/16131784-3cb6-4d01-a27f-6ce599437f3c-kube-api-access-gh8xd\" (UniqueName: \"kubernetes.io/projected/16131784-3cb6-4d01-a27f-6ce599437f3c-kube-api-access-gh8xd\") pod \"16131784-3cb6-4d01-a27f-6ce599437f3c\" (UID: \"16131784-3cb6-4d01-a27f-6ce599437f3c\") "
Oct 31 01:09:15.798496 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:15.797084    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/16131784-3cb6-4d01-a27f-6ce599437f3c-istio-data" pod "16131784-3cb6-4d01-a27f-6ce599437f3c" (UID: "16131784-3cb6-4d01-a27f-6ce599437f3c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:09:15.798496 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:15.797571    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/16131784-3cb6-4d01-a27f-6ce599437f3c-credential-socket" pod "16131784-3cb6-4d01-a27f-6ce599437f3c" (UID: "16131784-3cb6-4d01-a27f-6ce599437f3c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:09:15.799037 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:15.799017    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/16131784-3cb6-4d01-a27f-6ce599437f3c-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/16131784-3cb6-4d01-a27f-6ce599437f3c-istio-podinfo\") pod \"16131784-3cb6-4d01-a27f-6ce599437f3c\" (UID: \"16131784-3cb6-4d01-a27f-6ce599437f3c\") "
Oct 31 01:09:15.799548 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:15.799531    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/16131784-3cb6-4d01-a27f-6ce599437f3c-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/16131784-3cb6-4d01-a27f-6ce599437f3c-istio-envoy\") pod \"16131784-3cb6-4d01-a27f-6ce599437f3c\" (UID: \"16131784-3cb6-4d01-a27f-6ce599437f3c\") "
Oct 31 01:09:15.799769 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:15.799751    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/16131784-3cb6-4d01-a27f-6ce599437f3c-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/16131784-3cb6-4d01-a27f-6ce599437f3c-istiod-ca-cert\") pod \"16131784-3cb6-4d01-a27f-6ce599437f3c\" (UID: \"16131784-3cb6-4d01-a27f-6ce599437f3c\") "
Oct 31 01:09:15.799988 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:15.799970    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/16131784-3cb6-4d01-a27f-6ce599437f3c-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/16131784-3cb6-4d01-a27f-6ce599437f3c-workload-certs\") pod \"16131784-3cb6-4d01-a27f-6ce599437f3c\" (UID: \"16131784-3cb6-4d01-a27f-6ce599437f3c\") "
Oct 31 01:09:15.800117 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:15.800101    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/16131784-3cb6-4d01-a27f-6ce599437f3c-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/16131784-3cb6-4d01-a27f-6ce599437f3c-workload-socket\") pod \"16131784-3cb6-4d01-a27f-6ce599437f3c\" (UID: \"16131784-3cb6-4d01-a27f-6ce599437f3c\") "
Oct 31 01:09:15.801609 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:15.800806    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/16131784-3cb6-4d01-a27f-6ce599437f3c-workload-certs" pod "16131784-3cb6-4d01-a27f-6ce599437f3c" (UID: "16131784-3cb6-4d01-a27f-6ce599437f3c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:09:15.801609 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:15.800902    4284 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/16131784-3cb6-4d01-a27f-6ce599437f3c-workload-certs\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:09:15.801609 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:15.800929    4284 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/16131784-3cb6-4d01-a27f-6ce599437f3c-istio-data\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:09:15.801609 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:15.800945    4284 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/16131784-3cb6-4d01-a27f-6ce599437f3c-credential-socket\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:09:15.801609 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:15.801123    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/16131784-3cb6-4d01-a27f-6ce599437f3c-istiod-ca-cert" pod "16131784-3cb6-4d01-a27f-6ce599437f3c" (UID: "16131784-3cb6-4d01-a27f-6ce599437f3c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 01:09:15.801609 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:15.801595    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/16131784-3cb6-4d01-a27f-6ce599437f3c-workload-socket" pod "16131784-3cb6-4d01-a27f-6ce599437f3c" (UID: "16131784-3cb6-4d01-a27f-6ce599437f3c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:09:15.802167 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:15.802135    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/16131784-3cb6-4d01-a27f-6ce599437f3c-istio-token" pod "16131784-3cb6-4d01-a27f-6ce599437f3c" (UID: "16131784-3cb6-4d01-a27f-6ce599437f3c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:09:15.802684 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:15.802664    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/16131784-3cb6-4d01-a27f-6ce599437f3c-kube-api-access-gh8xd" pod "16131784-3cb6-4d01-a27f-6ce599437f3c" (UID: "16131784-3cb6-4d01-a27f-6ce599437f3c"). InnerVolumeSpecName "kube-api-access-gh8xd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:09:15.803457 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:15.803431    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/16131784-3cb6-4d01-a27f-6ce599437f3c-istio-envoy" pod "16131784-3cb6-4d01-a27f-6ce599437f3c" (UID: "16131784-3cb6-4d01-a27f-6ce599437f3c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:09:15.808410 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:15.804011    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/16131784-3cb6-4d01-a27f-6ce599437f3c-istio-podinfo" pod "16131784-3cb6-4d01-a27f-6ce599437f3c" (UID: "16131784-3cb6-4d01-a27f-6ce599437f3c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 01:09:15.902554 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:15.902493    4284 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/16131784-3cb6-4d01-a27f-6ce599437f3c-istiod-ca-cert\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:09:15.902836 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:15.902803    4284 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/16131784-3cb6-4d01-a27f-6ce599437f3c-workload-socket\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:09:15.902836 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:15.902837    4284 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/16131784-3cb6-4d01-a27f-6ce599437f3c-istio-token\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:09:15.903050 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:15.902853    4284 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gh8xd\" (UniqueName: \"kubernetes.io/projected/16131784-3cb6-4d01-a27f-6ce599437f3c-kube-api-access-gh8xd\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:09:15.903050 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:15.902870    4284 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/16131784-3cb6-4d01-a27f-6ce599437f3c-istio-podinfo\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:09:15.903050 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:15.902885    4284 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/16131784-3cb6-4d01-a27f-6ce599437f3c-istio-envoy\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:09:15.969517 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:15.969479    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:09:16.707942 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:16.707859    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-a-8c08e7-openshift-default-5d56d76c65-qnh6x" event={"ID":"16131784-3cb6-4d01-a27f-6ce599437f3c","Type":"ContainerDied","Data":"200c93936c05cb305667973fb5ea2a61061c49abffbd860bc4fe91959d41620e"}
Oct 31 01:09:16.707942 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:16.707885    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-praxis-a-8c08e7-openshift-default-5d56d76c65-qnh6x"
Oct 31 01:09:16.707942 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:16.707915    4284 scope.go:122] "RemoveContainer" containerID="94ac5923a1bf6d888ab9c36887d11735840e2818943bb261763441c8eac4b4db"
Oct 31 01:09:16.980261 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:16.980167    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-praxis-a-8c08e7-openshift-default-5d56d76c65-qnh6x"]
Oct 31 01:09:16.986062 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:16.986036    4284 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-praxis-a-8c08e7-openshift-default-5d56d76c65-qnh6x"]
Oct 31 01:09:17.013519 ip-10-0-132-173 kubenswrapper[4284]: E1031 01:09:17.013486    4284 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fd8sp" containerName="extractor"
Oct 31 01:09:17.471453 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:17.471419    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:09:17.962718 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:17.962685    4284 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="16131784-3cb6-4d01-a27f-6ce599437f3c" path="/var/lib/kubelet/pods/16131784-3cb6-4d01-a27f-6ce599437f3c/volumes"
Oct 31 01:09:18.425071 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:18.425037    4284 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-4jv2z"]
Oct 31 01:09:18.425542 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:18.425530    4284 cpu_manager.go:414] "containerMap: removing container" podUID="16131784-3cb6-4d01-a27f-6ce599437f3c" containerName="istio-proxy"
Oct 31 01:09:18.425619 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:18.425545    4284 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="16131784-3cb6-4d01-a27f-6ce599437f3c" containerName="istio-proxy"
Oct 31 01:09:18.425659 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:18.425650    4284 memory_manager.go:365] "RemoveStaleState removing state" podUID="16131784-3cb6-4d01-a27f-6ce599437f3c" containerName="istio-proxy"
Oct 31 01:09:18.425701 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:18.425663    4284 memory_manager.go:365] "RemoveStaleState removing state" podUID="16131784-3cb6-4d01-a27f-6ce599437f3c" containerName="istio-proxy"
Oct 31 01:09:18.430265 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:18.430244    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-4jv2z"
Oct 31 01:09:18.516903 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:18.516873    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-4jv2z"]
Oct 31 01:09:18.539699 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:18.539669    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/205f3d84-b321-479b-8931-7584aaf21b1a-istio-token\") pod \"e2e-worker-w4-ef2572-openshift-default-7649dd87-4jv2z\" (UID: \"205f3d84-b321-479b-8931-7584aaf21b1a\") " pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-4jv2z"
Oct 31 01:09:18.539915 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:18.539825    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n9kws\" (UniqueName: \"kubernetes.io/projected/205f3d84-b321-479b-8931-7584aaf21b1a-kube-api-access-n9kws\") pod \"e2e-worker-w4-ef2572-openshift-default-7649dd87-4jv2z\" (UID: \"205f3d84-b321-479b-8931-7584aaf21b1a\") " pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-4jv2z"
Oct 31 01:09:18.540003 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:18.539920    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/205f3d84-b321-479b-8931-7584aaf21b1a-istio-envoy\") pod \"e2e-worker-w4-ef2572-openshift-default-7649dd87-4jv2z\" (UID: \"205f3d84-b321-479b-8931-7584aaf21b1a\") " pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-4jv2z"
Oct 31 01:09:18.540067 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:18.540030    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/205f3d84-b321-479b-8931-7584aaf21b1a-istio-data\") pod \"e2e-worker-w4-ef2572-openshift-default-7649dd87-4jv2z\" (UID: \"205f3d84-b321-479b-8931-7584aaf21b1a\") " pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-4jv2z"
Oct 31 01:09:18.540123 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:18.540078    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/205f3d84-b321-479b-8931-7584aaf21b1a-istio-podinfo\") pod \"e2e-worker-w4-ef2572-openshift-default-7649dd87-4jv2z\" (UID: \"205f3d84-b321-479b-8931-7584aaf21b1a\") " pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-4jv2z"
Oct 31 01:09:18.540257 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:18.540198    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/205f3d84-b321-479b-8931-7584aaf21b1a-workload-socket\") pod \"e2e-worker-w4-ef2572-openshift-default-7649dd87-4jv2z\" (UID: \"205f3d84-b321-479b-8931-7584aaf21b1a\") " pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-4jv2z"
Oct 31 01:09:18.540400 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:18.540274    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/205f3d84-b321-479b-8931-7584aaf21b1a-credential-socket\") pod \"e2e-worker-w4-ef2572-openshift-default-7649dd87-4jv2z\" (UID: \"205f3d84-b321-479b-8931-7584aaf21b1a\") " pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-4jv2z"
Oct 31 01:09:18.540511 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:18.540471    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/205f3d84-b321-479b-8931-7584aaf21b1a-istiod-ca-cert\") pod \"e2e-worker-w4-ef2572-openshift-default-7649dd87-4jv2z\" (UID: \"205f3d84-b321-479b-8931-7584aaf21b1a\") " pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-4jv2z"
Oct 31 01:09:18.540580 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:18.540531    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/205f3d84-b321-479b-8931-7584aaf21b1a-workload-certs\") pod \"e2e-worker-w4-ef2572-openshift-default-7649dd87-4jv2z\" (UID: \"205f3d84-b321-479b-8931-7584aaf21b1a\") " pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-4jv2z"
Oct 31 01:09:18.641794 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:18.641761    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/205f3d84-b321-479b-8931-7584aaf21b1a-istio-envoy\") pod \"e2e-worker-w4-ef2572-openshift-default-7649dd87-4jv2z\" (UID: \"205f3d84-b321-479b-8931-7584aaf21b1a\") " pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-4jv2z"
Oct 31 01:09:18.641794 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:18.641799    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/205f3d84-b321-479b-8931-7584aaf21b1a-istio-data\") pod \"e2e-worker-w4-ef2572-openshift-default-7649dd87-4jv2z\" (UID: \"205f3d84-b321-479b-8931-7584aaf21b1a\") " pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-4jv2z"
Oct 31 01:09:18.641794 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:18.641826    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/205f3d84-b321-479b-8931-7584aaf21b1a-istio-podinfo\") pod \"e2e-worker-w4-ef2572-openshift-default-7649dd87-4jv2z\" (UID: \"205f3d84-b321-479b-8931-7584aaf21b1a\") " pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-4jv2z"
Oct 31 01:09:18.642131 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:18.641962    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/205f3d84-b321-479b-8931-7584aaf21b1a-workload-socket\") pod \"e2e-worker-w4-ef2572-openshift-default-7649dd87-4jv2z\" (UID: \"205f3d84-b321-479b-8931-7584aaf21b1a\") " pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-4jv2z"
Oct 31 01:09:18.642131 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:18.642001    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/205f3d84-b321-479b-8931-7584aaf21b1a-credential-socket\") pod \"e2e-worker-w4-ef2572-openshift-default-7649dd87-4jv2z\" (UID: \"205f3d84-b321-479b-8931-7584aaf21b1a\") " pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-4jv2z"
Oct 31 01:09:18.642131 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:18.642041    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/205f3d84-b321-479b-8931-7584aaf21b1a-istiod-ca-cert\") pod \"e2e-worker-w4-ef2572-openshift-default-7649dd87-4jv2z\" (UID: \"205f3d84-b321-479b-8931-7584aaf21b1a\") " pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-4jv2z"
Oct 31 01:09:18.642131 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:18.642081    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/205f3d84-b321-479b-8931-7584aaf21b1a-workload-certs\") pod \"e2e-worker-w4-ef2572-openshift-default-7649dd87-4jv2z\" (UID: \"205f3d84-b321-479b-8931-7584aaf21b1a\") " pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-4jv2z"
Oct 31 01:09:18.642131 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:18.642129    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/205f3d84-b321-479b-8931-7584aaf21b1a-istio-token\") pod \"e2e-worker-w4-ef2572-openshift-default-7649dd87-4jv2z\" (UID: \"205f3d84-b321-479b-8931-7584aaf21b1a\") " pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-4jv2z"
Oct 31 01:09:18.642433 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:18.642258    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n9kws\" (UniqueName: \"kubernetes.io/projected/205f3d84-b321-479b-8931-7584aaf21b1a-kube-api-access-n9kws\") pod \"e2e-worker-w4-ef2572-openshift-default-7649dd87-4jv2z\" (UID: \"205f3d84-b321-479b-8931-7584aaf21b1a\") " pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-4jv2z"
Oct 31 01:09:18.642433 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:18.642364    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/205f3d84-b321-479b-8931-7584aaf21b1a-istio-data\") pod \"e2e-worker-w4-ef2572-openshift-default-7649dd87-4jv2z\" (UID: \"205f3d84-b321-479b-8931-7584aaf21b1a\") " pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-4jv2z"
Oct 31 01:09:18.642433 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:18.642425    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/205f3d84-b321-479b-8931-7584aaf21b1a-workload-socket\") pod \"e2e-worker-w4-ef2572-openshift-default-7649dd87-4jv2z\" (UID: \"205f3d84-b321-479b-8931-7584aaf21b1a\") " pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-4jv2z"
Oct 31 01:09:18.642760 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:18.642600    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/205f3d84-b321-479b-8931-7584aaf21b1a-workload-certs\") pod \"e2e-worker-w4-ef2572-openshift-default-7649dd87-4jv2z\" (UID: \"205f3d84-b321-479b-8931-7584aaf21b1a\") " pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-4jv2z"
Oct 31 01:09:18.642760 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:18.642690    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/205f3d84-b321-479b-8931-7584aaf21b1a-istiod-ca-cert\") pod \"e2e-worker-w4-ef2572-openshift-default-7649dd87-4jv2z\" (UID: \"205f3d84-b321-479b-8931-7584aaf21b1a\") " pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-4jv2z"
Oct 31 01:09:18.642760 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:18.642720    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/205f3d84-b321-479b-8931-7584aaf21b1a-credential-socket\") pod \"e2e-worker-w4-ef2572-openshift-default-7649dd87-4jv2z\" (UID: \"205f3d84-b321-479b-8931-7584aaf21b1a\") " pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-4jv2z"
Oct 31 01:09:18.644346 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:18.644326    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/205f3d84-b321-479b-8931-7584aaf21b1a-istio-envoy\") pod \"e2e-worker-w4-ef2572-openshift-default-7649dd87-4jv2z\" (UID: \"205f3d84-b321-479b-8931-7584aaf21b1a\") " pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-4jv2z"
Oct 31 01:09:18.644556 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:18.644538    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/205f3d84-b321-479b-8931-7584aaf21b1a-istio-podinfo\") pod \"e2e-worker-w4-ef2572-openshift-default-7649dd87-4jv2z\" (UID: \"205f3d84-b321-479b-8931-7584aaf21b1a\") " pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-4jv2z"
Oct 31 01:09:18.713369 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:18.713291    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/205f3d84-b321-479b-8931-7584aaf21b1a-istio-token\") pod \"e2e-worker-w4-ef2572-openshift-default-7649dd87-4jv2z\" (UID: \"205f3d84-b321-479b-8931-7584aaf21b1a\") " pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-4jv2z"
Oct 31 01:09:18.714468 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:18.714437    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n9kws\" (UniqueName: \"kubernetes.io/projected/205f3d84-b321-479b-8931-7584aaf21b1a-kube-api-access-n9kws\") pod \"e2e-worker-w4-ef2572-openshift-default-7649dd87-4jv2z\" (UID: \"205f3d84-b321-479b-8931-7584aaf21b1a\") " pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-4jv2z"
Oct 31 01:09:18.743308 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:18.743277    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-4jv2z"
Oct 31 01:09:18.843236 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:18.843181    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:09:19.031096 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:19.022772    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:09:19.198947 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:19.198908    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-4jv2z"]
Oct 31 01:09:19.215739 ip-10-0-132-173 kubenswrapper[4284]: W1031 01:09:19.215705    4284 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod205f3d84_b321_479b_8931_7584aaf21b1a.slice/crio-dfc1d9e7d3eb9bec88f003af5c061230a971449fb94a6b75caa46dcdf8178ccc WatchSource:0}: Error finding container dfc1d9e7d3eb9bec88f003af5c061230a971449fb94a6b75caa46dcdf8178ccc: Status 404 returned error can't find the container with id dfc1d9e7d3eb9bec88f003af5c061230a971449fb94a6b75caa46dcdf8178ccc
Oct 31 01:09:19.218391 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:19.218357    4284 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 01:09:19.218506 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:19.218436    4284 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 01:09:19.218506 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:19.218483    4284 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 01:09:19.259436 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:19.259412    4284 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w4-8fb7f0-796d4996ff-25jt7"]
Oct 31 01:09:19.259925 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:19.259913    4284 cpu_manager.go:414] "containerMap: removing container" podUID="16131784-3cb6-4d01-a27f-6ce599437f3c" containerName="istio-proxy"
Oct 31 01:09:19.259968 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:19.259926    4284 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="16131784-3cb6-4d01-a27f-6ce599437f3c" containerName="istio-proxy"
Oct 31 01:09:19.269326 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:19.269306    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w4-8fb7f0-796d4996ff-25jt7"
Oct 31 01:09:19.318068 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:19.317983    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w4-8fb7f0-796d4996ff-25jt7"]
Oct 31 01:09:19.320505 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:19.320479    4284 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w4-8fb7f0\""
Oct 31 01:09:19.320658 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:19.320506    4284 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-w4-8fb7f0-dockercfg-mbnmj\""
Oct 31 01:09:19.374236 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:19.371934    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dwxhj\" (UniqueName: \"kubernetes.io/projected/0e3a22bb-0915-4025-af10-c9c3b01011f1-kube-api-access-dwxhj\") pod \"payload-pre-processing-e2e-shared-a-w4-8fb7f0-796d4996ff-25jt7\" (UID: \"0e3a22bb-0915-4025-af10-c9c3b01011f1\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w4-8fb7f0-796d4996ff-25jt7"
Oct 31 01:09:19.374236 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:19.372138    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0e3a22bb-0915-4025-af10-c9c3b01011f1-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w4-8fb7f0-796d4996ff-25jt7\" (UID: \"0e3a22bb-0915-4025-af10-c9c3b01011f1\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w4-8fb7f0-796d4996ff-25jt7"
Oct 31 01:09:19.448691 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:19.435663    4284 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w4-8fb7f0-5889f6f4ff-r76rq"]
Oct 31 01:09:19.456873 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:19.456494    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w4-8fb7f0-5889f6f4ff-r76rq"
Oct 31 01:09:19.474009 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:19.473389    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0e3a22bb-0915-4025-af10-c9c3b01011f1-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w4-8fb7f0-796d4996ff-25jt7\" (UID: \"0e3a22bb-0915-4025-af10-c9c3b01011f1\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w4-8fb7f0-796d4996ff-25jt7"
Oct 31 01:09:19.474009 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:19.473518    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ddda98fe-7ad3-4592-81bb-2b6ce526b882-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w4-8fb7f0-5889f6f4ff-r76rq\" (UID: \"ddda98fe-7ad3-4592-81bb-2b6ce526b882\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w4-8fb7f0-5889f6f4ff-r76rq"
Oct 31 01:09:19.474009 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:19.473557    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ggl82\" (UniqueName: \"kubernetes.io/projected/ddda98fe-7ad3-4592-81bb-2b6ce526b882-kube-api-access-ggl82\") pod \"payload-processing-e2e-shared-a-w4-8fb7f0-5889f6f4ff-r76rq\" (UID: \"ddda98fe-7ad3-4592-81bb-2b6ce526b882\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w4-8fb7f0-5889f6f4ff-r76rq"
Oct 31 01:09:19.474009 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:19.473611    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dwxhj\" (UniqueName: \"kubernetes.io/projected/0e3a22bb-0915-4025-af10-c9c3b01011f1-kube-api-access-dwxhj\") pod \"payload-pre-processing-e2e-shared-a-w4-8fb7f0-796d4996ff-25jt7\" (UID: \"0e3a22bb-0915-4025-af10-c9c3b01011f1\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w4-8fb7f0-796d4996ff-25jt7"
Oct 31 01:09:19.476247 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:19.475269    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0e3a22bb-0915-4025-af10-c9c3b01011f1-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w4-8fb7f0-796d4996ff-25jt7\" (UID: \"0e3a22bb-0915-4025-af10-c9c3b01011f1\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w4-8fb7f0-796d4996ff-25jt7"
Oct 31 01:09:19.497258 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:19.496237    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w4-8fb7f0-5889f6f4ff-r76rq"]
Oct 31 01:09:19.533836 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:19.533801    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dwxhj\" (UniqueName: \"kubernetes.io/projected/0e3a22bb-0915-4025-af10-c9c3b01011f1-kube-api-access-dwxhj\") pod \"payload-pre-processing-e2e-shared-a-w4-8fb7f0-796d4996ff-25jt7\" (UID: \"0e3a22bb-0915-4025-af10-c9c3b01011f1\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w4-8fb7f0-796d4996ff-25jt7"
Oct 31 01:09:19.574641 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:19.574556    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ddda98fe-7ad3-4592-81bb-2b6ce526b882-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w4-8fb7f0-5889f6f4ff-r76rq\" (UID: \"ddda98fe-7ad3-4592-81bb-2b6ce526b882\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w4-8fb7f0-5889f6f4ff-r76rq"
Oct 31 01:09:19.574641 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:19.574596    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ggl82\" (UniqueName: \"kubernetes.io/projected/ddda98fe-7ad3-4592-81bb-2b6ce526b882-kube-api-access-ggl82\") pod \"payload-processing-e2e-shared-a-w4-8fb7f0-5889f6f4ff-r76rq\" (UID: \"ddda98fe-7ad3-4592-81bb-2b6ce526b882\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w4-8fb7f0-5889f6f4ff-r76rq"
Oct 31 01:09:19.575635 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:19.575611    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ddda98fe-7ad3-4592-81bb-2b6ce526b882-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w4-8fb7f0-5889f6f4ff-r76rq\" (UID: \"ddda98fe-7ad3-4592-81bb-2b6ce526b882\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w4-8fb7f0-5889f6f4ff-r76rq"
Oct 31 01:09:19.583120 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:19.583098    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w4-8fb7f0-796d4996ff-25jt7"
Oct 31 01:09:19.633969 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:19.633927    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ggl82\" (UniqueName: \"kubernetes.io/projected/ddda98fe-7ad3-4592-81bb-2b6ce526b882-kube-api-access-ggl82\") pod \"payload-processing-e2e-shared-a-w4-8fb7f0-5889f6f4ff-r76rq\" (UID: \"ddda98fe-7ad3-4592-81bb-2b6ce526b882\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w4-8fb7f0-5889f6f4ff-r76rq"
Oct 31 01:09:19.731449 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:19.731412    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-4jv2z" event={"ID":"205f3d84-b321-479b-8931-7584aaf21b1a","Type":"ContainerStarted","Data":"cfdefd97e1de84f7778a864c751225e5c57abd9ba93244648c72e318acf28a88"}
Oct 31 01:09:19.731604 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:19.731475    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-4jv2z" event={"ID":"205f3d84-b321-479b-8931-7584aaf21b1a","Type":"ContainerStarted","Data":"dfc1d9e7d3eb9bec88f003af5c061230a971449fb94a6b75caa46dcdf8178ccc"}
Oct 31 01:09:19.743824 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:19.743791    4284 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-4jv2z"
Oct 31 01:09:19.792568 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:19.792536    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w4-8fb7f0-5889f6f4ff-r76rq"
Oct 31 01:09:19.850110 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:19.849004    4284 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-4jv2z" podStartSLOduration=1.8489876029999999 podStartE2EDuration="1.848987603s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 01:09:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 01:09:19.846830095 +0000 UTC m=+2473.991837011" watchObservedRunningTime="2026-10-31 01:09:19.848987603 +0000 UTC m=+2473.993994519"
Oct 31 01:09:19.959885 ip-10-0-132-173 kubenswrapper[4284]: W1031 01:09:19.959847    4284 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0e3a22bb_0915_4025_af10_c9c3b01011f1.slice/crio-e21a82c12b20917dd49f2e7fd02e073fd0c13b176b885ee465b98ef4ee9e12ac WatchSource:0}: Error finding container e21a82c12b20917dd49f2e7fd02e073fd0c13b176b885ee465b98ef4ee9e12ac: Status 404 returned error can't find the container with id e21a82c12b20917dd49f2e7fd02e073fd0c13b176b885ee465b98ef4ee9e12ac
Oct 31 01:09:19.964768 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:19.964747    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w4-8fb7f0-796d4996ff-25jt7"]
Oct 31 01:09:20.164418 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:20.164397    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w4-8fb7f0-5889f6f4ff-r76rq"]
Oct 31 01:09:20.165191 ip-10-0-132-173 kubenswrapper[4284]: W1031 01:09:20.165162    4284 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podddda98fe_7ad3_4592_81bb_2b6ce526b882.slice/crio-e11a398c65639729e018aa685b60345eb7903d068f4ecebdf17f358c46603a67 WatchSource:0}: Error finding container e11a398c65639729e018aa685b60345eb7903d068f4ecebdf17f358c46603a67: Status 404 returned error can't find the container with id e11a398c65639729e018aa685b60345eb7903d068f4ecebdf17f358c46603a67
Oct 31 01:09:20.738065 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:20.738026    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w4-8fb7f0-5889f6f4ff-r76rq" event={"ID":"ddda98fe-7ad3-4592-81bb-2b6ce526b882","Type":"ContainerStarted","Data":"e61a9cf1a0f18e656164387b56c018d2c3d74bea74a088f205fbe2f7b5f069e2"}
Oct 31 01:09:20.738282 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:20.738091    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w4-8fb7f0-5889f6f4ff-r76rq"
Oct 31 01:09:20.738282 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:20.738113    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w4-8fb7f0-5889f6f4ff-r76rq" event={"ID":"ddda98fe-7ad3-4592-81bb-2b6ce526b882","Type":"ContainerStarted","Data":"e11a398c65639729e018aa685b60345eb7903d068f4ecebdf17f358c46603a67"}
Oct 31 01:09:20.739545 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:20.739519    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w4-8fb7f0-796d4996ff-25jt7" event={"ID":"0e3a22bb-0915-4025-af10-c9c3b01011f1","Type":"ContainerStarted","Data":"22fb4eaee3719a5c3488797ccf2689eb080cd33bba6ccaed7ef117650b76b950"}
Oct 31 01:09:20.739672 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:20.739562    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w4-8fb7f0-796d4996ff-25jt7" event={"ID":"0e3a22bb-0915-4025-af10-c9c3b01011f1","Type":"ContainerStarted","Data":"e21a82c12b20917dd49f2e7fd02e073fd0c13b176b885ee465b98ef4ee9e12ac"}
Oct 31 01:09:20.739720 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:20.739694    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w4-8fb7f0-796d4996ff-25jt7"
Oct 31 01:09:20.745499 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:20.745479    4284 patch_prober.go:29] interesting pod/e2e-worker-w4-ef2572-openshift-default-7649dd87-4jv2z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.152:15021/healthz/ready\": dial tcp 10.134.0.152:15021: connect: connection refused" start-of-body=
Oct 31 01:09:20.745597 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:20.745520    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-4jv2z" podUID="205f3d84-b321-479b-8931-7584aaf21b1a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.152:15021/healthz/ready\": dial tcp 10.134.0.152:15021: connect: connection refused"
Oct 31 01:09:20.853674 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:20.853611    4284 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-a-w4-8fb7f0-5889f6f4ff-r76rq" podStartSLOduration=1.853596748 podStartE2EDuration="1.853596748s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 01:09:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 01:09:20.851667324 +0000 UTC m=+2474.996674240" watchObservedRunningTime="2026-10-31 01:09:20.853596748 +0000 UTC m=+2474.998603664"
Oct 31 01:09:21.743670 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:21.743636    4284 patch_prober.go:29] interesting pod/e2e-worker-w4-ef2572-openshift-default-7649dd87-4jv2z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.152:15021/healthz/ready\": dial tcp 10.134.0.152:15021: connect: connection refused" start-of-body=
Oct 31 01:09:21.744078 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:21.743687    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-4jv2z" podUID="205f3d84-b321-479b-8931-7584aaf21b1a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.152:15021/healthz/ready\": dial tcp 10.134.0.152:15021: connect: connection refused"
Oct 31 01:09:22.706242 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:22.700936    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:09:22.748058 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:22.744474    4284 patch_prober.go:29] interesting pod/e2e-worker-w4-ef2572-openshift-default-7649dd87-4jv2z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.152:15021/healthz/ready\": dial tcp 10.134.0.152:15021: connect: connection refused" start-of-body=
Oct 31 01:09:22.748058 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:22.744536    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-4jv2z" podUID="205f3d84-b321-479b-8931-7584aaf21b1a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.152:15021/healthz/ready\": dial tcp 10.134.0.152:15021: connect: connection refused"
Oct 31 01:09:23.744652 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:23.744615    4284 patch_prober.go:29] interesting pod/e2e-worker-w4-ef2572-openshift-default-7649dd87-4jv2z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.152:15021/healthz/ready\": dial tcp 10.134.0.152:15021: connect: connection refused" start-of-body=
Oct 31 01:09:23.744878 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:23.744692    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-4jv2z" podUID="205f3d84-b321-479b-8931-7584aaf21b1a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.152:15021/healthz/ready\": dial tcp 10.134.0.152:15021: connect: connection refused"
Oct 31 01:09:24.744004 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:24.743961    4284 patch_prober.go:29] interesting pod/e2e-worker-w4-ef2572-openshift-default-7649dd87-4jv2z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.152:15021/healthz/ready\": dial tcp 10.134.0.152:15021: connect: connection refused" start-of-body=
Oct 31 01:09:24.744416 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:24.744024    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-4jv2z" podUID="205f3d84-b321-479b-8931-7584aaf21b1a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.152:15021/healthz/ready\": dial tcp 10.134.0.152:15021: connect: connection refused"
Oct 31 01:09:25.743776 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:25.743734    4284 patch_prober.go:29] interesting pod/e2e-worker-w4-ef2572-openshift-default-7649dd87-4jv2z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.152:15021/healthz/ready\": dial tcp 10.134.0.152:15021: connect: connection refused" start-of-body=
Oct 31 01:09:25.744009 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:25.743798    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-4jv2z" podUID="205f3d84-b321-479b-8931-7584aaf21b1a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.152:15021/healthz/ready\": dial tcp 10.134.0.152:15021: connect: connection refused"
Oct 31 01:09:25.840629 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:25.840591    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:09:26.598117 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:26.598037    4284 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w4-8fb7f0-796d4996ff-25jt7" podStartSLOduration=7.598016142 podStartE2EDuration="7.598016142s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 01:09:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 01:09:20.981537478 +0000 UTC m=+2475.126544395" watchObservedRunningTime="2026-10-31 01:09:26.598016142 +0000 UTC m=+2480.743023058"
Oct 31 01:09:26.600126 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:26.600101    4284 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-w1-ba34eb-openshift-default-98cfdc4c5-f7gpd"]
Oct 31 01:09:26.605127 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:26.605109    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w1-ba34eb-openshift-default-98cfdc4c5-f7gpd"
Oct 31 01:09:26.690022 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:26.689976    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nsstt\" (UniqueName: \"kubernetes.io/projected/6aa40f46-c2b1-4c37-a341-d230158a7db9-kube-api-access-nsstt\") pod \"e2e-shared-a-w1-ba34eb-openshift-default-98cfdc4c5-f7gpd\" (UID: \"6aa40f46-c2b1-4c37-a341-d230158a7db9\") " pod="openshift-ingress/e2e-shared-a-w1-ba34eb-openshift-default-98cfdc4c5-f7gpd"
Oct 31 01:09:26.690249 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:26.690039    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6aa40f46-c2b1-4c37-a341-d230158a7db9-istio-data\") pod \"e2e-shared-a-w1-ba34eb-openshift-default-98cfdc4c5-f7gpd\" (UID: \"6aa40f46-c2b1-4c37-a341-d230158a7db9\") " pod="openshift-ingress/e2e-shared-a-w1-ba34eb-openshift-default-98cfdc4c5-f7gpd"
Oct 31 01:09:26.690249 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:26.690113    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6aa40f46-c2b1-4c37-a341-d230158a7db9-workload-certs\") pod \"e2e-shared-a-w1-ba34eb-openshift-default-98cfdc4c5-f7gpd\" (UID: \"6aa40f46-c2b1-4c37-a341-d230158a7db9\") " pod="openshift-ingress/e2e-shared-a-w1-ba34eb-openshift-default-98cfdc4c5-f7gpd"
Oct 31 01:09:26.690249 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:26.690142    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6aa40f46-c2b1-4c37-a341-d230158a7db9-istio-envoy\") pod \"e2e-shared-a-w1-ba34eb-openshift-default-98cfdc4c5-f7gpd\" (UID: \"6aa40f46-c2b1-4c37-a341-d230158a7db9\") " pod="openshift-ingress/e2e-shared-a-w1-ba34eb-openshift-default-98cfdc4c5-f7gpd"
Oct 31 01:09:26.690249 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:26.690251    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6aa40f46-c2b1-4c37-a341-d230158a7db9-istio-podinfo\") pod \"e2e-shared-a-w1-ba34eb-openshift-default-98cfdc4c5-f7gpd\" (UID: \"6aa40f46-c2b1-4c37-a341-d230158a7db9\") " pod="openshift-ingress/e2e-shared-a-w1-ba34eb-openshift-default-98cfdc4c5-f7gpd"
Oct 31 01:09:26.690519 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:26.690281    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6aa40f46-c2b1-4c37-a341-d230158a7db9-workload-socket\") pod \"e2e-shared-a-w1-ba34eb-openshift-default-98cfdc4c5-f7gpd\" (UID: \"6aa40f46-c2b1-4c37-a341-d230158a7db9\") " pod="openshift-ingress/e2e-shared-a-w1-ba34eb-openshift-default-98cfdc4c5-f7gpd"
Oct 31 01:09:26.690519 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:26.690302    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6aa40f46-c2b1-4c37-a341-d230158a7db9-credential-socket\") pod \"e2e-shared-a-w1-ba34eb-openshift-default-98cfdc4c5-f7gpd\" (UID: \"6aa40f46-c2b1-4c37-a341-d230158a7db9\") " pod="openshift-ingress/e2e-shared-a-w1-ba34eb-openshift-default-98cfdc4c5-f7gpd"
Oct 31 01:09:26.690519 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:26.690345    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6aa40f46-c2b1-4c37-a341-d230158a7db9-istio-token\") pod \"e2e-shared-a-w1-ba34eb-openshift-default-98cfdc4c5-f7gpd\" (UID: \"6aa40f46-c2b1-4c37-a341-d230158a7db9\") " pod="openshift-ingress/e2e-shared-a-w1-ba34eb-openshift-default-98cfdc4c5-f7gpd"
Oct 31 01:09:26.690519 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:26.690406    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6aa40f46-c2b1-4c37-a341-d230158a7db9-istiod-ca-cert\") pod \"e2e-shared-a-w1-ba34eb-openshift-default-98cfdc4c5-f7gpd\" (UID: \"6aa40f46-c2b1-4c37-a341-d230158a7db9\") " pod="openshift-ingress/e2e-shared-a-w1-ba34eb-openshift-default-98cfdc4c5-f7gpd"
Oct 31 01:09:26.691739 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:26.691685    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:09:26.700450 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:26.700419    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w1-ba34eb-openshift-default-98cfdc4c5-f7gpd"]
Oct 31 01:09:26.711162 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:26.711133    4284 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-w1-ba34eb-openshift-default-dockercfg-vpc8w\""
Oct 31 01:09:26.813886 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:26.813632    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nsstt\" (UniqueName: \"kubernetes.io/projected/6aa40f46-c2b1-4c37-a341-d230158a7db9-kube-api-access-nsstt\") pod \"e2e-shared-a-w1-ba34eb-openshift-default-98cfdc4c5-f7gpd\" (UID: \"6aa40f46-c2b1-4c37-a341-d230158a7db9\") " pod="openshift-ingress/e2e-shared-a-w1-ba34eb-openshift-default-98cfdc4c5-f7gpd"
Oct 31 01:09:26.816993 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:26.816958    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6aa40f46-c2b1-4c37-a341-d230158a7db9-istio-data\") pod \"e2e-shared-a-w1-ba34eb-openshift-default-98cfdc4c5-f7gpd\" (UID: \"6aa40f46-c2b1-4c37-a341-d230158a7db9\") " pod="openshift-ingress/e2e-shared-a-w1-ba34eb-openshift-default-98cfdc4c5-f7gpd"
Oct 31 01:09:26.817250 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:26.817226    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6aa40f46-c2b1-4c37-a341-d230158a7db9-workload-certs\") pod \"e2e-shared-a-w1-ba34eb-openshift-default-98cfdc4c5-f7gpd\" (UID: \"6aa40f46-c2b1-4c37-a341-d230158a7db9\") " pod="openshift-ingress/e2e-shared-a-w1-ba34eb-openshift-default-98cfdc4c5-f7gpd"
Oct 31 01:09:26.817510 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:26.817491    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6aa40f46-c2b1-4c37-a341-d230158a7db9-istio-envoy\") pod \"e2e-shared-a-w1-ba34eb-openshift-default-98cfdc4c5-f7gpd\" (UID: \"6aa40f46-c2b1-4c37-a341-d230158a7db9\") " pod="openshift-ingress/e2e-shared-a-w1-ba34eb-openshift-default-98cfdc4c5-f7gpd"
Oct 31 01:09:26.817727 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:26.817706    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6aa40f46-c2b1-4c37-a341-d230158a7db9-istio-podinfo\") pod \"e2e-shared-a-w1-ba34eb-openshift-default-98cfdc4c5-f7gpd\" (UID: \"6aa40f46-c2b1-4c37-a341-d230158a7db9\") " pod="openshift-ingress/e2e-shared-a-w1-ba34eb-openshift-default-98cfdc4c5-f7gpd"
Oct 31 01:09:26.817880 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:26.817863    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6aa40f46-c2b1-4c37-a341-d230158a7db9-workload-socket\") pod \"e2e-shared-a-w1-ba34eb-openshift-default-98cfdc4c5-f7gpd\" (UID: \"6aa40f46-c2b1-4c37-a341-d230158a7db9\") " pod="openshift-ingress/e2e-shared-a-w1-ba34eb-openshift-default-98cfdc4c5-f7gpd"
Oct 31 01:09:26.818058 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:26.818030    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6aa40f46-c2b1-4c37-a341-d230158a7db9-credential-socket\") pod \"e2e-shared-a-w1-ba34eb-openshift-default-98cfdc4c5-f7gpd\" (UID: \"6aa40f46-c2b1-4c37-a341-d230158a7db9\") " pod="openshift-ingress/e2e-shared-a-w1-ba34eb-openshift-default-98cfdc4c5-f7gpd"
Oct 31 01:09:26.818177 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:26.818128    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6aa40f46-c2b1-4c37-a341-d230158a7db9-istio-token\") pod \"e2e-shared-a-w1-ba34eb-openshift-default-98cfdc4c5-f7gpd\" (UID: \"6aa40f46-c2b1-4c37-a341-d230158a7db9\") " pod="openshift-ingress/e2e-shared-a-w1-ba34eb-openshift-default-98cfdc4c5-f7gpd"
Oct 31 01:09:26.818348 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:26.818264    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6aa40f46-c2b1-4c37-a341-d230158a7db9-istiod-ca-cert\") pod \"e2e-shared-a-w1-ba34eb-openshift-default-98cfdc4c5-f7gpd\" (UID: \"6aa40f46-c2b1-4c37-a341-d230158a7db9\") " pod="openshift-ingress/e2e-shared-a-w1-ba34eb-openshift-default-98cfdc4c5-f7gpd"
Oct 31 01:09:26.819141 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:26.819115    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6aa40f46-c2b1-4c37-a341-d230158a7db9-istiod-ca-cert\") pod \"e2e-shared-a-w1-ba34eb-openshift-default-98cfdc4c5-f7gpd\" (UID: \"6aa40f46-c2b1-4c37-a341-d230158a7db9\") " pod="openshift-ingress/e2e-shared-a-w1-ba34eb-openshift-default-98cfdc4c5-f7gpd"
Oct 31 01:09:26.820611 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:26.820588    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6aa40f46-c2b1-4c37-a341-d230158a7db9-istio-data\") pod \"e2e-shared-a-w1-ba34eb-openshift-default-98cfdc4c5-f7gpd\" (UID: \"6aa40f46-c2b1-4c37-a341-d230158a7db9\") " pod="openshift-ingress/e2e-shared-a-w1-ba34eb-openshift-default-98cfdc4c5-f7gpd"
Oct 31 01:09:26.821004 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:26.820983    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6aa40f46-c2b1-4c37-a341-d230158a7db9-workload-certs\") pod \"e2e-shared-a-w1-ba34eb-openshift-default-98cfdc4c5-f7gpd\" (UID: \"6aa40f46-c2b1-4c37-a341-d230158a7db9\") " pod="openshift-ingress/e2e-shared-a-w1-ba34eb-openshift-default-98cfdc4c5-f7gpd"
Oct 31 01:09:26.821719 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:26.821688    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6aa40f46-c2b1-4c37-a341-d230158a7db9-workload-socket\") pod \"e2e-shared-a-w1-ba34eb-openshift-default-98cfdc4c5-f7gpd\" (UID: \"6aa40f46-c2b1-4c37-a341-d230158a7db9\") " pod="openshift-ingress/e2e-shared-a-w1-ba34eb-openshift-default-98cfdc4c5-f7gpd"
Oct 31 01:09:26.822355 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:26.822329    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6aa40f46-c2b1-4c37-a341-d230158a7db9-credential-socket\") pod \"e2e-shared-a-w1-ba34eb-openshift-default-98cfdc4c5-f7gpd\" (UID: \"6aa40f46-c2b1-4c37-a341-d230158a7db9\") " pod="openshift-ingress/e2e-shared-a-w1-ba34eb-openshift-default-98cfdc4c5-f7gpd"
Oct 31 01:09:26.824354 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:26.824309    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6aa40f46-c2b1-4c37-a341-d230158a7db9-istio-envoy\") pod \"e2e-shared-a-w1-ba34eb-openshift-default-98cfdc4c5-f7gpd\" (UID: \"6aa40f46-c2b1-4c37-a341-d230158a7db9\") " pod="openshift-ingress/e2e-shared-a-w1-ba34eb-openshift-default-98cfdc4c5-f7gpd"
Oct 31 01:09:26.827285 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:26.827259    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6aa40f46-c2b1-4c37-a341-d230158a7db9-istio-podinfo\") pod \"e2e-shared-a-w1-ba34eb-openshift-default-98cfdc4c5-f7gpd\" (UID: \"6aa40f46-c2b1-4c37-a341-d230158a7db9\") " pod="openshift-ingress/e2e-shared-a-w1-ba34eb-openshift-default-98cfdc4c5-f7gpd"
Oct 31 01:09:26.848531 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:26.847692    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-b-3f25cd-587bfbf98-8mgpz"]
Oct 31 01:09:26.848531 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:26.848283    4284 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-3f25cd-587bfbf98-8mgpz" podUID="f9b75bd3-399f-4631-8561-c09222dc8b6f" containerName="maas-api" containerID="cri-o://2fcb392edcc529776efdfdbfe2e28025039db079c79d5c1b3e07c6d37e2dfe1a" gracePeriod=30
Oct 31 01:09:26.902593 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:26.902554    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:09:26.903940 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:26.903883    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6aa40f46-c2b1-4c37-a341-d230158a7db9-istio-token\") pod \"e2e-shared-a-w1-ba34eb-openshift-default-98cfdc4c5-f7gpd\" (UID: \"6aa40f46-c2b1-4c37-a341-d230158a7db9\") " pod="openshift-ingress/e2e-shared-a-w1-ba34eb-openshift-default-98cfdc4c5-f7gpd"
Oct 31 01:09:26.906818 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:26.906765    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nsstt\" (UniqueName: \"kubernetes.io/projected/6aa40f46-c2b1-4c37-a341-d230158a7db9-kube-api-access-nsstt\") pod \"e2e-shared-a-w1-ba34eb-openshift-default-98cfdc4c5-f7gpd\" (UID: \"6aa40f46-c2b1-4c37-a341-d230158a7db9\") " pod="openshift-ingress/e2e-shared-a-w1-ba34eb-openshift-default-98cfdc4c5-f7gpd"
Oct 31 01:09:26.919750 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:26.919486    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w1-ba34eb-openshift-default-98cfdc4c5-f7gpd"
Oct 31 01:09:27.113285 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:27.106527    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:09:27.167535 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:27.167478    4284 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-4jv2z"
Oct 31 01:09:27.168876 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:27.168846    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-4jv2z"
Oct 31 01:09:27.169742 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:27.169723    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-4jv2z"
Oct 31 01:09:27.212767 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:27.212735    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-3f25cd-587bfbf98-8mgpz"
Oct 31 01:09:27.329786 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:27.329754    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f9b75bd3-399f-4631-8561-c09222dc8b6f-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f9b75bd3-399f-4631-8561-c09222dc8b6f-openshift-service-ca\") pod \"f9b75bd3-399f-4631-8561-c09222dc8b6f\" (UID: \"f9b75bd3-399f-4631-8561-c09222dc8b6f\") "
Oct 31 01:09:27.329966 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:27.329806    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/f9b75bd3-399f-4631-8561-c09222dc8b6f-metrics-certs\" (UniqueName: \"kubernetes.io/secret/f9b75bd3-399f-4631-8561-c09222dc8b6f-metrics-certs\") pod \"f9b75bd3-399f-4631-8561-c09222dc8b6f\" (UID: \"f9b75bd3-399f-4631-8561-c09222dc8b6f\") "
Oct 31 01:09:27.329966 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:27.329848    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/f9b75bd3-399f-4631-8561-c09222dc8b6f-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f9b75bd3-399f-4631-8561-c09222dc8b6f-maas-api-tls\") pod \"f9b75bd3-399f-4631-8561-c09222dc8b6f\" (UID: \"f9b75bd3-399f-4631-8561-c09222dc8b6f\") "
Oct 31 01:09:27.329966 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:27.329884    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f9b75bd3-399f-4631-8561-c09222dc8b6f-kube-api-access-t8l7w\" (UniqueName: \"kubernetes.io/projected/f9b75bd3-399f-4631-8561-c09222dc8b6f-kube-api-access-t8l7w\") pod \"f9b75bd3-399f-4631-8561-c09222dc8b6f\" (UID: \"f9b75bd3-399f-4631-8561-c09222dc8b6f\") "
Oct 31 01:09:27.330266 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:27.330206    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f9b75bd3-399f-4631-8561-c09222dc8b6f-openshift-service-ca" pod "f9b75bd3-399f-4631-8561-c09222dc8b6f" (UID: "f9b75bd3-399f-4631-8561-c09222dc8b6f"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 01:09:27.330425 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:27.330409    4284 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f9b75bd3-399f-4631-8561-c09222dc8b6f-openshift-service-ca\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:09:27.332200 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:27.332177    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f9b75bd3-399f-4631-8561-c09222dc8b6f-metrics-certs" pod "f9b75bd3-399f-4631-8561-c09222dc8b6f" (UID: "f9b75bd3-399f-4631-8561-c09222dc8b6f"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 01:09:27.332683 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:27.332657    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f9b75bd3-399f-4631-8561-c09222dc8b6f-maas-api-tls" pod "f9b75bd3-399f-4631-8561-c09222dc8b6f" (UID: "f9b75bd3-399f-4631-8561-c09222dc8b6f"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 01:09:27.332782 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:27.332687    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f9b75bd3-399f-4631-8561-c09222dc8b6f-kube-api-access-t8l7w" pod "f9b75bd3-399f-4631-8561-c09222dc8b6f" (UID: "f9b75bd3-399f-4631-8561-c09222dc8b6f"). InnerVolumeSpecName "kube-api-access-t8l7w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:09:27.362343 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:27.360732    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w1-ba34eb-openshift-default-98cfdc4c5-f7gpd"]
Oct 31 01:09:27.362343 ip-10-0-132-173 kubenswrapper[4284]: W1031 01:09:27.361751    4284 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6aa40f46_c2b1_4c37_a341_d230158a7db9.slice/crio-9c9f0c711b27dc2cd3ad27444a0b333b91f7e109cd583393359e99262d944a90 WatchSource:0}: Error finding container 9c9f0c711b27dc2cd3ad27444a0b333b91f7e109cd583393359e99262d944a90: Status 404 returned error can't find the container with id 9c9f0c711b27dc2cd3ad27444a0b333b91f7e109cd583393359e99262d944a90
Oct 31 01:09:27.364349 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:27.364317    4284 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 01:09:27.364453 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:27.364395    4284 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 01:09:27.364453 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:27.364430    4284 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 01:09:27.431528 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:27.431489    4284 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f9b75bd3-399f-4631-8561-c09222dc8b6f-metrics-certs\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:09:27.431528 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:27.431531    4284 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f9b75bd3-399f-4631-8561-c09222dc8b6f-maas-api-tls\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:09:27.431804 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:27.431545    4284 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t8l7w\" (UniqueName: \"kubernetes.io/projected/f9b75bd3-399f-4631-8561-c09222dc8b6f-kube-api-access-t8l7w\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:09:27.879261 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:27.875436    4284 generic.go:356] "Generic (PLEG): container finished" podID="f9b75bd3-399f-4631-8561-c09222dc8b6f" containerID="2fcb392edcc529776efdfdbfe2e28025039db079c79d5c1b3e07c6d37e2dfe1a" exitCode=0
Oct 31 01:09:27.879261 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:27.875645    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-3f25cd-587bfbf98-8mgpz" event={"ID":"f9b75bd3-399f-4631-8561-c09222dc8b6f","Type":"ContainerDied","Data":"2fcb392edcc529776efdfdbfe2e28025039db079c79d5c1b3e07c6d37e2dfe1a"}
Oct 31 01:09:27.879261 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:27.875680    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-3f25cd-587bfbf98-8mgpz"
Oct 31 01:09:27.879261 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:27.875702    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-3f25cd-587bfbf98-8mgpz" event={"ID":"f9b75bd3-399f-4631-8561-c09222dc8b6f","Type":"ContainerDied","Data":"07463ed13d04a1cfeef8f6bb44d648f16cc7d11e51b9a8a93e4c29b8eeff5c49"}
Oct 31 01:09:27.879261 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:27.875726    4284 scope.go:122] "RemoveContainer" containerID="2fcb392edcc529776efdfdbfe2e28025039db079c79d5c1b3e07c6d37e2dfe1a"
Oct 31 01:09:27.880446 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:27.879917    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w1-ba34eb-openshift-default-98cfdc4c5-f7gpd" event={"ID":"6aa40f46-c2b1-4c37-a341-d230158a7db9","Type":"ContainerStarted","Data":"f4d91cbb71f4601158caf9415ba4d9cb617951c3cda297a3c341c334e23db70b"}
Oct 31 01:09:27.880446 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:27.879974    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w1-ba34eb-openshift-default-98cfdc4c5-f7gpd" event={"ID":"6aa40f46-c2b1-4c37-a341-d230158a7db9","Type":"ContainerStarted","Data":"9c9f0c711b27dc2cd3ad27444a0b333b91f7e109cd583393359e99262d944a90"}
Oct 31 01:09:27.893589 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:27.893494    4284 scope.go:122] "RemoveContainer" containerID="2fcb392edcc529776efdfdbfe2e28025039db079c79d5c1b3e07c6d37e2dfe1a"
Oct 31 01:09:27.893896 ip-10-0-132-173 kubenswrapper[4284]: E1031 01:09:27.893873    4284 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2fcb392edcc529776efdfdbfe2e28025039db079c79d5c1b3e07c6d37e2dfe1a\": container with ID starting with 2fcb392edcc529776efdfdbfe2e28025039db079c79d5c1b3e07c6d37e2dfe1a not found: ID does not exist" containerID="2fcb392edcc529776efdfdbfe2e28025039db079c79d5c1b3e07c6d37e2dfe1a"
Oct 31 01:09:27.893946 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:27.893909    4284 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2fcb392edcc529776efdfdbfe2e28025039db079c79d5c1b3e07c6d37e2dfe1a"} err="failed to get container status \"2fcb392edcc529776efdfdbfe2e28025039db079c79d5c1b3e07c6d37e2dfe1a\": rpc error: code = NotFound desc = could not find container \"2fcb392edcc529776efdfdbfe2e28025039db079c79d5c1b3e07c6d37e2dfe1a\": container with ID starting with 2fcb392edcc529776efdfdbfe2e28025039db079c79d5c1b3e07c6d37e2dfe1a not found: ID does not exist"
Oct 31 01:09:27.923169 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:27.922199    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-praxis-b-9e4801-748ccff66c-r2x25"]
Oct 31 01:09:27.923169 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:27.922316    4284 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w1-ba34eb-openshift-default-98cfdc4c5-f7gpd"
Oct 31 01:09:28.148610 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:28.148524    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-b-3f25cd-587bfbf98-8mgpz"]
Oct 31 01:09:28.151813 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:28.151790    4284 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-b-3f25cd-587bfbf98-8mgpz"]
Oct 31 01:09:28.171610 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:28.171573    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:09:28.268994 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:28.268939    4284 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-w1-ba34eb-openshift-default-98cfdc4c5-f7gpd" podStartSLOduration=2.2689261370000002 podStartE2EDuration="2.268926137s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 01:09:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 01:09:28.265561571 +0000 UTC m=+2482.410568487" watchObservedRunningTime="2026-10-31 01:09:28.268926137 +0000 UTC m=+2482.413933053"
Oct 31 01:09:28.323028 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:28.322986    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:09:28.623177 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:28.623143    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:09:28.793798 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:28.793766    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:09:28.926990 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:28.926960    4284 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w1-ba34eb-openshift-default-98cfdc4c5-f7gpd"
Oct 31 01:09:28.987974 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:28.987932    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:09:29.515653 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:29.515614    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:09:29.894741 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:29.894652    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w1-ba34eb-openshift-default-98cfdc4c5-f7gpd"
Oct 31 01:09:29.895546 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:29.895525    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w1-ba34eb-openshift-default-98cfdc4c5-f7gpd"
Oct 31 01:09:29.962934 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:29.962900    4284 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f9b75bd3-399f-4631-8561-c09222dc8b6f" path="/var/lib/kubelet/pods/f9b75bd3-399f-4631-8561-c09222dc8b6f/volumes"
Oct 31 01:09:31.354352 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:31.354320    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:09:31.529342 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:31.529297    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:09:31.717340 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:31.717301    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:09:31.747699 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:31.747665    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w4-8fb7f0-5889f6f4ff-r76rq"
Oct 31 01:09:31.747846 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:31.747725    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w4-8fb7f0-796d4996ff-25jt7"
Oct 31 01:09:32.849960 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:32.849923    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:09:32.932002 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:32.931924    4284 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-9e4801-748ccff66c-r2x25" podUID="aec30731-ea23-4b49-9d93-36243434cf3d" containerName="payload-pre-processing" containerID="cri-o://8adf85581319abb3ae3103136d9ad913b1c8b482204e96193bf3007cfc4eae4f" gracePeriod=25
Oct 31 01:09:33.251270 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:33.251239    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-9e4801-748ccff66c-r2x25"
Oct 31 01:09:33.387006 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:33.386972    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:09:33.430284 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:33.427637    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/aec30731-ea23-4b49-9d93-36243434cf3d-kube-api-access-lf5p2\" (UniqueName: \"kubernetes.io/projected/aec30731-ea23-4b49-9d93-36243434cf3d-kube-api-access-lf5p2\") pod \"aec30731-ea23-4b49-9d93-36243434cf3d\" (UID: \"aec30731-ea23-4b49-9d93-36243434cf3d\") "
Oct 31 01:09:33.430284 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:33.427847    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/aec30731-ea23-4b49-9d93-36243434cf3d-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/aec30731-ea23-4b49-9d93-36243434cf3d-plugins-config-volume\") pod \"aec30731-ea23-4b49-9d93-36243434cf3d\" (UID: \"aec30731-ea23-4b49-9d93-36243434cf3d\") "
Oct 31 01:09:33.448413 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:33.447640    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aec30731-ea23-4b49-9d93-36243434cf3d-kube-api-access-lf5p2" pod "aec30731-ea23-4b49-9d93-36243434cf3d" (UID: "aec30731-ea23-4b49-9d93-36243434cf3d"). InnerVolumeSpecName "kube-api-access-lf5p2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:09:33.465590 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:33.465552    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/aec30731-ea23-4b49-9d93-36243434cf3d-plugins-config-volume" pod "aec30731-ea23-4b49-9d93-36243434cf3d" (UID: "aec30731-ea23-4b49-9d93-36243434cf3d"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 01:09:33.529855 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:33.529813    4284 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/aec30731-ea23-4b49-9d93-36243434cf3d-plugins-config-volume\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:09:33.529855 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:33.529847    4284 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lf5p2\" (UniqueName: \"kubernetes.io/projected/aec30731-ea23-4b49-9d93-36243434cf3d-kube-api-access-lf5p2\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:09:33.957575 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:33.957540    4284 generic.go:356] "Generic (PLEG): container finished" podID="aec30731-ea23-4b49-9d93-36243434cf3d" containerID="8adf85581319abb3ae3103136d9ad913b1c8b482204e96193bf3007cfc4eae4f" exitCode=0
Oct 31 01:09:33.958022 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:33.957664    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-9e4801-748ccff66c-r2x25"
Oct 31 01:09:33.963762 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:33.963734    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-9e4801-748ccff66c-r2x25" event={"ID":"aec30731-ea23-4b49-9d93-36243434cf3d","Type":"ContainerDied","Data":"8adf85581319abb3ae3103136d9ad913b1c8b482204e96193bf3007cfc4eae4f"}
Oct 31 01:09:33.964039 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:33.963788    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-9e4801-748ccff66c-r2x25" event={"ID":"aec30731-ea23-4b49-9d93-36243434cf3d","Type":"ContainerDied","Data":"68aa9cdfed49f705236c9584394138599f70fef8ce3174f4c046ffb0cd288ff8"}
Oct 31 01:09:33.964039 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:33.963822    4284 scope.go:122] "RemoveContainer" containerID="8adf85581319abb3ae3103136d9ad913b1c8b482204e96193bf3007cfc4eae4f"
Oct 31 01:09:33.973342 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:33.973323    4284 scope.go:122] "RemoveContainer" containerID="8adf85581319abb3ae3103136d9ad913b1c8b482204e96193bf3007cfc4eae4f"
Oct 31 01:09:33.973642 ip-10-0-132-173 kubenswrapper[4284]: E1031 01:09:33.973620    4284 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8adf85581319abb3ae3103136d9ad913b1c8b482204e96193bf3007cfc4eae4f\": container with ID starting with 8adf85581319abb3ae3103136d9ad913b1c8b482204e96193bf3007cfc4eae4f not found: ID does not exist" containerID="8adf85581319abb3ae3103136d9ad913b1c8b482204e96193bf3007cfc4eae4f"
Oct 31 01:09:33.973708 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:33.973650    4284 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8adf85581319abb3ae3103136d9ad913b1c8b482204e96193bf3007cfc4eae4f"} err="failed to get container status \"8adf85581319abb3ae3103136d9ad913b1c8b482204e96193bf3007cfc4eae4f\": rpc error: code = NotFound desc = could not find container \"8adf85581319abb3ae3103136d9ad913b1c8b482204e96193bf3007cfc4eae4f\": container with ID starting with 8adf85581319abb3ae3103136d9ad913b1c8b482204e96193bf3007cfc4eae4f not found: ID does not exist"
Oct 31 01:09:34.115727 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:34.115696    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-praxis-b-9e4801-748ccff66c-r2x25"]
Oct 31 01:09:34.163161 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:34.163127    4284 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-praxis-b-9e4801-748ccff66c-r2x25"]
Oct 31 01:09:34.449226 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:34.449190    4284 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-f1d081-5c766b5c87-46qzr"]
Oct 31 01:09:34.449755 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:34.449742    4284 cpu_manager.go:414] "containerMap: removing container" podUID="f9b75bd3-399f-4631-8561-c09222dc8b6f" containerName="maas-api"
Oct 31 01:09:34.449797 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:34.449758    4284 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f9b75bd3-399f-4631-8561-c09222dc8b6f" containerName="maas-api"
Oct 31 01:09:34.449797 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:34.449777    4284 cpu_manager.go:414] "containerMap: removing container" podUID="aec30731-ea23-4b49-9d93-36243434cf3d" containerName="payload-pre-processing"
Oct 31 01:09:34.449797 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:34.449784    4284 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="aec30731-ea23-4b49-9d93-36243434cf3d" containerName="payload-pre-processing"
Oct 31 01:09:34.449898 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:34.449893    4284 memory_manager.go:365] "RemoveStaleState removing state" podUID="aec30731-ea23-4b49-9d93-36243434cf3d" containerName="payload-pre-processing"
Oct 31 01:09:34.449932 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:34.449906    4284 memory_manager.go:365] "RemoveStaleState removing state" podUID="f9b75bd3-399f-4631-8561-c09222dc8b6f" containerName="maas-api"
Oct 31 01:09:34.454625 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:34.454605    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-f1d081-5c766b5c87-46qzr"
Oct 31 01:09:34.508882 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:34.508855    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-f1d081-5c766b5c87-46qzr"]
Oct 31 01:09:34.513434 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:34.513398    4284 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-a-f1d081\""
Oct 31 01:09:34.539476 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:34.539437    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:09:34.541732 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:34.541710    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2d3da89a-1e5f-41af-95b5-6910bdd24aad-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-f1d081-5c766b5c87-46qzr\" (UID: \"2d3da89a-1e5f-41af-95b5-6910bdd24aad\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-f1d081-5c766b5c87-46qzr"
Oct 31 01:09:34.541911 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:34.541894    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7f9jj\" (UniqueName: \"kubernetes.io/projected/2d3da89a-1e5f-41af-95b5-6910bdd24aad-kube-api-access-7f9jj\") pod \"payload-pre-processing-e2e-ipp-a-f1d081-5c766b5c87-46qzr\" (UID: \"2d3da89a-1e5f-41af-95b5-6910bdd24aad\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-f1d081-5c766b5c87-46qzr"
Oct 31 01:09:34.552451 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:34.552426    4284 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-a-f1d081-dockercfg-7hd6w\""
Oct 31 01:09:34.647236 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:34.646121    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2d3da89a-1e5f-41af-95b5-6910bdd24aad-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-f1d081-5c766b5c87-46qzr\" (UID: \"2d3da89a-1e5f-41af-95b5-6910bdd24aad\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-f1d081-5c766b5c87-46qzr"
Oct 31 01:09:34.647236 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:34.646315    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7f9jj\" (UniqueName: \"kubernetes.io/projected/2d3da89a-1e5f-41af-95b5-6910bdd24aad-kube-api-access-7f9jj\") pod \"payload-pre-processing-e2e-ipp-a-f1d081-5c766b5c87-46qzr\" (UID: \"2d3da89a-1e5f-41af-95b5-6910bdd24aad\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-f1d081-5c766b5c87-46qzr"
Oct 31 01:09:34.662236 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:34.648337    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2d3da89a-1e5f-41af-95b5-6910bdd24aad-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-f1d081-5c766b5c87-46qzr\" (UID: \"2d3da89a-1e5f-41af-95b5-6910bdd24aad\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-f1d081-5c766b5c87-46qzr"
Oct 31 01:09:34.730023 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:34.729903    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7f9jj\" (UniqueName: \"kubernetes.io/projected/2d3da89a-1e5f-41af-95b5-6910bdd24aad-kube-api-access-7f9jj\") pod \"payload-pre-processing-e2e-ipp-a-f1d081-5c766b5c87-46qzr\" (UID: \"2d3da89a-1e5f-41af-95b5-6910bdd24aad\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-f1d081-5c766b5c87-46qzr"
Oct 31 01:09:34.731878 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:34.731516    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:09:34.768030 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:34.767993    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-f1d081-5c766b5c87-46qzr"
Oct 31 01:09:35.141828 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:35.141796    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-f1d081-5c766b5c87-46qzr"]
Oct 31 01:09:35.142573 ip-10-0-132-173 kubenswrapper[4284]: W1031 01:09:35.142546    4284 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2d3da89a_1e5f_41af_95b5_6910bdd24aad.slice/crio-a5107c91b37aaf29b4f9d8f73af7be9513b1b273989139a99522c27d53965cb5 WatchSource:0}: Error finding container a5107c91b37aaf29b4f9d8f73af7be9513b1b273989139a99522c27d53965cb5: Status 404 returned error can't find the container with id a5107c91b37aaf29b4f9d8f73af7be9513b1b273989139a99522c27d53965cb5
Oct 31 01:09:35.277251 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:35.276594    4284 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-f1d081-8bc8c89c8-j7s55"]
Oct 31 01:09:35.313715 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:35.313685    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-f1d081-8bc8c89c8-j7s55"
Oct 31 01:09:35.324713 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:35.324688    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-f1d081-8bc8c89c8-j7s55"]
Oct 31 01:09:35.462306 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:35.462263    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wh5hk\" (UniqueName: \"kubernetes.io/projected/6a061135-19b6-49e7-8917-b3a9a4274488-kube-api-access-wh5hk\") pod \"payload-processing-e2e-ipp-a-f1d081-8bc8c89c8-j7s55\" (UID: \"6a061135-19b6-49e7-8917-b3a9a4274488\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-f1d081-8bc8c89c8-j7s55"
Oct 31 01:09:35.462498 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:35.462338    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6a061135-19b6-49e7-8917-b3a9a4274488-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-f1d081-8bc8c89c8-j7s55\" (UID: \"6a061135-19b6-49e7-8917-b3a9a4274488\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-f1d081-8bc8c89c8-j7s55"
Oct 31 01:09:35.564187 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:35.564143    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wh5hk\" (UniqueName: \"kubernetes.io/projected/6a061135-19b6-49e7-8917-b3a9a4274488-kube-api-access-wh5hk\") pod \"payload-processing-e2e-ipp-a-f1d081-8bc8c89c8-j7s55\" (UID: \"6a061135-19b6-49e7-8917-b3a9a4274488\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-f1d081-8bc8c89c8-j7s55"
Oct 31 01:09:35.564431 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:35.564207    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6a061135-19b6-49e7-8917-b3a9a4274488-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-f1d081-8bc8c89c8-j7s55\" (UID: \"6a061135-19b6-49e7-8917-b3a9a4274488\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-f1d081-8bc8c89c8-j7s55"
Oct 31 01:09:35.565001 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:35.564977    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6a061135-19b6-49e7-8917-b3a9a4274488-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-f1d081-8bc8c89c8-j7s55\" (UID: \"6a061135-19b6-49e7-8917-b3a9a4274488\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-f1d081-8bc8c89c8-j7s55"
Oct 31 01:09:35.623247 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:35.623181    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wh5hk\" (UniqueName: \"kubernetes.io/projected/6a061135-19b6-49e7-8917-b3a9a4274488-kube-api-access-wh5hk\") pod \"payload-processing-e2e-ipp-a-f1d081-8bc8c89c8-j7s55\" (UID: \"6a061135-19b6-49e7-8917-b3a9a4274488\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-f1d081-8bc8c89c8-j7s55"
Oct 31 01:09:35.648107 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:35.648074    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-f1d081-8bc8c89c8-j7s55"
Oct 31 01:09:35.963481 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:35.963446    4284 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="aec30731-ea23-4b49-9d93-36243434cf3d" path="/var/lib/kubelet/pods/aec30731-ea23-4b49-9d93-36243434cf3d/volumes"
Oct 31 01:09:35.973696 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:35.973666    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-f1d081-5c766b5c87-46qzr" event={"ID":"2d3da89a-1e5f-41af-95b5-6910bdd24aad","Type":"ContainerStarted","Data":"388a516c0044a468282de7173c84f5f0fffb739e3f990814333f9a4eca30fe85"}
Oct 31 01:09:35.973885 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:35.973712    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-f1d081-5c766b5c87-46qzr" event={"ID":"2d3da89a-1e5f-41af-95b5-6910bdd24aad","Type":"ContainerStarted","Data":"a5107c91b37aaf29b4f9d8f73af7be9513b1b273989139a99522c27d53965cb5"}
Oct 31 01:09:35.973885 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:35.973743    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-f1d081-5c766b5c87-46qzr"
Oct 31 01:09:36.023247 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:36.023182    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-f1d081-8bc8c89c8-j7s55"]
Oct 31 01:09:36.030277 ip-10-0-132-173 kubenswrapper[4284]: W1031 01:09:36.030185    4284 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6a061135_19b6_49e7_8917_b3a9a4274488.slice/crio-9329510bf7ac884c3337da0964c9f4348d2dbf402fd09c97c934c8f1b9ef6743 WatchSource:0}: Error finding container 9329510bf7ac884c3337da0964c9f4348d2dbf402fd09c97c934c8f1b9ef6743: Status 404 returned error can't find the container with id 9329510bf7ac884c3337da0964c9f4348d2dbf402fd09c97c934c8f1b9ef6743
Oct 31 01:09:36.086320 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:36.086260    4284 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-f1d081-5c766b5c87-46qzr" podStartSLOduration=2.086241277 podStartE2EDuration="2.086241277s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 01:09:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 01:09:36.08550161 +0000 UTC m=+2490.230508526" watchObservedRunningTime="2026-10-31 01:09:36.086241277 +0000 UTC m=+2490.231248193"
Oct 31 01:09:36.241190 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:36.241092    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:09:36.751671 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:36.751628    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:09:36.912931 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:36.912895    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:09:36.981730 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:36.981695    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-f1d081-8bc8c89c8-j7s55" event={"ID":"6a061135-19b6-49e7-8917-b3a9a4274488","Type":"ContainerStarted","Data":"312431d8e47135034e52b59c2e1580ad8f5f326404a208ce1394d5f3c7fd1b9e"}
Oct 31 01:09:36.981948 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:36.981753    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-f1d081-8bc8c89c8-j7s55" event={"ID":"6a061135-19b6-49e7-8917-b3a9a4274488","Type":"ContainerStarted","Data":"9329510bf7ac884c3337da0964c9f4348d2dbf402fd09c97c934c8f1b9ef6743"}
Oct 31 01:09:36.981948 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:36.981787    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-f1d081-8bc8c89c8-j7s55"
Oct 31 01:09:37.125009 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:37.124894    4284 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-a-f1d081-8bc8c89c8-j7s55" podStartSLOduration=2.124873911 podStartE2EDuration="2.124873911s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 01:09:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 01:09:37.114485143 +0000 UTC m=+2491.259492057" watchObservedRunningTime="2026-10-31 01:09:37.124873911 +0000 UTC m=+2491.269880830"
Oct 31 01:09:37.173937 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:37.173899    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-praxis-b-3f25cd-openshift-default-576b49df5c-j48jb"]
Oct 31 01:09:37.174201 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:37.174175    4284 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-praxis-b-3f25cd-openshift-default-576b49df5c-j48jb" podUID="f8091219-d375-4a3c-9bea-5178857c8aeb" containerName="istio-proxy" containerID="cri-o://1cc87c9890541d3e71e42db728eb4eed630e21c8eaae733163e52aaa7ebcd128" gracePeriod=30
Oct 31 01:09:37.216898 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:37.216860    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:09:41.573654 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:41.573622    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:09:42.446706 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:42.446644    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-praxis-b-3f25cd-openshift-default-576b49df5c-j48jb"
Oct 31 01:09:42.564667 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:42.564627    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f8091219-d375-4a3c-9bea-5178857c8aeb-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f8091219-d375-4a3c-9bea-5178857c8aeb-istio-envoy\") pod \"f8091219-d375-4a3c-9bea-5178857c8aeb\" (UID: \"f8091219-d375-4a3c-9bea-5178857c8aeb\") "
Oct 31 01:09:42.564874 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:42.564755    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f8091219-d375-4a3c-9bea-5178857c8aeb-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f8091219-d375-4a3c-9bea-5178857c8aeb-istiod-ca-cert\") pod \"f8091219-d375-4a3c-9bea-5178857c8aeb\" (UID: \"f8091219-d375-4a3c-9bea-5178857c8aeb\") "
Oct 31 01:09:42.564874 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:42.564803    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f8091219-d375-4a3c-9bea-5178857c8aeb-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f8091219-d375-4a3c-9bea-5178857c8aeb-workload-certs\") pod \"f8091219-d375-4a3c-9bea-5178857c8aeb\" (UID: \"f8091219-d375-4a3c-9bea-5178857c8aeb\") "
Oct 31 01:09:42.564995 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:42.564874    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f8091219-d375-4a3c-9bea-5178857c8aeb-kube-api-access-hk8tm\" (UniqueName: \"kubernetes.io/projected/f8091219-d375-4a3c-9bea-5178857c8aeb-kube-api-access-hk8tm\") pod \"f8091219-d375-4a3c-9bea-5178857c8aeb\" (UID: \"f8091219-d375-4a3c-9bea-5178857c8aeb\") "
Oct 31 01:09:42.564995 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:42.564917    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f8091219-d375-4a3c-9bea-5178857c8aeb-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f8091219-d375-4a3c-9bea-5178857c8aeb-istio-data\") pod \"f8091219-d375-4a3c-9bea-5178857c8aeb\" (UID: \"f8091219-d375-4a3c-9bea-5178857c8aeb\") "
Oct 31 01:09:42.564995 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:42.564987    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f8091219-d375-4a3c-9bea-5178857c8aeb-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f8091219-d375-4a3c-9bea-5178857c8aeb-credential-socket\") pod \"f8091219-d375-4a3c-9bea-5178857c8aeb\" (UID: \"f8091219-d375-4a3c-9bea-5178857c8aeb\") "
Oct 31 01:09:42.565145 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:42.565039    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f8091219-d375-4a3c-9bea-5178857c8aeb-istio-token\" (UniqueName: \"kubernetes.io/projected/f8091219-d375-4a3c-9bea-5178857c8aeb-istio-token\") pod \"f8091219-d375-4a3c-9bea-5178857c8aeb\" (UID: \"f8091219-d375-4a3c-9bea-5178857c8aeb\") "
Oct 31 01:09:42.565145 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:42.565101    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/f8091219-d375-4a3c-9bea-5178857c8aeb-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f8091219-d375-4a3c-9bea-5178857c8aeb-istio-podinfo\") pod \"f8091219-d375-4a3c-9bea-5178857c8aeb\" (UID: \"f8091219-d375-4a3c-9bea-5178857c8aeb\") "
Oct 31 01:09:42.565145 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:42.565140    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f8091219-d375-4a3c-9bea-5178857c8aeb-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f8091219-d375-4a3c-9bea-5178857c8aeb-workload-socket\") pod \"f8091219-d375-4a3c-9bea-5178857c8aeb\" (UID: \"f8091219-d375-4a3c-9bea-5178857c8aeb\") "
Oct 31 01:09:42.565363 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:42.565158    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f8091219-d375-4a3c-9bea-5178857c8aeb-workload-certs" pod "f8091219-d375-4a3c-9bea-5178857c8aeb" (UID: "f8091219-d375-4a3c-9bea-5178857c8aeb"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:09:42.565363 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:42.565180    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f8091219-d375-4a3c-9bea-5178857c8aeb-istiod-ca-cert" pod "f8091219-d375-4a3c-9bea-5178857c8aeb" (UID: "f8091219-d375-4a3c-9bea-5178857c8aeb"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 01:09:42.565363 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:42.565320    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f8091219-d375-4a3c-9bea-5178857c8aeb-istio-data" pod "f8091219-d375-4a3c-9bea-5178857c8aeb" (UID: "f8091219-d375-4a3c-9bea-5178857c8aeb"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:09:42.565521 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:42.565385    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f8091219-d375-4a3c-9bea-5178857c8aeb-credential-socket" pod "f8091219-d375-4a3c-9bea-5178857c8aeb" (UID: "f8091219-d375-4a3c-9bea-5178857c8aeb"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:09:42.565521 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:42.565506    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f8091219-d375-4a3c-9bea-5178857c8aeb-workload-socket" pod "f8091219-d375-4a3c-9bea-5178857c8aeb" (UID: "f8091219-d375-4a3c-9bea-5178857c8aeb"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:09:42.565793 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:42.565712    4284 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f8091219-d375-4a3c-9bea-5178857c8aeb-istiod-ca-cert\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:09:42.565793 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:42.565736    4284 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f8091219-d375-4a3c-9bea-5178857c8aeb-workload-certs\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:09:42.565793 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:42.565749    4284 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f8091219-d375-4a3c-9bea-5178857c8aeb-istio-data\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:09:42.565793 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:42.565758    4284 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f8091219-d375-4a3c-9bea-5178857c8aeb-credential-socket\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:09:42.565793 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:42.565766    4284 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f8091219-d375-4a3c-9bea-5178857c8aeb-workload-socket\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:09:42.567419 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:42.567391    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f8091219-d375-4a3c-9bea-5178857c8aeb-istio-token" pod "f8091219-d375-4a3c-9bea-5178857c8aeb" (UID: "f8091219-d375-4a3c-9bea-5178857c8aeb"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:09:42.567691 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:42.567669    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/f8091219-d375-4a3c-9bea-5178857c8aeb-istio-podinfo" pod "f8091219-d375-4a3c-9bea-5178857c8aeb" (UID: "f8091219-d375-4a3c-9bea-5178857c8aeb"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 01:09:42.567763 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:42.567725    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f8091219-d375-4a3c-9bea-5178857c8aeb-istio-envoy" pod "f8091219-d375-4a3c-9bea-5178857c8aeb" (UID: "f8091219-d375-4a3c-9bea-5178857c8aeb"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:09:42.567802 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:42.567769    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f8091219-d375-4a3c-9bea-5178857c8aeb-kube-api-access-hk8tm" pod "f8091219-d375-4a3c-9bea-5178857c8aeb" (UID: "f8091219-d375-4a3c-9bea-5178857c8aeb"). InnerVolumeSpecName "kube-api-access-hk8tm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:09:42.666905 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:42.666873    4284 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f8091219-d375-4a3c-9bea-5178857c8aeb-istio-token\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:09:42.666905 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:42.666898    4284 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f8091219-d375-4a3c-9bea-5178857c8aeb-istio-podinfo\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:09:42.666905 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:42.666908    4284 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f8091219-d375-4a3c-9bea-5178857c8aeb-istio-envoy\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:09:42.667434 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:42.666917    4284 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hk8tm\" (UniqueName: \"kubernetes.io/projected/f8091219-d375-4a3c-9bea-5178857c8aeb-kube-api-access-hk8tm\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:09:43.031110 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:43.031077    4284 generic.go:356] "Generic (PLEG): container finished" podID="f8091219-d375-4a3c-9bea-5178857c8aeb" containerID="1cc87c9890541d3e71e42db728eb4eed630e21c8eaae733163e52aaa7ebcd128" exitCode=0
Oct 31 01:09:43.031314 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:43.031136    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-praxis-b-3f25cd-openshift-default-576b49df5c-j48jb"
Oct 31 01:09:43.031314 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:43.031154    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-b-3f25cd-openshift-default-576b49df5c-j48jb" event={"ID":"f8091219-d375-4a3c-9bea-5178857c8aeb","Type":"ContainerDied","Data":"1cc87c9890541d3e71e42db728eb4eed630e21c8eaae733163e52aaa7ebcd128"}
Oct 31 01:09:43.031314 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:43.031224    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-b-3f25cd-openshift-default-576b49df5c-j48jb" event={"ID":"f8091219-d375-4a3c-9bea-5178857c8aeb","Type":"ContainerDied","Data":"7fa37467df5b70967318730a1ea4c09b8098850ef4977d9c8c2acf36f35881bf"}
Oct 31 01:09:43.031314 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:43.031251    4284 scope.go:122] "RemoveContainer" containerID="1cc87c9890541d3e71e42db728eb4eed630e21c8eaae733163e52aaa7ebcd128"
Oct 31 01:09:43.041788 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:43.041771    4284 scope.go:122] "RemoveContainer" containerID="8afeb0d90d1ab3373c5f42e8c8c1f6805667a9feb4fc80a449c18d332cdafa8b"
Oct 31 01:09:43.051951 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:43.051935    4284 scope.go:122] "RemoveContainer" containerID="1cc87c9890541d3e71e42db728eb4eed630e21c8eaae733163e52aaa7ebcd128"
Oct 31 01:09:43.052267 ip-10-0-132-173 kubenswrapper[4284]: E1031 01:09:43.052239    4284 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1cc87c9890541d3e71e42db728eb4eed630e21c8eaae733163e52aaa7ebcd128\": container with ID starting with 1cc87c9890541d3e71e42db728eb4eed630e21c8eaae733163e52aaa7ebcd128 not found: ID does not exist" containerID="1cc87c9890541d3e71e42db728eb4eed630e21c8eaae733163e52aaa7ebcd128"
Oct 31 01:09:43.052336 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:43.052278    4284 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1cc87c9890541d3e71e42db728eb4eed630e21c8eaae733163e52aaa7ebcd128"} err="failed to get container status \"1cc87c9890541d3e71e42db728eb4eed630e21c8eaae733163e52aaa7ebcd128\": rpc error: code = NotFound desc = could not find container \"1cc87c9890541d3e71e42db728eb4eed630e21c8eaae733163e52aaa7ebcd128\": container with ID starting with 1cc87c9890541d3e71e42db728eb4eed630e21c8eaae733163e52aaa7ebcd128 not found: ID does not exist"
Oct 31 01:09:43.052336 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:43.052298    4284 scope.go:122] "RemoveContainer" containerID="8afeb0d90d1ab3373c5f42e8c8c1f6805667a9feb4fc80a449c18d332cdafa8b"
Oct 31 01:09:43.052582 ip-10-0-132-173 kubenswrapper[4284]: E1031 01:09:43.052565    4284 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8afeb0d90d1ab3373c5f42e8c8c1f6805667a9feb4fc80a449c18d332cdafa8b\": container with ID starting with 8afeb0d90d1ab3373c5f42e8c8c1f6805667a9feb4fc80a449c18d332cdafa8b not found: ID does not exist" containerID="8afeb0d90d1ab3373c5f42e8c8c1f6805667a9feb4fc80a449c18d332cdafa8b"
Oct 31 01:09:43.052634 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:43.052587    4284 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8afeb0d90d1ab3373c5f42e8c8c1f6805667a9feb4fc80a449c18d332cdafa8b"} err="failed to get container status \"8afeb0d90d1ab3373c5f42e8c8c1f6805667a9feb4fc80a449c18d332cdafa8b\": rpc error: code = NotFound desc = could not find container \"8afeb0d90d1ab3373c5f42e8c8c1f6805667a9feb4fc80a449c18d332cdafa8b\": container with ID starting with 8afeb0d90d1ab3373c5f42e8c8c1f6805667a9feb4fc80a449c18d332cdafa8b not found: ID does not exist"
Oct 31 01:09:43.190115 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:43.190086    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-praxis-b-3f25cd-openshift-default-576b49df5c-j48jb"]
Oct 31 01:09:43.237634 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:43.237598    4284 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-praxis-b-3f25cd-openshift-default-576b49df5c-j48jb"]
Oct 31 01:09:43.962875 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:43.962841    4284 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f8091219-d375-4a3c-9bea-5178857c8aeb" path="/var/lib/kubelet/pods/f8091219-d375-4a3c-9bea-5178857c8aeb/volumes"
Oct 31 01:09:46.212405 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:46.212369    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:09:46.743646 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:46.743611    4284 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-w2-4397fc-openshift-default-6b458f575f-tvgdj"]
Oct 31 01:09:46.744199 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:46.744185    4284 cpu_manager.go:414] "containerMap: removing container" podUID="f8091219-d375-4a3c-9bea-5178857c8aeb" containerName="istio-proxy"
Oct 31 01:09:46.744282 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:46.744201    4284 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f8091219-d375-4a3c-9bea-5178857c8aeb" containerName="istio-proxy"
Oct 31 01:09:46.744333 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:46.744321    4284 memory_manager.go:365] "RemoveStaleState removing state" podUID="f8091219-d375-4a3c-9bea-5178857c8aeb" containerName="istio-proxy"
Oct 31 01:09:46.748954 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:46.748934    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-4397fc-openshift-default-6b458f575f-tvgdj"
Oct 31 01:09:46.876758 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:46.876718    4284 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-w2-4397fc-openshift-default-dockercfg-8vmv4\""
Oct 31 01:09:46.879704 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:46.879668    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:09:46.880862 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:46.880816    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fd46495b-8c51-476d-91f9-750529060bbd-credential-socket\") pod \"e2e-shared-a-w2-4397fc-openshift-default-6b458f575f-tvgdj\" (UID: \"fd46495b-8c51-476d-91f9-750529060bbd\") " pod="openshift-ingress/e2e-shared-a-w2-4397fc-openshift-default-6b458f575f-tvgdj"
Oct 31 01:09:46.881135 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:46.881108    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fd46495b-8c51-476d-91f9-750529060bbd-workload-certs\") pod \"e2e-shared-a-w2-4397fc-openshift-default-6b458f575f-tvgdj\" (UID: \"fd46495b-8c51-476d-91f9-750529060bbd\") " pod="openshift-ingress/e2e-shared-a-w2-4397fc-openshift-default-6b458f575f-tvgdj"
Oct 31 01:09:46.881278 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:46.881158    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fd46495b-8c51-476d-91f9-750529060bbd-workload-socket\") pod \"e2e-shared-a-w2-4397fc-openshift-default-6b458f575f-tvgdj\" (UID: \"fd46495b-8c51-476d-91f9-750529060bbd\") " pod="openshift-ingress/e2e-shared-a-w2-4397fc-openshift-default-6b458f575f-tvgdj"
Oct 31 01:09:46.881278 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:46.881185    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fd46495b-8c51-476d-91f9-750529060bbd-istio-data\") pod \"e2e-shared-a-w2-4397fc-openshift-default-6b458f575f-tvgdj\" (UID: \"fd46495b-8c51-476d-91f9-750529060bbd\") " pod="openshift-ingress/e2e-shared-a-w2-4397fc-openshift-default-6b458f575f-tvgdj"
Oct 31 01:09:46.881403 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:46.881344    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fd46495b-8c51-476d-91f9-750529060bbd-istio-podinfo\") pod \"e2e-shared-a-w2-4397fc-openshift-default-6b458f575f-tvgdj\" (UID: \"fd46495b-8c51-476d-91f9-750529060bbd\") " pod="openshift-ingress/e2e-shared-a-w2-4397fc-openshift-default-6b458f575f-tvgdj"
Oct 31 01:09:46.881403 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:46.881391    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fd46495b-8c51-476d-91f9-750529060bbd-istiod-ca-cert\") pod \"e2e-shared-a-w2-4397fc-openshift-default-6b458f575f-tvgdj\" (UID: \"fd46495b-8c51-476d-91f9-750529060bbd\") " pod="openshift-ingress/e2e-shared-a-w2-4397fc-openshift-default-6b458f575f-tvgdj"
Oct 31 01:09:46.881513 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:46.881461    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fd46495b-8c51-476d-91f9-750529060bbd-istio-token\") pod \"e2e-shared-a-w2-4397fc-openshift-default-6b458f575f-tvgdj\" (UID: \"fd46495b-8c51-476d-91f9-750529060bbd\") " pod="openshift-ingress/e2e-shared-a-w2-4397fc-openshift-default-6b458f575f-tvgdj"
Oct 31 01:09:46.881564 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:46.881509    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jcbsm\" (UniqueName: \"kubernetes.io/projected/fd46495b-8c51-476d-91f9-750529060bbd-kube-api-access-jcbsm\") pod \"e2e-shared-a-w2-4397fc-openshift-default-6b458f575f-tvgdj\" (UID: \"fd46495b-8c51-476d-91f9-750529060bbd\") " pod="openshift-ingress/e2e-shared-a-w2-4397fc-openshift-default-6b458f575f-tvgdj"
Oct 31 01:09:46.881619 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:46.881588    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fd46495b-8c51-476d-91f9-750529060bbd-istio-envoy\") pod \"e2e-shared-a-w2-4397fc-openshift-default-6b458f575f-tvgdj\" (UID: \"fd46495b-8c51-476d-91f9-750529060bbd\") " pod="openshift-ingress/e2e-shared-a-w2-4397fc-openshift-default-6b458f575f-tvgdj"
Oct 31 01:09:46.886395 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:46.886369    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-4397fc-openshift-default-6b458f575f-tvgdj"]
Oct 31 01:09:46.983024 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:46.982981    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fd46495b-8c51-476d-91f9-750529060bbd-credential-socket\") pod \"e2e-shared-a-w2-4397fc-openshift-default-6b458f575f-tvgdj\" (UID: \"fd46495b-8c51-476d-91f9-750529060bbd\") " pod="openshift-ingress/e2e-shared-a-w2-4397fc-openshift-default-6b458f575f-tvgdj"
Oct 31 01:09:46.983251 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:46.983036    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fd46495b-8c51-476d-91f9-750529060bbd-workload-certs\") pod \"e2e-shared-a-w2-4397fc-openshift-default-6b458f575f-tvgdj\" (UID: \"fd46495b-8c51-476d-91f9-750529060bbd\") " pod="openshift-ingress/e2e-shared-a-w2-4397fc-openshift-default-6b458f575f-tvgdj"
Oct 31 01:09:46.983251 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:46.983073    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fd46495b-8c51-476d-91f9-750529060bbd-workload-socket\") pod \"e2e-shared-a-w2-4397fc-openshift-default-6b458f575f-tvgdj\" (UID: \"fd46495b-8c51-476d-91f9-750529060bbd\") " pod="openshift-ingress/e2e-shared-a-w2-4397fc-openshift-default-6b458f575f-tvgdj"
Oct 31 01:09:46.983251 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:46.983098    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fd46495b-8c51-476d-91f9-750529060bbd-istio-data\") pod \"e2e-shared-a-w2-4397fc-openshift-default-6b458f575f-tvgdj\" (UID: \"fd46495b-8c51-476d-91f9-750529060bbd\") " pod="openshift-ingress/e2e-shared-a-w2-4397fc-openshift-default-6b458f575f-tvgdj"
Oct 31 01:09:46.983251 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:46.983151    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fd46495b-8c51-476d-91f9-750529060bbd-istio-podinfo\") pod \"e2e-shared-a-w2-4397fc-openshift-default-6b458f575f-tvgdj\" (UID: \"fd46495b-8c51-476d-91f9-750529060bbd\") " pod="openshift-ingress/e2e-shared-a-w2-4397fc-openshift-default-6b458f575f-tvgdj"
Oct 31 01:09:46.983251 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:46.983190    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fd46495b-8c51-476d-91f9-750529060bbd-istiod-ca-cert\") pod \"e2e-shared-a-w2-4397fc-openshift-default-6b458f575f-tvgdj\" (UID: \"fd46495b-8c51-476d-91f9-750529060bbd\") " pod="openshift-ingress/e2e-shared-a-w2-4397fc-openshift-default-6b458f575f-tvgdj"
Oct 31 01:09:46.983536 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:46.983274    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fd46495b-8c51-476d-91f9-750529060bbd-istio-token\") pod \"e2e-shared-a-w2-4397fc-openshift-default-6b458f575f-tvgdj\" (UID: \"fd46495b-8c51-476d-91f9-750529060bbd\") " pod="openshift-ingress/e2e-shared-a-w2-4397fc-openshift-default-6b458f575f-tvgdj"
Oct 31 01:09:46.983536 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:46.983326    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jcbsm\" (UniqueName: \"kubernetes.io/projected/fd46495b-8c51-476d-91f9-750529060bbd-kube-api-access-jcbsm\") pod \"e2e-shared-a-w2-4397fc-openshift-default-6b458f575f-tvgdj\" (UID: \"fd46495b-8c51-476d-91f9-750529060bbd\") " pod="openshift-ingress/e2e-shared-a-w2-4397fc-openshift-default-6b458f575f-tvgdj"
Oct 31 01:09:46.983536 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:46.983401    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fd46495b-8c51-476d-91f9-750529060bbd-istio-envoy\") pod \"e2e-shared-a-w2-4397fc-openshift-default-6b458f575f-tvgdj\" (UID: \"fd46495b-8c51-476d-91f9-750529060bbd\") " pod="openshift-ingress/e2e-shared-a-w2-4397fc-openshift-default-6b458f575f-tvgdj"
Oct 31 01:09:46.983536 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:46.983443    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fd46495b-8c51-476d-91f9-750529060bbd-credential-socket\") pod \"e2e-shared-a-w2-4397fc-openshift-default-6b458f575f-tvgdj\" (UID: \"fd46495b-8c51-476d-91f9-750529060bbd\") " pod="openshift-ingress/e2e-shared-a-w2-4397fc-openshift-default-6b458f575f-tvgdj"
Oct 31 01:09:46.983536 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:46.983505    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fd46495b-8c51-476d-91f9-750529060bbd-workload-certs\") pod \"e2e-shared-a-w2-4397fc-openshift-default-6b458f575f-tvgdj\" (UID: \"fd46495b-8c51-476d-91f9-750529060bbd\") " pod="openshift-ingress/e2e-shared-a-w2-4397fc-openshift-default-6b458f575f-tvgdj"
Oct 31 01:09:46.984065 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:46.984026    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fd46495b-8c51-476d-91f9-750529060bbd-istiod-ca-cert\") pod \"e2e-shared-a-w2-4397fc-openshift-default-6b458f575f-tvgdj\" (UID: \"fd46495b-8c51-476d-91f9-750529060bbd\") " pod="openshift-ingress/e2e-shared-a-w2-4397fc-openshift-default-6b458f575f-tvgdj"
Oct 31 01:09:46.984140 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:46.984102    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-f1d081-5c766b5c87-46qzr"
Oct 31 01:09:46.984734 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:46.984708    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fd46495b-8c51-476d-91f9-750529060bbd-istio-data\") pod \"e2e-shared-a-w2-4397fc-openshift-default-6b458f575f-tvgdj\" (UID: \"fd46495b-8c51-476d-91f9-750529060bbd\") " pod="openshift-ingress/e2e-shared-a-w2-4397fc-openshift-default-6b458f575f-tvgdj"
Oct 31 01:09:46.985010 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:46.984989    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fd46495b-8c51-476d-91f9-750529060bbd-workload-socket\") pod \"e2e-shared-a-w2-4397fc-openshift-default-6b458f575f-tvgdj\" (UID: \"fd46495b-8c51-476d-91f9-750529060bbd\") " pod="openshift-ingress/e2e-shared-a-w2-4397fc-openshift-default-6b458f575f-tvgdj"
Oct 31 01:09:46.986761 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:46.986738    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fd46495b-8c51-476d-91f9-750529060bbd-istio-podinfo\") pod \"e2e-shared-a-w2-4397fc-openshift-default-6b458f575f-tvgdj\" (UID: \"fd46495b-8c51-476d-91f9-750529060bbd\") " pod="openshift-ingress/e2e-shared-a-w2-4397fc-openshift-default-6b458f575f-tvgdj"
Oct 31 01:09:46.987132 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:46.987116    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fd46495b-8c51-476d-91f9-750529060bbd-istio-envoy\") pod \"e2e-shared-a-w2-4397fc-openshift-default-6b458f575f-tvgdj\" (UID: \"fd46495b-8c51-476d-91f9-750529060bbd\") " pod="openshift-ingress/e2e-shared-a-w2-4397fc-openshift-default-6b458f575f-tvgdj"
Oct 31 01:09:47.013789 ip-10-0-132-173 kubenswrapper[4284]: E1031 01:09:47.013711    4284 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fd8sp" containerName="extractor"
Oct 31 01:09:47.029970 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:47.029935    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:09:47.042361 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:47.042328    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fd46495b-8c51-476d-91f9-750529060bbd-istio-token\") pod \"e2e-shared-a-w2-4397fc-openshift-default-6b458f575f-tvgdj\" (UID: \"fd46495b-8c51-476d-91f9-750529060bbd\") " pod="openshift-ingress/e2e-shared-a-w2-4397fc-openshift-default-6b458f575f-tvgdj"
Oct 31 01:09:47.042496 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:47.042374    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jcbsm\" (UniqueName: \"kubernetes.io/projected/fd46495b-8c51-476d-91f9-750529060bbd-kube-api-access-jcbsm\") pod \"e2e-shared-a-w2-4397fc-openshift-default-6b458f575f-tvgdj\" (UID: \"fd46495b-8c51-476d-91f9-750529060bbd\") " pod="openshift-ingress/e2e-shared-a-w2-4397fc-openshift-default-6b458f575f-tvgdj"
Oct 31 01:09:47.098518 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:47.098484    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-4397fc-openshift-default-6b458f575f-tvgdj"
Oct 31 01:09:47.173669 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:47.173502    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:09:47.525289 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:47.525263    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-4397fc-openshift-default-6b458f575f-tvgdj"]
Oct 31 01:09:47.525804 ip-10-0-132-173 kubenswrapper[4284]: W1031 01:09:47.525780    4284 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfd46495b_8c51_476d_91f9_750529060bbd.slice/crio-b417862b33116ed4bb863f8c3144aef2a173736adbcbf30ca6bf23c5dc0134e2 WatchSource:0}: Error finding container b417862b33116ed4bb863f8c3144aef2a173736adbcbf30ca6bf23c5dc0134e2: Status 404 returned error can't find the container with id b417862b33116ed4bb863f8c3144aef2a173736adbcbf30ca6bf23c5dc0134e2
Oct 31 01:09:47.528546 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:47.528517    4284 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 01:09:47.528623 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:47.528593    4284 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 01:09:47.528672 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:47.528632    4284 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 01:09:47.989877 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:47.989849    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-f1d081-8bc8c89c8-j7s55"
Oct 31 01:09:48.063036 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:48.062995    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-4397fc-openshift-default-6b458f575f-tvgdj" event={"ID":"fd46495b-8c51-476d-91f9-750529060bbd","Type":"ContainerStarted","Data":"1c21af988df8b2ba6536445f69f14974cbba7dbb136bb74eb436fca4bf48fefe"}
Oct 31 01:09:48.063229 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:48.063047    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-4397fc-openshift-default-6b458f575f-tvgdj" event={"ID":"fd46495b-8c51-476d-91f9-750529060bbd","Type":"ContainerStarted","Data":"b417862b33116ed4bb863f8c3144aef2a173736adbcbf30ca6bf23c5dc0134e2"}
Oct 31 01:09:48.098953 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:48.098919    4284 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w2-4397fc-openshift-default-6b458f575f-tvgdj"
Oct 31 01:09:48.105270 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:48.105243    4284 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w2-4397fc-openshift-default-6b458f575f-tvgdj"
Oct 31 01:09:48.222324 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:48.222268    4284 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-w2-4397fc-openshift-default-6b458f575f-tvgdj" podStartSLOduration=2.222249474 podStartE2EDuration="2.222249474s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 01:09:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 01:09:48.220765702 +0000 UTC m=+2502.365772618" watchObservedRunningTime="2026-10-31 01:09:48.222249474 +0000 UTC m=+2502.367256391"
Oct 31 01:09:49.078246 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:49.078183    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w2-4397fc-openshift-default-6b458f575f-tvgdj"
Oct 31 01:09:49.079089 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:49.079071    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w2-4397fc-openshift-default-6b458f575f-tvgdj"
Oct 31 01:09:50.840707 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:50.840668    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:09:51.045556 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:51.045511    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:09:51.222434 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:51.222402    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:09:51.371679 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:51.371644    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:09:51.603360 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:51.603275    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:09:51.814237 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:51.810357    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:09:53.267062 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:53.267028    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:09:53.820250 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:53.820197    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:09:54.102710 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:54.102622    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:09:54.251928 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:54.251892    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:09:54.411206 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:54.411169    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:09:54.849398 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:54.849088    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:09:56.678290 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:56.678250    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:09:56.878674 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:56.878640    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:09:57.023911 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:57.023813    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:09:58.351041 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:58.351009    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:09:58.494937 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:58.494896    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:09:58.773780 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:09:58.773741    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:10:01.141225 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:10:01.141184    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:10:01.287170 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:10:01.287079    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:10:01.571169 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:10:01.571081    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:10:02.997037 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:10:02.997003    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:10:03.443358 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:10:03.443323    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:10:03.585801 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:10:03.585765    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:10:03.741376 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:10:03.739925    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:10:03.831820 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:10:03.831786    4284 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-66e263-56774d5b9f-g6gxl"]
Oct 31 01:10:03.832356 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:10:03.832341    4284 cpu_manager.go:414] "containerMap: removing container" podUID="f8091219-d375-4a3c-9bea-5178857c8aeb" containerName="istio-proxy"
Oct 31 01:10:03.832405 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:10:03.832358    4284 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f8091219-d375-4a3c-9bea-5178857c8aeb" containerName="istio-proxy"
Oct 31 01:10:03.832448 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:10:03.832439    4284 memory_manager.go:365] "RemoveStaleState removing state" podUID="f8091219-d375-4a3c-9bea-5178857c8aeb" containerName="istio-proxy"
Oct 31 01:10:03.835899 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:10:03.835872    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-66e263-56774d5b9f-g6gxl"
Oct 31 01:10:03.891758 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:10:03.891718    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qb57d\" (UniqueName: \"kubernetes.io/projected/854ab963-5fa1-4b6e-be4c-52ffce0e3e15-kube-api-access-qb57d\") pod \"payload-pre-processing-e2e-ipp-b-66e263-56774d5b9f-g6gxl\" (UID: \"854ab963-5fa1-4b6e-be4c-52ffce0e3e15\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-66e263-56774d5b9f-g6gxl"
Oct 31 01:10:03.891956 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:10:03.891877    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/854ab963-5fa1-4b6e-be4c-52ffce0e3e15-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-66e263-56774d5b9f-g6gxl\" (UID: \"854ab963-5fa1-4b6e-be4c-52ffce0e3e15\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-66e263-56774d5b9f-g6gxl"
Oct 31 01:10:03.966255 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:10:03.958158    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-66e263-56774d5b9f-g6gxl"]
Oct 31 01:10:03.979630 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:10:03.979597    4284 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-b-66e263-dockercfg-xfz4n\""
Oct 31 01:10:03.980029 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:10:03.979991    4284 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-b-66e263\""
Oct 31 01:10:04.000604 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:10:04.000489    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qb57d\" (UniqueName: \"kubernetes.io/projected/854ab963-5fa1-4b6e-be4c-52ffce0e3e15-kube-api-access-qb57d\") pod \"payload-pre-processing-e2e-ipp-b-66e263-56774d5b9f-g6gxl\" (UID: \"854ab963-5fa1-4b6e-be4c-52ffce0e3e15\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-66e263-56774d5b9f-g6gxl"
Oct 31 01:10:04.004747 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:10:04.004322    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/854ab963-5fa1-4b6e-be4c-52ffce0e3e15-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-66e263-56774d5b9f-g6gxl\" (UID: \"854ab963-5fa1-4b6e-be4c-52ffce0e3e15\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-66e263-56774d5b9f-g6gxl"
Oct 31 01:10:04.005965 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:10:04.005942    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/854ab963-5fa1-4b6e-be4c-52ffce0e3e15-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-66e263-56774d5b9f-g6gxl\" (UID: \"854ab963-5fa1-4b6e-be4c-52ffce0e3e15\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-66e263-56774d5b9f-g6gxl"
Oct 31 01:10:04.064666 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:10:04.064633    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qb57d\" (UniqueName: \"kubernetes.io/projected/854ab963-5fa1-4b6e-be4c-52ffce0e3e15-kube-api-access-qb57d\") pod \"payload-pre-processing-e2e-ipp-b-66e263-56774d5b9f-g6gxl\" (UID: \"854ab963-5fa1-4b6e-be4c-52ffce0e3e15\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-66e263-56774d5b9f-g6gxl"
Oct 31 01:10:04.150267 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:10:04.150200    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-66e263-56774d5b9f-g6gxl"
Oct 31 01:10:04.527789 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:10:04.527748    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-66e263-56774d5b9f-g6gxl"]
Oct 31 01:10:04.529060 ip-10-0-132-173 kubenswrapper[4284]: W1031 01:10:04.529030    4284 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod854ab963_5fa1_4b6e_be4c_52ffce0e3e15.slice/crio-54ac89a274d381a3d088b339f656a16161a86104e05fa722af52c5f167e21f86 WatchSource:0}: Error finding container 54ac89a274d381a3d088b339f656a16161a86104e05fa722af52c5f167e21f86: Status 404 returned error can't find the container with id 54ac89a274d381a3d088b339f656a16161a86104e05fa722af52c5f167e21f86
Oct 31 01:10:05.181977 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:10:05.181938    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-66e263-56774d5b9f-g6gxl" event={"ID":"854ab963-5fa1-4b6e-be4c-52ffce0e3e15","Type":"ContainerStarted","Data":"2c64179fc0965283b57b6804aeb9217b44c594728d57ab968ab04e889be5e590"}
Oct 31 01:10:05.182504 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:10:05.182000    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-66e263-56774d5b9f-g6gxl" event={"ID":"854ab963-5fa1-4b6e-be4c-52ffce0e3e15","Type":"ContainerStarted","Data":"54ac89a274d381a3d088b339f656a16161a86104e05fa722af52c5f167e21f86"}
Oct 31 01:10:05.182504 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:10:05.182029    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-66e263-56774d5b9f-g6gxl"
Oct 31 01:10:05.295908 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:10:05.295858    4284 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-66e263-56774d5b9f-g6gxl" podStartSLOduration=2.295842646 podStartE2EDuration="2.295842646s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 01:10:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 01:10:05.293949651 +0000 UTC m=+2519.438956565" watchObservedRunningTime="2026-10-31 01:10:05.295842646 +0000 UTC m=+2519.440849591"
Oct 31 01:10:08.372008 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:10:08.371968    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:10:13.237166 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:10:13.237128    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:10:13.481035 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:10:13.480994    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:10:13.922400 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:10:13.914858    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:10:14.174112 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:10:14.174025    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:10:16.192906 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:10:16.192875    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-66e263-56774d5b9f-g6gxl"
Oct 31 01:10:17.013998 ip-10-0-132-173 kubenswrapper[4284]: E1031 01:10:17.013962    4284 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fd8sp" containerName="extractor"
Oct 31 01:10:18.322916 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:10:18.322880    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:10:18.485238 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:10:18.485171    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:10:18.640009 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:10:18.639910    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:10:19.942461 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:10:19.942430    4284 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-ba34eb-6c74885c96-8smj9"]
Oct 31 01:10:19.947677 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:10:19.947656    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-ba34eb-6c74885c96-8smj9"
Oct 31 01:10:19.998621 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:10:19.998589    4284 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-w1-ba34eb-serving-cert\""
Oct 31 01:10:20.002482 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:10:20.002455    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-ba34eb-6c74885c96-8smj9"]
Oct 31 01:10:20.014345 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:10:20.014320    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a11c7b2d-901c-4c69-9be2-a0bd6eeaf9c0-metrics-certs\") pod \"maas-api-e2e-shared-a-w1-ba34eb-6c74885c96-8smj9\" (UID: \"a11c7b2d-901c-4c69-9be2-a0bd6eeaf9c0\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-ba34eb-6c74885c96-8smj9"
Oct 31 01:10:20.014518 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:10:20.014490    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a11c7b2d-901c-4c69-9be2-a0bd6eeaf9c0-maas-api-tls\") pod \"maas-api-e2e-shared-a-w1-ba34eb-6c74885c96-8smj9\" (UID: \"a11c7b2d-901c-4c69-9be2-a0bd6eeaf9c0\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-ba34eb-6c74885c96-8smj9"
Oct 31 01:10:20.014622 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:10:20.014604    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a11c7b2d-901c-4c69-9be2-a0bd6eeaf9c0-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w1-ba34eb-6c74885c96-8smj9\" (UID: \"a11c7b2d-901c-4c69-9be2-a0bd6eeaf9c0\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-ba34eb-6c74885c96-8smj9"
Oct 31 01:10:20.014689 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:10:20.014629    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zjxd6\" (UniqueName: \"kubernetes.io/projected/a11c7b2d-901c-4c69-9be2-a0bd6eeaf9c0-kube-api-access-zjxd6\") pod \"maas-api-e2e-shared-a-w1-ba34eb-6c74885c96-8smj9\" (UID: \"a11c7b2d-901c-4c69-9be2-a0bd6eeaf9c0\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-ba34eb-6c74885c96-8smj9"
Oct 31 01:10:20.116416 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:10:20.116357    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a11c7b2d-901c-4c69-9be2-a0bd6eeaf9c0-maas-api-tls\") pod \"maas-api-e2e-shared-a-w1-ba34eb-6c74885c96-8smj9\" (UID: \"a11c7b2d-901c-4c69-9be2-a0bd6eeaf9c0\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-ba34eb-6c74885c96-8smj9"
Oct 31 01:10:20.116849 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:10:20.116478    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a11c7b2d-901c-4c69-9be2-a0bd6eeaf9c0-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w1-ba34eb-6c74885c96-8smj9\" (UID: \"a11c7b2d-901c-4c69-9be2-a0bd6eeaf9c0\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-ba34eb-6c74885c96-8smj9"
Oct 31 01:10:20.116849 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:10:20.116508    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zjxd6\" (UniqueName: \"kubernetes.io/projected/a11c7b2d-901c-4c69-9be2-a0bd6eeaf9c0-kube-api-access-zjxd6\") pod \"maas-api-e2e-shared-a-w1-ba34eb-6c74885c96-8smj9\" (UID: \"a11c7b2d-901c-4c69-9be2-a0bd6eeaf9c0\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-ba34eb-6c74885c96-8smj9"
Oct 31 01:10:20.116849 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:10:20.116660    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a11c7b2d-901c-4c69-9be2-a0bd6eeaf9c0-metrics-certs\") pod \"maas-api-e2e-shared-a-w1-ba34eb-6c74885c96-8smj9\" (UID: \"a11c7b2d-901c-4c69-9be2-a0bd6eeaf9c0\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-ba34eb-6c74885c96-8smj9"
Oct 31 01:10:20.117307 ip-10-0-132-173 kubenswrapper[4284]: E1031 01:10:20.117287    4284 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-ba34eb-serving-cert: secret "maas-api-e2e-shared-a-w1-ba34eb-serving-cert" not found
Oct 31 01:10:20.117413 ip-10-0-132-173 kubenswrapper[4284]: E1031 01:10:20.117362    4284 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a11c7b2d-901c-4c69-9be2-a0bd6eeaf9c0-maas-api-tls podName:a11c7b2d-901c-4c69-9be2-a0bd6eeaf9c0 nodeName:}" failed. No retries permitted until 2026-10-31 01:10:20.617340671 +0000 UTC m=+2534.762347564 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/a11c7b2d-901c-4c69-9be2-a0bd6eeaf9c0-maas-api-tls") pod "maas-api-e2e-shared-a-w1-ba34eb-6c74885c96-8smj9" (UID: "a11c7b2d-901c-4c69-9be2-a0bd6eeaf9c0") : secret "maas-api-e2e-shared-a-w1-ba34eb-serving-cert" not found
Oct 31 01:10:20.118184 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:10:20.118160    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a11c7b2d-901c-4c69-9be2-a0bd6eeaf9c0-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w1-ba34eb-6c74885c96-8smj9\" (UID: \"a11c7b2d-901c-4c69-9be2-a0bd6eeaf9c0\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-ba34eb-6c74885c96-8smj9"
Oct 31 01:10:20.122772 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:10:20.122748    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a11c7b2d-901c-4c69-9be2-a0bd6eeaf9c0-metrics-certs\") pod \"maas-api-e2e-shared-a-w1-ba34eb-6c74885c96-8smj9\" (UID: \"a11c7b2d-901c-4c69-9be2-a0bd6eeaf9c0\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-ba34eb-6c74885c96-8smj9"
Oct 31 01:10:20.177072 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:10:20.177029    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:10:20.177483 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:10:20.177466    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zjxd6\" (UniqueName: \"kubernetes.io/projected/a11c7b2d-901c-4c69-9be2-a0bd6eeaf9c0-kube-api-access-zjxd6\") pod \"maas-api-e2e-shared-a-w1-ba34eb-6c74885c96-8smj9\" (UID: \"a11c7b2d-901c-4c69-9be2-a0bd6eeaf9c0\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-ba34eb-6c74885c96-8smj9"
Oct 31 01:10:20.356959 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:10:20.356865    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:10:20.510003 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:10:20.509970    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:10:20.638358 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:10:20.638274    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a11c7b2d-901c-4c69-9be2-a0bd6eeaf9c0-maas-api-tls\") pod \"maas-api-e2e-shared-a-w1-ba34eb-6c74885c96-8smj9\" (UID: \"a11c7b2d-901c-4c69-9be2-a0bd6eeaf9c0\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-ba34eb-6c74885c96-8smj9"
Oct 31 01:10:20.640848 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:10:20.640828    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a11c7b2d-901c-4c69-9be2-a0bd6eeaf9c0-maas-api-tls\") pod \"maas-api-e2e-shared-a-w1-ba34eb-6c74885c96-8smj9\" (UID: \"a11c7b2d-901c-4c69-9be2-a0bd6eeaf9c0\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-ba34eb-6c74885c96-8smj9"
Oct 31 01:10:20.861449 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:10:20.861410    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-ba34eb-6c74885c96-8smj9"
Oct 31 01:10:21.235434 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:10:21.235404    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-ba34eb-6c74885c96-8smj9"]
Oct 31 01:10:21.235927 ip-10-0-132-173 kubenswrapper[4284]: W1031 01:10:21.235899    4284 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda11c7b2d_901c_4c69_9be2_a0bd6eeaf9c0.slice/crio-fbd7bed1dae58dc6212172f8cef516caaa17f99dab18254f456a5deb01130fe7 WatchSource:0}: Error finding container fbd7bed1dae58dc6212172f8cef516caaa17f99dab18254f456a5deb01130fe7: Status 404 returned error can't find the container with id fbd7bed1dae58dc6212172f8cef516caaa17f99dab18254f456a5deb01130fe7
Oct 31 01:10:21.284491 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:10:21.284455    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-ba34eb-6c74885c96-8smj9" event={"ID":"a11c7b2d-901c-4c69-9be2-a0bd6eeaf9c0","Type":"ContainerStarted","Data":"fbd7bed1dae58dc6212172f8cef516caaa17f99dab18254f456a5deb01130fe7"}
Oct 31 01:10:22.291960 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:10:22.291925    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-ba34eb-6c74885c96-8smj9" event={"ID":"a11c7b2d-901c-4c69-9be2-a0bd6eeaf9c0","Type":"ContainerStarted","Data":"46a2b477abe13c9c91c8aa486dd689a73a9eb40e0667fbd36e846a708023d212"}
Oct 31 01:10:22.292453 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:10:22.291986    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-ba34eb-6c74885c96-8smj9"
Oct 31 01:10:22.445453 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:10:22.445355    4284 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-ba34eb-6c74885c96-8smj9" podStartSLOduration=2.515387333 podStartE2EDuration="3.445341532s" totalImagesPullingTime="929.954199ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 01:10:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 01:10:22.443206801 +0000 UTC m=+2536.588213716" watchObservedRunningTime="2026-10-31 01:10:22.445341532 +0000 UTC m=+2536.590348447"
Oct 31 01:10:23.223272 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:10:23.223230    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:10:23.405283 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:10:23.400977    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:10:23.814240 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:10:23.812900    4284 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w0-cb9a07-d74f7465c-pcn8h"]
Oct 31 01:10:23.833234 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:10:23.825234    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w0-cb9a07-d74f7465c-pcn8h"
Oct 31 01:10:23.876724 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:10:23.876696    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w0-cb9a07-d74f7465c-pcn8h"]
Oct 31 01:10:23.905799 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:10:23.905764    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vcsgw\" (UniqueName: \"kubernetes.io/projected/293f3856-d42d-456a-afad-a0a2b0b60081-kube-api-access-vcsgw\") pod \"payload-processing-e2e-shared-a-w0-cb9a07-d74f7465c-pcn8h\" (UID: \"293f3856-d42d-456a-afad-a0a2b0b60081\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w0-cb9a07-d74f7465c-pcn8h"
Oct 31 01:10:23.905983 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:10:23.905823    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/293f3856-d42d-456a-afad-a0a2b0b60081-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w0-cb9a07-d74f7465c-pcn8h\" (UID: \"293f3856-d42d-456a-afad-a0a2b0b60081\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w0-cb9a07-d74f7465c-pcn8h"
Oct 31 01:10:23.919394 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:10:23.919366    4284 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-w0-cb9a07-dockercfg-6hlhw\""
Oct 31 01:10:23.919509 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:10:23.919410    4284 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w0-cb9a07\""
Oct 31 01:10:24.006883 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:10:24.006854    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vcsgw\" (UniqueName: \"kubernetes.io/projected/293f3856-d42d-456a-afad-a0a2b0b60081-kube-api-access-vcsgw\") pod \"payload-processing-e2e-shared-a-w0-cb9a07-d74f7465c-pcn8h\" (UID: \"293f3856-d42d-456a-afad-a0a2b0b60081\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w0-cb9a07-d74f7465c-pcn8h"
Oct 31 01:10:24.007070 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:10:24.006898    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/293f3856-d42d-456a-afad-a0a2b0b60081-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w0-cb9a07-d74f7465c-pcn8h\" (UID: \"293f3856-d42d-456a-afad-a0a2b0b60081\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w0-cb9a07-d74f7465c-pcn8h"
Oct 31 01:10:24.007655 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:10:24.007633    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/293f3856-d42d-456a-afad-a0a2b0b60081-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w0-cb9a07-d74f7465c-pcn8h\" (UID: \"293f3856-d42d-456a-afad-a0a2b0b60081\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w0-cb9a07-d74f7465c-pcn8h"
Oct 31 01:10:24.065657 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:10:24.065593    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vcsgw\" (UniqueName: \"kubernetes.io/projected/293f3856-d42d-456a-afad-a0a2b0b60081-kube-api-access-vcsgw\") pod \"payload-processing-e2e-shared-a-w0-cb9a07-d74f7465c-pcn8h\" (UID: \"293f3856-d42d-456a-afad-a0a2b0b60081\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w0-cb9a07-d74f7465c-pcn8h"
Oct 31 01:10:24.148667 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:10:24.148635    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w0-cb9a07-d74f7465c-pcn8h"
Oct 31 01:10:24.521022 ip-10-0-132-173 kubenswrapper[4284]: W1031 01:10:24.520991    4284 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod293f3856_d42d_456a_afad_a0a2b0b60081.slice/crio-63c4e3686dd159e7f6f71fc71b61ed6ad6a6638fd8b07817228f43e14a335f22 WatchSource:0}: Error finding container 63c4e3686dd159e7f6f71fc71b61ed6ad6a6638fd8b07817228f43e14a335f22: Status 404 returned error can't find the container with id 63c4e3686dd159e7f6f71fc71b61ed6ad6a6638fd8b07817228f43e14a335f22
Oct 31 01:10:24.521403 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:10:24.521378    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w0-cb9a07-d74f7465c-pcn8h"]
Oct 31 01:10:25.311937 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:10:25.311900    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w0-cb9a07-d74f7465c-pcn8h" event={"ID":"293f3856-d42d-456a-afad-a0a2b0b60081","Type":"ContainerStarted","Data":"9f7347989cb631dfc6fb8a5e54ab00796b3730cc15e9f67b70d160d25e1a620d"}
Oct 31 01:10:25.312134 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:10:25.311956    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w0-cb9a07-d74f7465c-pcn8h" event={"ID":"293f3856-d42d-456a-afad-a0a2b0b60081","Type":"ContainerStarted","Data":"63c4e3686dd159e7f6f71fc71b61ed6ad6a6638fd8b07817228f43e14a335f22"}
Oct 31 01:10:25.312134 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:10:25.312105    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w0-cb9a07-d74f7465c-pcn8h"
Oct 31 01:10:25.472661 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:10:25.472598    4284 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-a-w0-cb9a07-d74f7465c-pcn8h" podStartSLOduration=2.472581277 podStartE2EDuration="2.472581277s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 01:10:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 01:10:25.423329454 +0000 UTC m=+2539.568336371" watchObservedRunningTime="2026-10-31 01:10:25.472581277 +0000 UTC m=+2539.617588215"
Oct 31 01:10:25.570292 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:10:25.570175    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:10:28.314581 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:10:28.314551    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-ba34eb-6c74885c96-8smj9"
Oct 31 01:10:30.946894 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:10:30.946861    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:10:35.885558 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:10:35.885515    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:10:36.320758 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:10:36.320721    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w0-cb9a07-d74f7465c-pcn8h"
Oct 31 01:10:37.382499 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:10:37.382403    4284 generic.go:356] "Generic (PLEG): container finished" podID="8af02a59-8229-4932-a556-bfd6ea4e99e0" containerID="f7aad86c2c59b80bdf1c3ebaeb3464cc25b23ac0e09f031cbae0f32e5a09549a" exitCode=0
Oct 31 01:10:37.382937 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:10:37.382488    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w4-8fb7f0-openshift-default-77b6794b69-bqwdf" event={"ID":"8af02a59-8229-4932-a556-bfd6ea4e99e0","Type":"ContainerDied","Data":"f7aad86c2c59b80bdf1c3ebaeb3464cc25b23ac0e09f031cbae0f32e5a09549a"}
Oct 31 01:10:37.382937 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:10:37.382918    4284 scope.go:122] "RemoveContainer" containerID="f7aad86c2c59b80bdf1c3ebaeb3464cc25b23ac0e09f031cbae0f32e5a09549a"
Oct 31 01:10:37.384575 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:10:37.384550    4284 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 01:10:37.384684 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:10:37.384610    4284 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 01:10:37.384684 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:10:37.384640    4284 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 01:10:38.417115 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:10:38.417080    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w4-8fb7f0-openshift-default-77b6794b69-bqwdf" event={"ID":"8af02a59-8229-4932-a556-bfd6ea4e99e0","Type":"ContainerStarted","Data":"7cb5a2ecaaf5044a6ed8838a063816ef0b8555c12022651fbb86fe5ff0319e4e"}
Oct 31 01:10:39.209412 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:10:39.209362    4284 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w4-8fb7f0-openshift-default-77b6794b69-bqwdf"
Oct 31 01:10:39.210896 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:10:39.210869    4284 patch_prober.go:29] interesting pod/e2e-shared-a-w4-8fb7f0-openshift-default-77b6794b69-bqwdf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.151:15021/healthz/ready\": dial tcp 10.134.0.151:15021: connect: connection refused" start-of-body=
Oct 31 01:10:39.211041 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:10:39.210925    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w4-8fb7f0-openshift-default-77b6794b69-bqwdf" podUID="8af02a59-8229-4932-a556-bfd6ea4e99e0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.151:15021/healthz/ready\": dial tcp 10.134.0.151:15021: connect: connection refused"
Oct 31 01:10:40.209697 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:10:40.209656    4284 patch_prober.go:29] interesting pod/e2e-shared-a-w4-8fb7f0-openshift-default-77b6794b69-bqwdf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.151:15021/healthz/ready\": dial tcp 10.134.0.151:15021: connect: connection refused" start-of-body=
Oct 31 01:10:40.210285 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:10:40.209730    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w4-8fb7f0-openshift-default-77b6794b69-bqwdf" podUID="8af02a59-8229-4932-a556-bfd6ea4e99e0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.151:15021/healthz/ready\": dial tcp 10.134.0.151:15021: connect: connection refused"
Oct 31 01:10:40.559100 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:10:40.559011    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:10:40.714137 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:10:40.714098    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:10:40.865312 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:10:40.864241    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:10:41.209395 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:10:41.209355    4284 patch_prober.go:29] interesting pod/e2e-shared-a-w4-8fb7f0-openshift-default-77b6794b69-bqwdf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.151:15021/healthz/ready\": dial tcp 10.134.0.151:15021: connect: connection refused" start-of-body=
Oct 31 01:10:41.209628 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:10:41.209431    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w4-8fb7f0-openshift-default-77b6794b69-bqwdf" podUID="8af02a59-8229-4932-a556-bfd6ea4e99e0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.151:15021/healthz/ready\": dial tcp 10.134.0.151:15021: connect: connection refused"
Oct 31 01:10:42.209603 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:10:42.209562    4284 patch_prober.go:29] interesting pod/e2e-shared-a-w4-8fb7f0-openshift-default-77b6794b69-bqwdf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.151:15021/healthz/ready\": dial tcp 10.134.0.151:15021: connect: connection refused" start-of-body=
Oct 31 01:10:42.210098 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:10:42.209622    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w4-8fb7f0-openshift-default-77b6794b69-bqwdf" podUID="8af02a59-8229-4932-a556-bfd6ea4e99e0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.151:15021/healthz/ready\": dial tcp 10.134.0.151:15021: connect: connection refused"
Oct 31 01:10:43.207236 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:10:43.206695    4284 patch_prober.go:29] interesting pod/e2e-shared-a-w4-8fb7f0-openshift-default-77b6794b69-bqwdf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.151:15021/healthz/ready\": dial tcp 10.134.0.151:15021: connect: connection refused" start-of-body=
Oct 31 01:10:43.207236 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:10:43.206757    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w4-8fb7f0-openshift-default-77b6794b69-bqwdf" podUID="8af02a59-8229-4932-a556-bfd6ea4e99e0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.151:15021/healthz/ready\": dial tcp 10.134.0.151:15021: connect: connection refused"
Oct 31 01:10:43.616298 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:10:43.616194    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w4-8fb7f0-openshift-default-77b6794b69-bqwdf"
Oct 31 01:10:44.211791 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:10:44.211762    4284 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w4-8fb7f0-openshift-default-77b6794b69-bqwdf"
Oct 31 01:10:44.459582 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:10:44.459553    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w4-8fb7f0-openshift-default-77b6794b69-bqwdf"
Oct 31 01:10:45.589559 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:10:45.589525    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:10:47.014575 ip-10-0-132-173 kubenswrapper[4284]: E1031 01:10:47.014489    4284 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fd8sp" containerName="extractor"
Oct 31 01:10:50.360362 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:10:50.360329    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:10:50.516370 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:10:50.516334    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:10:50.664461 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:10:50.664426    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:10:55.329857 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:10:55.329814    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:10:57.555469 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:10:57.555437    4284 generic.go:356] "Generic (PLEG): container finished" podID="6aa40f46-c2b1-4c37-a341-d230158a7db9" containerID="f4d91cbb71f4601158caf9415ba4d9cb617951c3cda297a3c341c334e23db70b" exitCode=0
Oct 31 01:10:57.555979 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:10:57.555512    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w1-ba34eb-openshift-default-98cfdc4c5-f7gpd" event={"ID":"6aa40f46-c2b1-4c37-a341-d230158a7db9","Type":"ContainerDied","Data":"f4d91cbb71f4601158caf9415ba4d9cb617951c3cda297a3c341c334e23db70b"}
Oct 31 01:10:57.555979 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:10:57.555905    4284 scope.go:122] "RemoveContainer" containerID="f4d91cbb71f4601158caf9415ba4d9cb617951c3cda297a3c341c334e23db70b"
Oct 31 01:10:57.557441 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:10:57.557403    4284 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 01:10:57.557553 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:10:57.557480    4284 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 01:10:57.557553 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:10:57.557518    4284 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 01:10:58.563610 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:10:58.563576    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w1-ba34eb-openshift-default-98cfdc4c5-f7gpd" event={"ID":"6aa40f46-c2b1-4c37-a341-d230158a7db9","Type":"ContainerStarted","Data":"08a063371a0f3fb4e2cf6ac1b9f2a14878be6ec744e71b10aa82481b76895771"}
Oct 31 01:10:58.921725 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:10:58.921691    4284 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w1-ba34eb-openshift-default-98cfdc4c5-f7gpd"
Oct 31 01:10:58.922687 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:10:58.922658    4284 patch_prober.go:29] interesting pod/e2e-shared-a-w1-ba34eb-openshift-default-98cfdc4c5-f7gpd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.155:15021/healthz/ready\": dial tcp 10.134.0.155:15021: connect: connection refused" start-of-body=
Oct 31 01:10:58.922857 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:10:58.922834    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w1-ba34eb-openshift-default-98cfdc4c5-f7gpd" podUID="6aa40f46-c2b1-4c37-a341-d230158a7db9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.155:15021/healthz/ready\": dial tcp 10.134.0.155:15021: connect: connection refused"
Oct 31 01:10:59.900366 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:10:59.900325    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w1-ba34eb-openshift-default-98cfdc4c5-f7gpd"
Oct 31 01:10:59.921590 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:10:59.921552    4284 patch_prober.go:29] interesting pod/e2e-shared-a-w1-ba34eb-openshift-default-98cfdc4c5-f7gpd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.155:15021/healthz/ready\": dial tcp 10.134.0.155:15021: connect: connection refused" start-of-body=
Oct 31 01:10:59.921766 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:10:59.921617    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w1-ba34eb-openshift-default-98cfdc4c5-f7gpd" podUID="6aa40f46-c2b1-4c37-a341-d230158a7db9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.155:15021/healthz/ready\": dial tcp 10.134.0.155:15021: connect: connection refused"
Oct 31 01:11:00.255895 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:11:00.255862    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:11:00.921932 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:11:00.921896    4284 patch_prober.go:29] interesting pod/e2e-shared-a-w1-ba34eb-openshift-default-98cfdc4c5-f7gpd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.155:15021/healthz/ready\": dial tcp 10.134.0.155:15021: connect: connection refused" start-of-body=
Oct 31 01:11:00.922393 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:11:00.921964    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w1-ba34eb-openshift-default-98cfdc4c5-f7gpd" podUID="6aa40f46-c2b1-4c37-a341-d230158a7db9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.155:15021/healthz/ready\": dial tcp 10.134.0.155:15021: connect: connection refused"
Oct 31 01:11:01.922027 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:11:01.921989    4284 patch_prober.go:29] interesting pod/e2e-shared-a-w1-ba34eb-openshift-default-98cfdc4c5-f7gpd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.155:15021/healthz/ready\": dial tcp 10.134.0.155:15021: connect: connection refused" start-of-body=
Oct 31 01:11:01.922483 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:11:01.922051    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w1-ba34eb-openshift-default-98cfdc4c5-f7gpd" podUID="6aa40f46-c2b1-4c37-a341-d230158a7db9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.155:15021/healthz/ready\": dial tcp 10.134.0.155:15021: connect: connection refused"
Oct 31 01:11:02.921587 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:11:02.921548    4284 patch_prober.go:29] interesting pod/e2e-shared-a-w1-ba34eb-openshift-default-98cfdc4c5-f7gpd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.155:15021/healthz/ready\": dial tcp 10.134.0.155:15021: connect: connection refused" start-of-body=
Oct 31 01:11:02.921816 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:11:02.921608    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w1-ba34eb-openshift-default-98cfdc4c5-f7gpd" podUID="6aa40f46-c2b1-4c37-a341-d230158a7db9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.155:15021/healthz/ready\": dial tcp 10.134.0.155:15021: connect: connection refused"
Oct 31 01:11:03.072193 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:11:03.072149    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:11:03.331281 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:11:03.331183    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:11:03.544783 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:11:03.544747    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:11:03.926688 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:11:03.926660    4284 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w1-ba34eb-openshift-default-98cfdc4c5-f7gpd"
Oct 31 01:11:03.927828 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:11:03.927810    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w1-ba34eb-openshift-default-98cfdc4c5-f7gpd"
Oct 31 01:11:04.073409 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:11:04.073373    4284 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-w4-a72754-openshift-default-6dbf8c96d5-c4xcb"]
Oct 31 01:11:04.076838 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:11:04.076809    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w4-a72754-openshift-default-6dbf8c96d5-c4xcb"
Oct 31 01:11:04.208809 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:11:04.208708    4284 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-w4-a72754-openshift-default-dockercfg-wgqr9\""
Oct 31 01:11:04.215147 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:11:04.215111    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w4-a72754-openshift-default-6dbf8c96d5-c4xcb"]
Oct 31 01:11:04.221984 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:11:04.221836    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:11:04.331256 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:11:04.324597    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e6cf830a-f9d2-4d0c-90de-672a81d3a441-istio-podinfo\") pod \"e2e-shared-b-w4-a72754-openshift-default-6dbf8c96d5-c4xcb\" (UID: \"e6cf830a-f9d2-4d0c-90de-672a81d3a441\") " pod="openshift-ingress/e2e-shared-b-w4-a72754-openshift-default-6dbf8c96d5-c4xcb"
Oct 31 01:11:04.331256 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:11:04.324647    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e6cf830a-f9d2-4d0c-90de-672a81d3a441-workload-certs\") pod \"e2e-shared-b-w4-a72754-openshift-default-6dbf8c96d5-c4xcb\" (UID: \"e6cf830a-f9d2-4d0c-90de-672a81d3a441\") " pod="openshift-ingress/e2e-shared-b-w4-a72754-openshift-default-6dbf8c96d5-c4xcb"
Oct 31 01:11:04.331256 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:11:04.324679    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e6cf830a-f9d2-4d0c-90de-672a81d3a441-istiod-ca-cert\") pod \"e2e-shared-b-w4-a72754-openshift-default-6dbf8c96d5-c4xcb\" (UID: \"e6cf830a-f9d2-4d0c-90de-672a81d3a441\") " pod="openshift-ingress/e2e-shared-b-w4-a72754-openshift-default-6dbf8c96d5-c4xcb"
Oct 31 01:11:04.331256 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:11:04.324699    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-svtpt\" (UniqueName: \"kubernetes.io/projected/e6cf830a-f9d2-4d0c-90de-672a81d3a441-kube-api-access-svtpt\") pod \"e2e-shared-b-w4-a72754-openshift-default-6dbf8c96d5-c4xcb\" (UID: \"e6cf830a-f9d2-4d0c-90de-672a81d3a441\") " pod="openshift-ingress/e2e-shared-b-w4-a72754-openshift-default-6dbf8c96d5-c4xcb"
Oct 31 01:11:04.331256 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:11:04.324757    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e6cf830a-f9d2-4d0c-90de-672a81d3a441-istio-data\") pod \"e2e-shared-b-w4-a72754-openshift-default-6dbf8c96d5-c4xcb\" (UID: \"e6cf830a-f9d2-4d0c-90de-672a81d3a441\") " pod="openshift-ingress/e2e-shared-b-w4-a72754-openshift-default-6dbf8c96d5-c4xcb"
Oct 31 01:11:04.331256 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:11:04.324817    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e6cf830a-f9d2-4d0c-90de-672a81d3a441-workload-socket\") pod \"e2e-shared-b-w4-a72754-openshift-default-6dbf8c96d5-c4xcb\" (UID: \"e6cf830a-f9d2-4d0c-90de-672a81d3a441\") " pod="openshift-ingress/e2e-shared-b-w4-a72754-openshift-default-6dbf8c96d5-c4xcb"
Oct 31 01:11:04.331256 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:11:04.324910    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e6cf830a-f9d2-4d0c-90de-672a81d3a441-credential-socket\") pod \"e2e-shared-b-w4-a72754-openshift-default-6dbf8c96d5-c4xcb\" (UID: \"e6cf830a-f9d2-4d0c-90de-672a81d3a441\") " pod="openshift-ingress/e2e-shared-b-w4-a72754-openshift-default-6dbf8c96d5-c4xcb"
Oct 31 01:11:04.331256 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:11:04.324930    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e6cf830a-f9d2-4d0c-90de-672a81d3a441-istio-token\") pod \"e2e-shared-b-w4-a72754-openshift-default-6dbf8c96d5-c4xcb\" (UID: \"e6cf830a-f9d2-4d0c-90de-672a81d3a441\") " pod="openshift-ingress/e2e-shared-b-w4-a72754-openshift-default-6dbf8c96d5-c4xcb"
Oct 31 01:11:04.331256 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:11:04.324961    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e6cf830a-f9d2-4d0c-90de-672a81d3a441-istio-envoy\") pod \"e2e-shared-b-w4-a72754-openshift-default-6dbf8c96d5-c4xcb\" (UID: \"e6cf830a-f9d2-4d0c-90de-672a81d3a441\") " pod="openshift-ingress/e2e-shared-b-w4-a72754-openshift-default-6dbf8c96d5-c4xcb"
Oct 31 01:11:04.426303 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:11:04.426259    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e6cf830a-f9d2-4d0c-90de-672a81d3a441-istiod-ca-cert\") pod \"e2e-shared-b-w4-a72754-openshift-default-6dbf8c96d5-c4xcb\" (UID: \"e6cf830a-f9d2-4d0c-90de-672a81d3a441\") " pod="openshift-ingress/e2e-shared-b-w4-a72754-openshift-default-6dbf8c96d5-c4xcb"
Oct 31 01:11:04.426698 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:11:04.426314    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-svtpt\" (UniqueName: \"kubernetes.io/projected/e6cf830a-f9d2-4d0c-90de-672a81d3a441-kube-api-access-svtpt\") pod \"e2e-shared-b-w4-a72754-openshift-default-6dbf8c96d5-c4xcb\" (UID: \"e6cf830a-f9d2-4d0c-90de-672a81d3a441\") " pod="openshift-ingress/e2e-shared-b-w4-a72754-openshift-default-6dbf8c96d5-c4xcb"
Oct 31 01:11:04.426698 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:11:04.426396    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e6cf830a-f9d2-4d0c-90de-672a81d3a441-istio-data\") pod \"e2e-shared-b-w4-a72754-openshift-default-6dbf8c96d5-c4xcb\" (UID: \"e6cf830a-f9d2-4d0c-90de-672a81d3a441\") " pod="openshift-ingress/e2e-shared-b-w4-a72754-openshift-default-6dbf8c96d5-c4xcb"
Oct 31 01:11:04.426698 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:11:04.426481    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e6cf830a-f9d2-4d0c-90de-672a81d3a441-workload-socket\") pod \"e2e-shared-b-w4-a72754-openshift-default-6dbf8c96d5-c4xcb\" (UID: \"e6cf830a-f9d2-4d0c-90de-672a81d3a441\") " pod="openshift-ingress/e2e-shared-b-w4-a72754-openshift-default-6dbf8c96d5-c4xcb"
Oct 31 01:11:04.443116 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:11:04.439241    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e6cf830a-f9d2-4d0c-90de-672a81d3a441-credential-socket\") pod \"e2e-shared-b-w4-a72754-openshift-default-6dbf8c96d5-c4xcb\" (UID: \"e6cf830a-f9d2-4d0c-90de-672a81d3a441\") " pod="openshift-ingress/e2e-shared-b-w4-a72754-openshift-default-6dbf8c96d5-c4xcb"
Oct 31 01:11:04.443116 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:11:04.439337    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e6cf830a-f9d2-4d0c-90de-672a81d3a441-istio-token\") pod \"e2e-shared-b-w4-a72754-openshift-default-6dbf8c96d5-c4xcb\" (UID: \"e6cf830a-f9d2-4d0c-90de-672a81d3a441\") " pod="openshift-ingress/e2e-shared-b-w4-a72754-openshift-default-6dbf8c96d5-c4xcb"
Oct 31 01:11:04.443116 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:11:04.439395    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e6cf830a-f9d2-4d0c-90de-672a81d3a441-istio-envoy\") pod \"e2e-shared-b-w4-a72754-openshift-default-6dbf8c96d5-c4xcb\" (UID: \"e6cf830a-f9d2-4d0c-90de-672a81d3a441\") " pod="openshift-ingress/e2e-shared-b-w4-a72754-openshift-default-6dbf8c96d5-c4xcb"
Oct 31 01:11:04.443116 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:11:04.439444    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e6cf830a-f9d2-4d0c-90de-672a81d3a441-istio-podinfo\") pod \"e2e-shared-b-w4-a72754-openshift-default-6dbf8c96d5-c4xcb\" (UID: \"e6cf830a-f9d2-4d0c-90de-672a81d3a441\") " pod="openshift-ingress/e2e-shared-b-w4-a72754-openshift-default-6dbf8c96d5-c4xcb"
Oct 31 01:11:04.443116 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:11:04.439479    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e6cf830a-f9d2-4d0c-90de-672a81d3a441-workload-certs\") pod \"e2e-shared-b-w4-a72754-openshift-default-6dbf8c96d5-c4xcb\" (UID: \"e6cf830a-f9d2-4d0c-90de-672a81d3a441\") " pod="openshift-ingress/e2e-shared-b-w4-a72754-openshift-default-6dbf8c96d5-c4xcb"
Oct 31 01:11:04.443116 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:11:04.439949    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e6cf830a-f9d2-4d0c-90de-672a81d3a441-workload-certs\") pod \"e2e-shared-b-w4-a72754-openshift-default-6dbf8c96d5-c4xcb\" (UID: \"e6cf830a-f9d2-4d0c-90de-672a81d3a441\") " pod="openshift-ingress/e2e-shared-b-w4-a72754-openshift-default-6dbf8c96d5-c4xcb"
Oct 31 01:11:04.443116 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:11:04.440197    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e6cf830a-f9d2-4d0c-90de-672a81d3a441-workload-socket\") pod \"e2e-shared-b-w4-a72754-openshift-default-6dbf8c96d5-c4xcb\" (UID: \"e6cf830a-f9d2-4d0c-90de-672a81d3a441\") " pod="openshift-ingress/e2e-shared-b-w4-a72754-openshift-default-6dbf8c96d5-c4xcb"
Oct 31 01:11:04.443116 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:11:04.440460    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e6cf830a-f9d2-4d0c-90de-672a81d3a441-credential-socket\") pod \"e2e-shared-b-w4-a72754-openshift-default-6dbf8c96d5-c4xcb\" (UID: \"e6cf830a-f9d2-4d0c-90de-672a81d3a441\") " pod="openshift-ingress/e2e-shared-b-w4-a72754-openshift-default-6dbf8c96d5-c4xcb"
Oct 31 01:11:04.444328 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:11:04.444051    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e6cf830a-f9d2-4d0c-90de-672a81d3a441-istiod-ca-cert\") pod \"e2e-shared-b-w4-a72754-openshift-default-6dbf8c96d5-c4xcb\" (UID: \"e6cf830a-f9d2-4d0c-90de-672a81d3a441\") " pod="openshift-ingress/e2e-shared-b-w4-a72754-openshift-default-6dbf8c96d5-c4xcb"
Oct 31 01:11:04.473604 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:11:04.458631    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e6cf830a-f9d2-4d0c-90de-672a81d3a441-istio-data\") pod \"e2e-shared-b-w4-a72754-openshift-default-6dbf8c96d5-c4xcb\" (UID: \"e6cf830a-f9d2-4d0c-90de-672a81d3a441\") " pod="openshift-ingress/e2e-shared-b-w4-a72754-openshift-default-6dbf8c96d5-c4xcb"
Oct 31 01:11:04.481238 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:11:04.478070    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e6cf830a-f9d2-4d0c-90de-672a81d3a441-istio-envoy\") pod \"e2e-shared-b-w4-a72754-openshift-default-6dbf8c96d5-c4xcb\" (UID: \"e6cf830a-f9d2-4d0c-90de-672a81d3a441\") " pod="openshift-ingress/e2e-shared-b-w4-a72754-openshift-default-6dbf8c96d5-c4xcb"
Oct 31 01:11:04.481238 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:11:04.480015    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e6cf830a-f9d2-4d0c-90de-672a81d3a441-istio-podinfo\") pod \"e2e-shared-b-w4-a72754-openshift-default-6dbf8c96d5-c4xcb\" (UID: \"e6cf830a-f9d2-4d0c-90de-672a81d3a441\") " pod="openshift-ingress/e2e-shared-b-w4-a72754-openshift-default-6dbf8c96d5-c4xcb"
Oct 31 01:11:04.489559 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:11:04.489517    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:11:04.499686 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:11:04.499655    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e6cf830a-f9d2-4d0c-90de-672a81d3a441-istio-token\") pod \"e2e-shared-b-w4-a72754-openshift-default-6dbf8c96d5-c4xcb\" (UID: \"e6cf830a-f9d2-4d0c-90de-672a81d3a441\") " pod="openshift-ingress/e2e-shared-b-w4-a72754-openshift-default-6dbf8c96d5-c4xcb"
Oct 31 01:11:04.517358 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:11:04.517331    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-svtpt\" (UniqueName: \"kubernetes.io/projected/e6cf830a-f9d2-4d0c-90de-672a81d3a441-kube-api-access-svtpt\") pod \"e2e-shared-b-w4-a72754-openshift-default-6dbf8c96d5-c4xcb\" (UID: \"e6cf830a-f9d2-4d0c-90de-672a81d3a441\") " pod="openshift-ingress/e2e-shared-b-w4-a72754-openshift-default-6dbf8c96d5-c4xcb"
Oct 31 01:11:04.717092 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:11:04.717057    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w4-a72754-openshift-default-6dbf8c96d5-c4xcb"
Oct 31 01:11:05.153382 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:11:05.153357    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w4-a72754-openshift-default-6dbf8c96d5-c4xcb"]
Oct 31 01:11:05.153797 ip-10-0-132-173 kubenswrapper[4284]: W1031 01:11:05.153772    4284 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode6cf830a_f9d2_4d0c_90de_672a81d3a441.slice/crio-bc61762bfb713cbd0c1c9e53988d1065519ef17087c591c15f850bc6ff85e1a8 WatchSource:0}: Error finding container bc61762bfb713cbd0c1c9e53988d1065519ef17087c591c15f850bc6ff85e1a8: Status 404 returned error can't find the container with id bc61762bfb713cbd0c1c9e53988d1065519ef17087c591c15f850bc6ff85e1a8
Oct 31 01:11:05.156525 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:11:05.156488    4284 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 01:11:05.156642 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:11:05.156553    4284 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 01:11:05.156642 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:11:05.156584    4284 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 01:11:05.643606 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:11:05.643564    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w4-a72754-openshift-default-6dbf8c96d5-c4xcb" event={"ID":"e6cf830a-f9d2-4d0c-90de-672a81d3a441","Type":"ContainerStarted","Data":"f159ec40f54ddaa13f0ed4da6afe2d0b0843c937e7b42b9fb4f37a1b49fed326"}
Oct 31 01:11:05.643806 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:11:05.643628    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w4-a72754-openshift-default-6dbf8c96d5-c4xcb" event={"ID":"e6cf830a-f9d2-4d0c-90de-672a81d3a441","Type":"ContainerStarted","Data":"bc61762bfb713cbd0c1c9e53988d1065519ef17087c591c15f850bc6ff85e1a8"}
Oct 31 01:11:05.717583 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:11:05.717542    4284 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w4-a72754-openshift-default-6dbf8c96d5-c4xcb"
Oct 31 01:11:05.731812 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:11:05.731779    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:11:06.723692 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:11:06.723661    4284 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w4-a72754-openshift-default-6dbf8c96d5-c4xcb"
Oct 31 01:11:06.951483 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:11:06.951432    4284 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-w4-a72754-openshift-default-6dbf8c96d5-c4xcb" podStartSLOduration=2.951416338 podStartE2EDuration="2.951416338s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 01:11:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 01:11:05.796623578 +0000 UTC m=+2579.941630533" watchObservedRunningTime="2026-10-31 01:11:06.951416338 +0000 UTC m=+2581.096423232"
Oct 31 01:11:07.655447 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:11:07.655413    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w4-a72754-openshift-default-6dbf8c96d5-c4xcb"
Oct 31 01:11:07.656229 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:11:07.656197    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w4-a72754-openshift-default-6dbf8c96d5-c4xcb"
Oct 31 01:11:08.666139 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:11:08.666100    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:11:08.894369 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:11:08.894271    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:11:11.199365 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:11:11.199319    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:11:11.846986 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:11:11.846948    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:11:12.043036 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:11:12.042161    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:11:15.719356 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:11:15.719320    4284 generic.go:356] "Generic (PLEG): container finished" podID="fd46495b-8c51-476d-91f9-750529060bbd" containerID="1c21af988df8b2ba6536445f69f14974cbba7dbb136bb74eb436fca4bf48fefe" exitCode=0
Oct 31 01:11:15.719815 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:11:15.719392    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-4397fc-openshift-default-6b458f575f-tvgdj" event={"ID":"fd46495b-8c51-476d-91f9-750529060bbd","Type":"ContainerDied","Data":"1c21af988df8b2ba6536445f69f14974cbba7dbb136bb74eb436fca4bf48fefe"}
Oct 31 01:11:15.719869 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:11:15.719849    4284 scope.go:122] "RemoveContainer" containerID="1c21af988df8b2ba6536445f69f14974cbba7dbb136bb74eb436fca4bf48fefe"
Oct 31 01:11:15.721405 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:11:15.721376    4284 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 01:11:15.721542 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:11:15.721463    4284 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 01:11:15.721542 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:11:15.721534    4284 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 01:11:16.727928 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:11:16.727893    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-4397fc-openshift-default-6b458f575f-tvgdj" event={"ID":"fd46495b-8c51-476d-91f9-750529060bbd","Type":"ContainerStarted","Data":"e9ebf0b02fcd8550bde2f0e6cae0ad98fe7c51b171e1397767f0556648bdb0b5"}
Oct 31 01:11:17.036702 ip-10-0-132-173 kubenswrapper[4284]: E1031 01:11:17.036606    4284 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fd8sp" containerName="extractor"
Oct 31 01:11:17.107248 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:11:17.106602    4284 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w2-4397fc-openshift-default-6b458f575f-tvgdj"
Oct 31 01:11:17.107248 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:11:17.106782    4284 patch_prober.go:29] interesting pod/e2e-shared-a-w2-4397fc-openshift-default-6b458f575f-tvgdj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.158:15021/healthz/ready\": dial tcp 10.134.0.158:15021: connect: connection refused" start-of-body=
Oct 31 01:11:17.107248 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:11:17.106849    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-4397fc-openshift-default-6b458f575f-tvgdj" podUID="fd46495b-8c51-476d-91f9-750529060bbd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.158:15021/healthz/ready\": dial tcp 10.134.0.158:15021: connect: connection refused"
Oct 31 01:11:17.612709 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:11:17.612677    4284 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-cb9a07-5877444fcd-d6mg2"]
Oct 31 01:11:17.616674 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:11:17.616653    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-cb9a07-5877444fcd-d6mg2"
Oct 31 01:11:17.732391 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:11:17.732347    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4dqpp\" (UniqueName: \"kubernetes.io/projected/b8db1668-6ee8-4d2e-b531-73300b46d926-kube-api-access-4dqpp\") pod \"maas-api-e2e-shared-a-w0-cb9a07-5877444fcd-d6mg2\" (UID: \"b8db1668-6ee8-4d2e-b531-73300b46d926\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-cb9a07-5877444fcd-d6mg2"
Oct 31 01:11:17.732904 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:11:17.732552    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b8db1668-6ee8-4d2e-b531-73300b46d926-metrics-certs\") pod \"maas-api-e2e-shared-a-w0-cb9a07-5877444fcd-d6mg2\" (UID: \"b8db1668-6ee8-4d2e-b531-73300b46d926\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-cb9a07-5877444fcd-d6mg2"
Oct 31 01:11:17.732904 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:11:17.732610    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b8db1668-6ee8-4d2e-b531-73300b46d926-maas-api-tls\") pod \"maas-api-e2e-shared-a-w0-cb9a07-5877444fcd-d6mg2\" (UID: \"b8db1668-6ee8-4d2e-b531-73300b46d926\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-cb9a07-5877444fcd-d6mg2"
Oct 31 01:11:17.732904 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:11:17.732635    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b8db1668-6ee8-4d2e-b531-73300b46d926-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w0-cb9a07-5877444fcd-d6mg2\" (UID: \"b8db1668-6ee8-4d2e-b531-73300b46d926\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-cb9a07-5877444fcd-d6mg2"
Oct 31 01:11:17.747461 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:11:17.747439    4284 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-w0-cb9a07-serving-cert\""
Oct 31 01:11:17.752581 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:11:17.752548    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-cb9a07-5877444fcd-d6mg2"]
Oct 31 01:11:17.810654 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:11:17.810614    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:11:17.842255 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:11:17.841087    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b8db1668-6ee8-4d2e-b531-73300b46d926-metrics-certs\") pod \"maas-api-e2e-shared-a-w0-cb9a07-5877444fcd-d6mg2\" (UID: \"b8db1668-6ee8-4d2e-b531-73300b46d926\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-cb9a07-5877444fcd-d6mg2"
Oct 31 01:11:17.842255 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:11:17.841267    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b8db1668-6ee8-4d2e-b531-73300b46d926-maas-api-tls\") pod \"maas-api-e2e-shared-a-w0-cb9a07-5877444fcd-d6mg2\" (UID: \"b8db1668-6ee8-4d2e-b531-73300b46d926\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-cb9a07-5877444fcd-d6mg2"
Oct 31 01:11:17.842255 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:11:17.841312    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b8db1668-6ee8-4d2e-b531-73300b46d926-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w0-cb9a07-5877444fcd-d6mg2\" (UID: \"b8db1668-6ee8-4d2e-b531-73300b46d926\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-cb9a07-5877444fcd-d6mg2"
Oct 31 01:11:17.842255 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:11:17.841522    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4dqpp\" (UniqueName: \"kubernetes.io/projected/b8db1668-6ee8-4d2e-b531-73300b46d926-kube-api-access-4dqpp\") pod \"maas-api-e2e-shared-a-w0-cb9a07-5877444fcd-d6mg2\" (UID: \"b8db1668-6ee8-4d2e-b531-73300b46d926\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-cb9a07-5877444fcd-d6mg2"
Oct 31 01:11:17.853232 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:11:17.849765    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b8db1668-6ee8-4d2e-b531-73300b46d926-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w0-cb9a07-5877444fcd-d6mg2\" (UID: \"b8db1668-6ee8-4d2e-b531-73300b46d926\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-cb9a07-5877444fcd-d6mg2"
Oct 31 01:11:17.853524 ip-10-0-132-173 kubenswrapper[4284]: E1031 01:11:17.853498    4284 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-cb9a07-serving-cert: secret "maas-api-e2e-shared-a-w0-cb9a07-serving-cert" not found
Oct 31 01:11:17.853747 ip-10-0-132-173 kubenswrapper[4284]: E1031 01:11:17.853734    4284 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b8db1668-6ee8-4d2e-b531-73300b46d926-maas-api-tls podName:b8db1668-6ee8-4d2e-b531-73300b46d926 nodeName:}" failed. No retries permitted until 2026-10-31 01:11:18.3537108 +0000 UTC m=+2592.498717708 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/b8db1668-6ee8-4d2e-b531-73300b46d926-maas-api-tls") pod "maas-api-e2e-shared-a-w0-cb9a07-5877444fcd-d6mg2" (UID: "b8db1668-6ee8-4d2e-b531-73300b46d926") : secret "maas-api-e2e-shared-a-w0-cb9a07-serving-cert" not found
Oct 31 01:11:17.882312 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:11:17.882237    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b8db1668-6ee8-4d2e-b531-73300b46d926-metrics-certs\") pod \"maas-api-e2e-shared-a-w0-cb9a07-5877444fcd-d6mg2\" (UID: \"b8db1668-6ee8-4d2e-b531-73300b46d926\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-cb9a07-5877444fcd-d6mg2"
Oct 31 01:11:17.911996 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:11:17.911961    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4dqpp\" (UniqueName: \"kubernetes.io/projected/b8db1668-6ee8-4d2e-b531-73300b46d926-kube-api-access-4dqpp\") pod \"maas-api-e2e-shared-a-w0-cb9a07-5877444fcd-d6mg2\" (UID: \"b8db1668-6ee8-4d2e-b531-73300b46d926\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-cb9a07-5877444fcd-d6mg2"
Oct 31 01:11:17.967242 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:11:17.967195    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:11:18.100637 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:11:18.100599    4284 patch_prober.go:29] interesting pod/e2e-shared-a-w2-4397fc-openshift-default-6b458f575f-tvgdj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.158:15021/healthz/ready\": dial tcp 10.134.0.158:15021: connect: connection refused" start-of-body=
Oct 31 01:11:18.100931 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:11:18.100898    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-4397fc-openshift-default-6b458f575f-tvgdj" podUID="fd46495b-8c51-476d-91f9-750529060bbd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.158:15021/healthz/ready\": dial tcp 10.134.0.158:15021: connect: connection refused"
Oct 31 01:11:18.163245 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:11:18.163181    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:11:18.448811 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:11:18.448719    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b8db1668-6ee8-4d2e-b531-73300b46d926-maas-api-tls\") pod \"maas-api-e2e-shared-a-w0-cb9a07-5877444fcd-d6mg2\" (UID: \"b8db1668-6ee8-4d2e-b531-73300b46d926\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-cb9a07-5877444fcd-d6mg2"
Oct 31 01:11:18.451419 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:11:18.451393    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b8db1668-6ee8-4d2e-b531-73300b46d926-maas-api-tls\") pod \"maas-api-e2e-shared-a-w0-cb9a07-5877444fcd-d6mg2\" (UID: \"b8db1668-6ee8-4d2e-b531-73300b46d926\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-cb9a07-5877444fcd-d6mg2"
Oct 31 01:11:18.530657 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:11:18.530622    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-cb9a07-5877444fcd-d6mg2"
Oct 31 01:11:18.925778 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:11:18.925751    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-cb9a07-5877444fcd-d6mg2"]
Oct 31 01:11:18.926376 ip-10-0-132-173 kubenswrapper[4284]: W1031 01:11:18.926349    4284 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb8db1668_6ee8_4d2e_b531_73300b46d926.slice/crio-3ccad606a8b78cf6c880ada2b0dea51d3178e3da0b0f492b88649a17c87a3ddb WatchSource:0}: Error finding container 3ccad606a8b78cf6c880ada2b0dea51d3178e3da0b0f492b88649a17c87a3ddb: Status 404 returned error can't find the container with id 3ccad606a8b78cf6c880ada2b0dea51d3178e3da0b0f492b88649a17c87a3ddb
Oct 31 01:11:19.078626 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:11:19.078587    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w2-4397fc-openshift-default-6b458f575f-tvgdj"
Oct 31 01:11:19.105252 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:11:19.104326    4284 patch_prober.go:29] interesting pod/e2e-shared-a-w2-4397fc-openshift-default-6b458f575f-tvgdj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.158:15021/healthz/ready\": dial tcp 10.134.0.158:15021: connect: connection refused" start-of-body=
Oct 31 01:11:19.105252 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:11:19.104388    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-4397fc-openshift-default-6b458f575f-tvgdj" podUID="fd46495b-8c51-476d-91f9-750529060bbd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.158:15021/healthz/ready\": dial tcp 10.134.0.158:15021: connect: connection refused"
Oct 31 01:11:19.749303 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:11:19.749249    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-cb9a07-5877444fcd-d6mg2" event={"ID":"b8db1668-6ee8-4d2e-b531-73300b46d926","Type":"ContainerStarted","Data":"3ccad606a8b78cf6c880ada2b0dea51d3178e3da0b0f492b88649a17c87a3ddb"}
Oct 31 01:11:20.106733 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:11:20.106700    4284 patch_prober.go:29] interesting pod/e2e-shared-a-w2-4397fc-openshift-default-6b458f575f-tvgdj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.158:15021/healthz/ready\": dial tcp 10.134.0.158:15021: connect: connection refused" start-of-body=
Oct 31 01:11:20.107361 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:11:20.107326    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-4397fc-openshift-default-6b458f575f-tvgdj" podUID="fd46495b-8c51-476d-91f9-750529060bbd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.158:15021/healthz/ready\": dial tcp 10.134.0.158:15021: connect: connection refused"
Oct 31 01:11:20.757709 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:11:20.757674    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-cb9a07-5877444fcd-d6mg2" event={"ID":"b8db1668-6ee8-4d2e-b531-73300b46d926","Type":"ContainerStarted","Data":"b7e492fcafcb503afb146c68f7604163908ebd5efa0e350f2d56094b4169bc26"}
Oct 31 01:11:20.757894 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:11:20.757750    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-cb9a07-5877444fcd-d6mg2"
Oct 31 01:11:20.911745 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:11:20.911690    4284 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-cb9a07-5877444fcd-d6mg2" podStartSLOduration=2.806017662 podStartE2EDuration="3.911673122s" totalImagesPullingTime="1.10565546s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 01:11:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 01:11:20.874400594 +0000 UTC m=+2595.019407522" watchObservedRunningTime="2026-10-31 01:11:20.911673122 +0000 UTC m=+2595.056680037"
Oct 31 01:11:21.101700 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:11:21.101602    4284 patch_prober.go:29] interesting pod/e2e-shared-a-w2-4397fc-openshift-default-6b458f575f-tvgdj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.158:15021/healthz/ready\": dial tcp 10.134.0.158:15021: connect: connection refused" start-of-body=
Oct 31 01:11:21.101983 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:11:21.101956    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-4397fc-openshift-default-6b458f575f-tvgdj" podUID="fd46495b-8c51-476d-91f9-750529060bbd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.158:15021/healthz/ready\": dial tcp 10.134.0.158:15021: connect: connection refused"
Oct 31 01:11:22.103961 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:11:22.103931    4284 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w2-4397fc-openshift-default-6b458f575f-tvgdj"
Oct 31 01:11:22.105008 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:11:22.104990    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w2-4397fc-openshift-default-6b458f575f-tvgdj"
Oct 31 01:11:22.892867 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:11:22.892825    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:11:26.769257 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:11:26.769227    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-cb9a07-5877444fcd-d6mg2"
Oct 31 01:11:27.772544 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:11:27.772489    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:11:28.921872 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:11:28.921820    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:11:29.376456 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:11:29.376361    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:11:30.396224 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:11:30.396170    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:11:30.706058 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:11:30.706022    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:11:32.792321 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:11:32.792285    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:11:35.204987 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:11:35.204947    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:11:35.428346 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:11:35.428308    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:11:37.947631 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:11:37.947593    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:11:38.370832 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:11:38.370738    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:11:38.569660 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:11:38.569554    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:11:42.988929 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:11:42.988893    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:11:47.014301 ip-10-0-132-173 kubenswrapper[4284]: E1031 01:11:47.014266    4284 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fd8sp" containerName="extractor"
Oct 31 01:11:47.889840 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:11:47.889797    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:11:48.957843 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:11:48.957808    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:11:49.398058 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:11:49.397975    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:11:50.986406 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:11:50.986362    4284 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-w2-f9925a-openshift-default-6d99c6d9c-nfzmj"]
Oct 31 01:11:50.991887 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:11:50.991865    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-f9925a-openshift-default-6d99c6d9c-nfzmj"
Oct 31 01:11:51.044076 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:11:51.044038    4284 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-w2-f9925a-openshift-default-dockercfg-x7qsm\""
Oct 31 01:11:51.087526 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:11:51.087481    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:11:51.095583 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:11:51.095554    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-f9925a-openshift-default-6d99c6d9c-nfzmj"]
Oct 31 01:11:51.119117 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:11:51.119080    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ed93ba84-d871-4811-ade6-5581127ca3dc-istio-envoy\") pod \"e2e-shared-b-w2-f9925a-openshift-default-6d99c6d9c-nfzmj\" (UID: \"ed93ba84-d871-4811-ade6-5581127ca3dc\") " pod="openshift-ingress/e2e-shared-b-w2-f9925a-openshift-default-6d99c6d9c-nfzmj"
Oct 31 01:11:51.119117 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:11:51.119122    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ed93ba84-d871-4811-ade6-5581127ca3dc-istio-data\") pod \"e2e-shared-b-w2-f9925a-openshift-default-6d99c6d9c-nfzmj\" (UID: \"ed93ba84-d871-4811-ade6-5581127ca3dc\") " pod="openshift-ingress/e2e-shared-b-w2-f9925a-openshift-default-6d99c6d9c-nfzmj"
Oct 31 01:11:51.119394 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:11:51.119257    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ed93ba84-d871-4811-ade6-5581127ca3dc-istiod-ca-cert\") pod \"e2e-shared-b-w2-f9925a-openshift-default-6d99c6d9c-nfzmj\" (UID: \"ed93ba84-d871-4811-ade6-5581127ca3dc\") " pod="openshift-ingress/e2e-shared-b-w2-f9925a-openshift-default-6d99c6d9c-nfzmj"
Oct 31 01:11:51.119394 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:11:51.119320    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ed93ba84-d871-4811-ade6-5581127ca3dc-istio-podinfo\") pod \"e2e-shared-b-w2-f9925a-openshift-default-6d99c6d9c-nfzmj\" (UID: \"ed93ba84-d871-4811-ade6-5581127ca3dc\") " pod="openshift-ingress/e2e-shared-b-w2-f9925a-openshift-default-6d99c6d9c-nfzmj"
Oct 31 01:11:51.119544 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:11:51.119442    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ed93ba84-d871-4811-ade6-5581127ca3dc-workload-certs\") pod \"e2e-shared-b-w2-f9925a-openshift-default-6d99c6d9c-nfzmj\" (UID: \"ed93ba84-d871-4811-ade6-5581127ca3dc\") " pod="openshift-ingress/e2e-shared-b-w2-f9925a-openshift-default-6d99c6d9c-nfzmj"
Oct 31 01:11:51.119544 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:11:51.119472    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ed93ba84-d871-4811-ade6-5581127ca3dc-workload-socket\") pod \"e2e-shared-b-w2-f9925a-openshift-default-6d99c6d9c-nfzmj\" (UID: \"ed93ba84-d871-4811-ade6-5581127ca3dc\") " pod="openshift-ingress/e2e-shared-b-w2-f9925a-openshift-default-6d99c6d9c-nfzmj"
Oct 31 01:11:51.119655 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:11:51.119636    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ed93ba84-d871-4811-ade6-5581127ca3dc-credential-socket\") pod \"e2e-shared-b-w2-f9925a-openshift-default-6d99c6d9c-nfzmj\" (UID: \"ed93ba84-d871-4811-ade6-5581127ca3dc\") " pod="openshift-ingress/e2e-shared-b-w2-f9925a-openshift-default-6d99c6d9c-nfzmj"
Oct 31 01:11:51.119804 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:11:51.119777    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cgll2\" (UniqueName: \"kubernetes.io/projected/ed93ba84-d871-4811-ade6-5581127ca3dc-kube-api-access-cgll2\") pod \"e2e-shared-b-w2-f9925a-openshift-default-6d99c6d9c-nfzmj\" (UID: \"ed93ba84-d871-4811-ade6-5581127ca3dc\") " pod="openshift-ingress/e2e-shared-b-w2-f9925a-openshift-default-6d99c6d9c-nfzmj"
Oct 31 01:11:51.119900 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:11:51.119882    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ed93ba84-d871-4811-ade6-5581127ca3dc-istio-token\") pod \"e2e-shared-b-w2-f9925a-openshift-default-6d99c6d9c-nfzmj\" (UID: \"ed93ba84-d871-4811-ade6-5581127ca3dc\") " pod="openshift-ingress/e2e-shared-b-w2-f9925a-openshift-default-6d99c6d9c-nfzmj"
Oct 31 01:11:51.221637 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:11:51.221602    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ed93ba84-d871-4811-ade6-5581127ca3dc-istio-envoy\") pod \"e2e-shared-b-w2-f9925a-openshift-default-6d99c6d9c-nfzmj\" (UID: \"ed93ba84-d871-4811-ade6-5581127ca3dc\") " pod="openshift-ingress/e2e-shared-b-w2-f9925a-openshift-default-6d99c6d9c-nfzmj"
Oct 31 01:11:51.221637 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:11:51.221643    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ed93ba84-d871-4811-ade6-5581127ca3dc-istio-data\") pod \"e2e-shared-b-w2-f9925a-openshift-default-6d99c6d9c-nfzmj\" (UID: \"ed93ba84-d871-4811-ade6-5581127ca3dc\") " pod="openshift-ingress/e2e-shared-b-w2-f9925a-openshift-default-6d99c6d9c-nfzmj"
Oct 31 01:11:51.221905 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:11:51.221682    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ed93ba84-d871-4811-ade6-5581127ca3dc-istiod-ca-cert\") pod \"e2e-shared-b-w2-f9925a-openshift-default-6d99c6d9c-nfzmj\" (UID: \"ed93ba84-d871-4811-ade6-5581127ca3dc\") " pod="openshift-ingress/e2e-shared-b-w2-f9925a-openshift-default-6d99c6d9c-nfzmj"
Oct 31 01:11:51.221905 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:11:51.221708    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ed93ba84-d871-4811-ade6-5581127ca3dc-istio-podinfo\") pod \"e2e-shared-b-w2-f9925a-openshift-default-6d99c6d9c-nfzmj\" (UID: \"ed93ba84-d871-4811-ade6-5581127ca3dc\") " pod="openshift-ingress/e2e-shared-b-w2-f9925a-openshift-default-6d99c6d9c-nfzmj"
Oct 31 01:11:51.221905 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:11:51.221748    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ed93ba84-d871-4811-ade6-5581127ca3dc-workload-certs\") pod \"e2e-shared-b-w2-f9925a-openshift-default-6d99c6d9c-nfzmj\" (UID: \"ed93ba84-d871-4811-ade6-5581127ca3dc\") " pod="openshift-ingress/e2e-shared-b-w2-f9925a-openshift-default-6d99c6d9c-nfzmj"
Oct 31 01:11:51.221905 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:11:51.221763    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ed93ba84-d871-4811-ade6-5581127ca3dc-workload-socket\") pod \"e2e-shared-b-w2-f9925a-openshift-default-6d99c6d9c-nfzmj\" (UID: \"ed93ba84-d871-4811-ade6-5581127ca3dc\") " pod="openshift-ingress/e2e-shared-b-w2-f9925a-openshift-default-6d99c6d9c-nfzmj"
Oct 31 01:11:51.221905 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:11:51.221828    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ed93ba84-d871-4811-ade6-5581127ca3dc-credential-socket\") pod \"e2e-shared-b-w2-f9925a-openshift-default-6d99c6d9c-nfzmj\" (UID: \"ed93ba84-d871-4811-ade6-5581127ca3dc\") " pod="openshift-ingress/e2e-shared-b-w2-f9925a-openshift-default-6d99c6d9c-nfzmj"
Oct 31 01:11:51.222184 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:11:51.221932    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cgll2\" (UniqueName: \"kubernetes.io/projected/ed93ba84-d871-4811-ade6-5581127ca3dc-kube-api-access-cgll2\") pod \"e2e-shared-b-w2-f9925a-openshift-default-6d99c6d9c-nfzmj\" (UID: \"ed93ba84-d871-4811-ade6-5581127ca3dc\") " pod="openshift-ingress/e2e-shared-b-w2-f9925a-openshift-default-6d99c6d9c-nfzmj"
Oct 31 01:11:51.222184 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:11:51.221998    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ed93ba84-d871-4811-ade6-5581127ca3dc-istio-token\") pod \"e2e-shared-b-w2-f9925a-openshift-default-6d99c6d9c-nfzmj\" (UID: \"ed93ba84-d871-4811-ade6-5581127ca3dc\") " pod="openshift-ingress/e2e-shared-b-w2-f9925a-openshift-default-6d99c6d9c-nfzmj"
Oct 31 01:11:51.222371 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:11:51.222350    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ed93ba84-d871-4811-ade6-5581127ca3dc-istio-data\") pod \"e2e-shared-b-w2-f9925a-openshift-default-6d99c6d9c-nfzmj\" (UID: \"ed93ba84-d871-4811-ade6-5581127ca3dc\") " pod="openshift-ingress/e2e-shared-b-w2-f9925a-openshift-default-6d99c6d9c-nfzmj"
Oct 31 01:11:51.222436 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:11:51.222396    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ed93ba84-d871-4811-ade6-5581127ca3dc-workload-certs\") pod \"e2e-shared-b-w2-f9925a-openshift-default-6d99c6d9c-nfzmj\" (UID: \"ed93ba84-d871-4811-ade6-5581127ca3dc\") " pod="openshift-ingress/e2e-shared-b-w2-f9925a-openshift-default-6d99c6d9c-nfzmj"
Oct 31 01:11:51.222497 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:11:51.222478    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ed93ba84-d871-4811-ade6-5581127ca3dc-credential-socket\") pod \"e2e-shared-b-w2-f9925a-openshift-default-6d99c6d9c-nfzmj\" (UID: \"ed93ba84-d871-4811-ade6-5581127ca3dc\") " pod="openshift-ingress/e2e-shared-b-w2-f9925a-openshift-default-6d99c6d9c-nfzmj"
Oct 31 01:11:51.222595 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:11:51.222572    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ed93ba84-d871-4811-ade6-5581127ca3dc-istiod-ca-cert\") pod \"e2e-shared-b-w2-f9925a-openshift-default-6d99c6d9c-nfzmj\" (UID: \"ed93ba84-d871-4811-ade6-5581127ca3dc\") " pod="openshift-ingress/e2e-shared-b-w2-f9925a-openshift-default-6d99c6d9c-nfzmj"
Oct 31 01:11:51.222695 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:11:51.222672    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ed93ba84-d871-4811-ade6-5581127ca3dc-workload-socket\") pod \"e2e-shared-b-w2-f9925a-openshift-default-6d99c6d9c-nfzmj\" (UID: \"ed93ba84-d871-4811-ade6-5581127ca3dc\") " pod="openshift-ingress/e2e-shared-b-w2-f9925a-openshift-default-6d99c6d9c-nfzmj"
Oct 31 01:11:51.224279 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:11:51.224259    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ed93ba84-d871-4811-ade6-5581127ca3dc-istio-envoy\") pod \"e2e-shared-b-w2-f9925a-openshift-default-6d99c6d9c-nfzmj\" (UID: \"ed93ba84-d871-4811-ade6-5581127ca3dc\") " pod="openshift-ingress/e2e-shared-b-w2-f9925a-openshift-default-6d99c6d9c-nfzmj"
Oct 31 01:11:51.224517 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:11:51.224315    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ed93ba84-d871-4811-ade6-5581127ca3dc-istio-podinfo\") pod \"e2e-shared-b-w2-f9925a-openshift-default-6d99c6d9c-nfzmj\" (UID: \"ed93ba84-d871-4811-ade6-5581127ca3dc\") " pod="openshift-ingress/e2e-shared-b-w2-f9925a-openshift-default-6d99c6d9c-nfzmj"
Oct 31 01:11:51.282427 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:11:51.282346    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cgll2\" (UniqueName: \"kubernetes.io/projected/ed93ba84-d871-4811-ade6-5581127ca3dc-kube-api-access-cgll2\") pod \"e2e-shared-b-w2-f9925a-openshift-default-6d99c6d9c-nfzmj\" (UID: \"ed93ba84-d871-4811-ade6-5581127ca3dc\") " pod="openshift-ingress/e2e-shared-b-w2-f9925a-openshift-default-6d99c6d9c-nfzmj"
Oct 31 01:11:51.283347 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:11:51.283320    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ed93ba84-d871-4811-ade6-5581127ca3dc-istio-token\") pod \"e2e-shared-b-w2-f9925a-openshift-default-6d99c6d9c-nfzmj\" (UID: \"ed93ba84-d871-4811-ade6-5581127ca3dc\") " pod="openshift-ingress/e2e-shared-b-w2-f9925a-openshift-default-6d99c6d9c-nfzmj"
Oct 31 01:11:51.304846 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:11:51.304811    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-f9925a-openshift-default-6d99c6d9c-nfzmj"
Oct 31 01:11:51.359237 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:11:51.346410    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:11:51.799970 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:11:51.799927    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-f9925a-openshift-default-6d99c6d9c-nfzmj"]
Oct 31 01:11:51.823262 ip-10-0-132-173 kubenswrapper[4284]: W1031 01:11:51.823186    4284 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poded93ba84_d871_4811_ade6_5581127ca3dc.slice/crio-00c9f69c4c0600dd2f33044f13114e507d5fea62cc6e7c9a5f98ce401ab57ef5 WatchSource:0}: Error finding container 00c9f69c4c0600dd2f33044f13114e507d5fea62cc6e7c9a5f98ce401ab57ef5: Status 404 returned error can't find the container with id 00c9f69c4c0600dd2f33044f13114e507d5fea62cc6e7c9a5f98ce401ab57ef5
Oct 31 01:11:51.825972 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:11:51.825940    4284 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 01:11:51.826051 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:11:51.826009    4284 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 01:11:51.826051 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:11:51.826036    4284 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 01:11:52.034939 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:11:52.034902    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-f9925a-openshift-default-6d99c6d9c-nfzmj" event={"ID":"ed93ba84-d871-4811-ade6-5581127ca3dc","Type":"ContainerStarted","Data":"1e11a386dd30b7efa3351e22ec346f09bfc78fcefd6e2657c82f6c7e3a9d8dff"}
Oct 31 01:11:52.035382 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:11:52.034968    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-f9925a-openshift-default-6d99c6d9c-nfzmj" event={"ID":"ed93ba84-d871-4811-ade6-5581127ca3dc","Type":"ContainerStarted","Data":"00c9f69c4c0600dd2f33044f13114e507d5fea62cc6e7c9a5f98ce401ab57ef5"}
Oct 31 01:11:52.158335 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:11:52.158276    4284 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-w2-f9925a-openshift-default-6d99c6d9c-nfzmj" podStartSLOduration=2.15826039 podStartE2EDuration="2.15826039s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 01:11:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 01:11:52.154922084 +0000 UTC m=+2626.299929011" watchObservedRunningTime="2026-10-31 01:11:52.15826039 +0000 UTC m=+2626.303267306"
Oct 31 01:11:52.318260 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:11:52.309285    4284 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w2-f9925a-openshift-default-6d99c6d9c-nfzmj"
Oct 31 01:11:52.337751 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:11:52.337715    4284 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w2-f9925a-openshift-default-6d99c6d9c-nfzmj"
Oct 31 01:11:53.041725 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:11:53.041697    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w2-f9925a-openshift-default-6d99c6d9c-nfzmj"
Oct 31 01:11:53.042639 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:11:53.042620    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w2-f9925a-openshift-default-6d99c6d9c-nfzmj"
Oct 31 01:11:53.184702 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:11:53.184650    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:11:55.099093 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:11:55.099058    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:11:55.330868 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:11:55.330830    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:11:58.392859 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:11:58.392786    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:11:58.774015 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:11:58.773973    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:11:58.929572 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:11:58.929269    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:12:03.661154 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:12:03.661119    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:12:03.821653 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:12:03.821603    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:12:04.430590 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:12:04.430550    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-66e263-56774d5b9f-g6gxl"]
Oct 31 01:12:04.575643 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:12:04.575606    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:12:04.762116 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:12:04.762033    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:12:04.937891 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:12:04.937858    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:12:08.913109 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:12:08.913067    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:12:09.431339 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:12:09.431298    4284 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-66e263-56774d5b9f-g6gxl" podUID="854ab963-5fa1-4b6e-be4c-52ffce0e3e15" containerName="payload-pre-processing" containerID="cri-o://2c64179fc0965283b57b6804aeb9217b44c594728d57ab968ab04e889be5e590" gracePeriod=25
Oct 31 01:12:09.713952 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:12:09.713924    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-66e263-56774d5b9f-g6gxl"
Oct 31 01:12:09.812943 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:12:09.812900    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/854ab963-5fa1-4b6e-be4c-52ffce0e3e15-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/854ab963-5fa1-4b6e-be4c-52ffce0e3e15-plugins-config-volume\") pod \"854ab963-5fa1-4b6e-be4c-52ffce0e3e15\" (UID: \"854ab963-5fa1-4b6e-be4c-52ffce0e3e15\") "
Oct 31 01:12:09.813122 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:12:09.813096    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/854ab963-5fa1-4b6e-be4c-52ffce0e3e15-kube-api-access-qb57d\" (UniqueName: \"kubernetes.io/projected/854ab963-5fa1-4b6e-be4c-52ffce0e3e15-kube-api-access-qb57d\") pod \"854ab963-5fa1-4b6e-be4c-52ffce0e3e15\" (UID: \"854ab963-5fa1-4b6e-be4c-52ffce0e3e15\") "
Oct 31 01:12:09.813385 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:12:09.813356    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/854ab963-5fa1-4b6e-be4c-52ffce0e3e15-plugins-config-volume" pod "854ab963-5fa1-4b6e-be4c-52ffce0e3e15" (UID: "854ab963-5fa1-4b6e-be4c-52ffce0e3e15"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 01:12:09.813669 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:12:09.813650    4284 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/854ab963-5fa1-4b6e-be4c-52ffce0e3e15-plugins-config-volume\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:12:09.815365 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:12:09.815347    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/854ab963-5fa1-4b6e-be4c-52ffce0e3e15-kube-api-access-qb57d" pod "854ab963-5fa1-4b6e-be4c-52ffce0e3e15" (UID: "854ab963-5fa1-4b6e-be4c-52ffce0e3e15"). InnerVolumeSpecName "kube-api-access-qb57d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:12:09.914656 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:12:09.914616    4284 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qb57d\" (UniqueName: \"kubernetes.io/projected/854ab963-5fa1-4b6e-be4c-52ffce0e3e15-kube-api-access-qb57d\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:12:10.163724 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:12:10.163690    4284 generic.go:356] "Generic (PLEG): container finished" podID="854ab963-5fa1-4b6e-be4c-52ffce0e3e15" containerID="2c64179fc0965283b57b6804aeb9217b44c594728d57ab968ab04e889be5e590" exitCode=0
Oct 31 01:12:10.163905 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:12:10.163741    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-66e263-56774d5b9f-g6gxl"
Oct 31 01:12:10.163905 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:12:10.163770    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-66e263-56774d5b9f-g6gxl" event={"ID":"854ab963-5fa1-4b6e-be4c-52ffce0e3e15","Type":"ContainerDied","Data":"2c64179fc0965283b57b6804aeb9217b44c594728d57ab968ab04e889be5e590"}
Oct 31 01:12:10.163905 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:12:10.163824    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-66e263-56774d5b9f-g6gxl" event={"ID":"854ab963-5fa1-4b6e-be4c-52ffce0e3e15","Type":"ContainerDied","Data":"54ac89a274d381a3d088b339f656a16161a86104e05fa722af52c5f167e21f86"}
Oct 31 01:12:10.163905 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:12:10.163849    4284 scope.go:122] "RemoveContainer" containerID="2c64179fc0965283b57b6804aeb9217b44c594728d57ab968ab04e889be5e590"
Oct 31 01:12:10.174576 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:12:10.174552    4284 scope.go:122] "RemoveContainer" containerID="2c64179fc0965283b57b6804aeb9217b44c594728d57ab968ab04e889be5e590"
Oct 31 01:12:10.174909 ip-10-0-132-173 kubenswrapper[4284]: E1031 01:12:10.174889    4284 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2c64179fc0965283b57b6804aeb9217b44c594728d57ab968ab04e889be5e590\": container with ID starting with 2c64179fc0965283b57b6804aeb9217b44c594728d57ab968ab04e889be5e590 not found: ID does not exist" containerID="2c64179fc0965283b57b6804aeb9217b44c594728d57ab968ab04e889be5e590"
Oct 31 01:12:10.174959 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:12:10.174921    4284 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2c64179fc0965283b57b6804aeb9217b44c594728d57ab968ab04e889be5e590"} err="failed to get container status \"2c64179fc0965283b57b6804aeb9217b44c594728d57ab968ab04e889be5e590\": rpc error: code = NotFound desc = could not find container \"2c64179fc0965283b57b6804aeb9217b44c594728d57ab968ab04e889be5e590\": container with ID starting with 2c64179fc0965283b57b6804aeb9217b44c594728d57ab968ab04e889be5e590 not found: ID does not exist"
Oct 31 01:12:10.328124 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:12:10.328050    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-66e263-56774d5b9f-g6gxl"]
Oct 31 01:12:10.332401 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:12:10.332375    4284 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-66e263-56774d5b9f-g6gxl"]
Oct 31 01:12:11.963395 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:12:11.963364    4284 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="854ab963-5fa1-4b6e-be4c-52ffce0e3e15" path="/var/lib/kubelet/pods/854ab963-5fa1-4b6e-be4c-52ffce0e3e15/volumes"
Oct 31 01:12:13.635892 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:12:13.635858    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:12:13.844323 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:12:13.844274    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:12:17.014839 ip-10-0-132-173 kubenswrapper[4284]: E1031 01:12:17.014758    4284 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fd8sp" containerName="extractor"
Oct 31 01:12:18.725171 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:12:18.725128    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:12:25.209439 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:12:25.209406    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:12:30.030545 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:12:30.030501    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:12:31.737273 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:12:31.737234    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:12:32.226342 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:12:32.226303    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:12:33.098148 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:12:33.098110    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:12:33.363257 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:12:33.363132    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:12:35.549962 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:12:35.549913    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:12:37.480496 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:12:37.480458    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:12:37.633013 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:12:37.632978    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:12:37.780345 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:12:37.780264    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:12:37.932489 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:12:37.932444    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:12:40.773928 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:12:40.773894    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:12:40.948243 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:12:40.942495    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:12:41.100708 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:12:41.100626    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:12:42.479817 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:12:42.479779    4284 generic.go:356] "Generic (PLEG): container finished" podID="e6cf830a-f9d2-4d0c-90de-672a81d3a441" containerID="f159ec40f54ddaa13f0ed4da6afe2d0b0843c937e7b42b9fb4f37a1b49fed326" exitCode=0
Oct 31 01:12:42.480352 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:12:42.479849    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w4-a72754-openshift-default-6dbf8c96d5-c4xcb" event={"ID":"e6cf830a-f9d2-4d0c-90de-672a81d3a441","Type":"ContainerDied","Data":"f159ec40f54ddaa13f0ed4da6afe2d0b0843c937e7b42b9fb4f37a1b49fed326"}
Oct 31 01:12:42.480352 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:12:42.480300    4284 scope.go:122] "RemoveContainer" containerID="f159ec40f54ddaa13f0ed4da6afe2d0b0843c937e7b42b9fb4f37a1b49fed326"
Oct 31 01:12:42.481947 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:12:42.481921    4284 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 01:12:42.482014 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:12:42.481987    4284 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 01:12:42.482057 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:12:42.482020    4284 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 01:12:43.512792 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:12:43.512758    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w4-a72754-openshift-default-6dbf8c96d5-c4xcb" event={"ID":"e6cf830a-f9d2-4d0c-90de-672a81d3a441","Type":"ContainerStarted","Data":"92c2545939d3489e8348b50f70f67c0efcde7fb11fe267c5b4f7d95142315d1a"}
Oct 31 01:12:43.717290 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:12:43.717254    4284 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w4-a72754-openshift-default-6dbf8c96d5-c4xcb"
Oct 31 01:12:43.717498 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:12:43.717464    4284 patch_prober.go:29] interesting pod/e2e-shared-b-w4-a72754-openshift-default-6dbf8c96d5-c4xcb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.162:15021/healthz/ready\": dial tcp 10.134.0.162:15021: connect: connection refused" start-of-body=
Oct 31 01:12:43.717559 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:12:43.717515    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w4-a72754-openshift-default-6dbf8c96d5-c4xcb" podUID="e6cf830a-f9d2-4d0c-90de-672a81d3a441" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.162:15021/healthz/ready\": dial tcp 10.134.0.162:15021: connect: connection refused"
Oct 31 01:12:44.717867 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:12:44.717833    4284 patch_prober.go:29] interesting pod/e2e-shared-b-w4-a72754-openshift-default-6dbf8c96d5-c4xcb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.162:15021/healthz/ready\": dial tcp 10.134.0.162:15021: connect: connection refused" start-of-body=
Oct 31 01:12:44.718282 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:12:44.717898    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w4-a72754-openshift-default-6dbf8c96d5-c4xcb" podUID="e6cf830a-f9d2-4d0c-90de-672a81d3a441" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.162:15021/healthz/ready\": dial tcp 10.134.0.162:15021: connect: connection refused"
Oct 31 01:12:45.426607 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:12:45.426572    4284 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-3be6fa-5db9dc4847-46thz"]
Oct 31 01:12:45.427206 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:12:45.427191    4284 cpu_manager.go:414] "containerMap: removing container" podUID="854ab963-5fa1-4b6e-be4c-52ffce0e3e15" containerName="payload-pre-processing"
Oct 31 01:12:45.427281 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:12:45.427221    4284 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="854ab963-5fa1-4b6e-be4c-52ffce0e3e15" containerName="payload-pre-processing"
Oct 31 01:12:45.427346 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:12:45.427336    4284 memory_manager.go:365] "RemoveStaleState removing state" podUID="854ab963-5fa1-4b6e-be4c-52ffce0e3e15" containerName="payload-pre-processing"
Oct 31 01:12:45.430096 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:12:45.430080    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-3be6fa-5db9dc4847-46thz"
Oct 31 01:12:45.476695 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:12:45.476669    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e38c82a4-98f4-48a6-b888-f95877335bd1-metrics-certs\") pod \"maas-api-e2e-shared-b-w1-3be6fa-5db9dc4847-46thz\" (UID: \"e38c82a4-98f4-48a6-b888-f95877335bd1\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-3be6fa-5db9dc4847-46thz"
Oct 31 01:12:45.476816 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:12:45.476725    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6vqcf\" (UniqueName: \"kubernetes.io/projected/e38c82a4-98f4-48a6-b888-f95877335bd1-kube-api-access-6vqcf\") pod \"maas-api-e2e-shared-b-w1-3be6fa-5db9dc4847-46thz\" (UID: \"e38c82a4-98f4-48a6-b888-f95877335bd1\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-3be6fa-5db9dc4847-46thz"
Oct 31 01:12:45.476937 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:12:45.476914    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e38c82a4-98f4-48a6-b888-f95877335bd1-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w1-3be6fa-5db9dc4847-46thz\" (UID: \"e38c82a4-98f4-48a6-b888-f95877335bd1\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-3be6fa-5db9dc4847-46thz"
Oct 31 01:12:45.476990 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:12:45.476948    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e38c82a4-98f4-48a6-b888-f95877335bd1-maas-api-tls\") pod \"maas-api-e2e-shared-b-w1-3be6fa-5db9dc4847-46thz\" (UID: \"e38c82a4-98f4-48a6-b888-f95877335bd1\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-3be6fa-5db9dc4847-46thz"
Oct 31 01:12:45.482388 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:12:45.482366    4284 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-w1-3be6fa-serving-cert\""
Oct 31 01:12:45.535004 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:12:45.534971    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-3be6fa-5db9dc4847-46thz"]
Oct 31 01:12:45.577867 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:12:45.577830    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e38c82a4-98f4-48a6-b888-f95877335bd1-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w1-3be6fa-5db9dc4847-46thz\" (UID: \"e38c82a4-98f4-48a6-b888-f95877335bd1\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-3be6fa-5db9dc4847-46thz"
Oct 31 01:12:45.577867 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:12:45.577870    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e38c82a4-98f4-48a6-b888-f95877335bd1-maas-api-tls\") pod \"maas-api-e2e-shared-b-w1-3be6fa-5db9dc4847-46thz\" (UID: \"e38c82a4-98f4-48a6-b888-f95877335bd1\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-3be6fa-5db9dc4847-46thz"
Oct 31 01:12:45.578127 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:12:45.577937    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e38c82a4-98f4-48a6-b888-f95877335bd1-metrics-certs\") pod \"maas-api-e2e-shared-b-w1-3be6fa-5db9dc4847-46thz\" (UID: \"e38c82a4-98f4-48a6-b888-f95877335bd1\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-3be6fa-5db9dc4847-46thz"
Oct 31 01:12:45.578127 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:12:45.577982    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6vqcf\" (UniqueName: \"kubernetes.io/projected/e38c82a4-98f4-48a6-b888-f95877335bd1-kube-api-access-6vqcf\") pod \"maas-api-e2e-shared-b-w1-3be6fa-5db9dc4847-46thz\" (UID: \"e38c82a4-98f4-48a6-b888-f95877335bd1\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-3be6fa-5db9dc4847-46thz"
Oct 31 01:12:45.578886 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:12:45.578861    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e38c82a4-98f4-48a6-b888-f95877335bd1-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w1-3be6fa-5db9dc4847-46thz\" (UID: \"e38c82a4-98f4-48a6-b888-f95877335bd1\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-3be6fa-5db9dc4847-46thz"
Oct 31 01:12:45.578995 ip-10-0-132-173 kubenswrapper[4284]: E1031 01:12:45.578946    4284 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-3be6fa-serving-cert: secret "maas-api-e2e-shared-b-w1-3be6fa-serving-cert" not found
Oct 31 01:12:45.578995 ip-10-0-132-173 kubenswrapper[4284]: E1031 01:12:45.578988    4284 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e38c82a4-98f4-48a6-b888-f95877335bd1-maas-api-tls podName:e38c82a4-98f4-48a6-b888-f95877335bd1 nodeName:}" failed. No retries permitted until 2026-10-31 01:12:46.078975664 +0000 UTC m=+2680.223982558 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/e38c82a4-98f4-48a6-b888-f95877335bd1-maas-api-tls") pod "maas-api-e2e-shared-b-w1-3be6fa-5db9dc4847-46thz" (UID: "e38c82a4-98f4-48a6-b888-f95877335bd1") : secret "maas-api-e2e-shared-b-w1-3be6fa-serving-cert" not found
Oct 31 01:12:45.582093 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:12:45.582073    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e38c82a4-98f4-48a6-b888-f95877335bd1-metrics-certs\") pod \"maas-api-e2e-shared-b-w1-3be6fa-5db9dc4847-46thz\" (UID: \"e38c82a4-98f4-48a6-b888-f95877335bd1\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-3be6fa-5db9dc4847-46thz"
Oct 31 01:12:45.625367 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:12:45.625333    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:12:45.638047 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:12:45.638018    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6vqcf\" (UniqueName: \"kubernetes.io/projected/e38c82a4-98f4-48a6-b888-f95877335bd1-kube-api-access-6vqcf\") pod \"maas-api-e2e-shared-b-w1-3be6fa-5db9dc4847-46thz\" (UID: \"e38c82a4-98f4-48a6-b888-f95877335bd1\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-3be6fa-5db9dc4847-46thz"
Oct 31 01:12:45.721284 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:12:45.720719    4284 patch_prober.go:29] interesting pod/e2e-shared-b-w4-a72754-openshift-default-6dbf8c96d5-c4xcb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.162:15021/healthz/ready\": dial tcp 10.134.0.162:15021: connect: connection refused" start-of-body=
Oct 31 01:12:45.721284 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:12:45.720775    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w4-a72754-openshift-default-6dbf8c96d5-c4xcb" podUID="e6cf830a-f9d2-4d0c-90de-672a81d3a441" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.162:15021/healthz/ready\": dial tcp 10.134.0.162:15021: connect: connection refused"
Oct 31 01:12:45.817367 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:12:45.817326    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:12:45.971110 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:12:45.971074    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:12:46.109993 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:12:46.109916    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e38c82a4-98f4-48a6-b888-f95877335bd1-maas-api-tls\") pod \"maas-api-e2e-shared-b-w1-3be6fa-5db9dc4847-46thz\" (UID: \"e38c82a4-98f4-48a6-b888-f95877335bd1\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-3be6fa-5db9dc4847-46thz"
Oct 31 01:12:46.114017 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:12:46.113983    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e38c82a4-98f4-48a6-b888-f95877335bd1-maas-api-tls\") pod \"maas-api-e2e-shared-b-w1-3be6fa-5db9dc4847-46thz\" (UID: \"e38c82a4-98f4-48a6-b888-f95877335bd1\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-3be6fa-5db9dc4847-46thz"
Oct 31 01:12:46.342668 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:12:46.342625    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-3be6fa-5db9dc4847-46thz"
Oct 31 01:12:46.717579 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:12:46.717502    4284 patch_prober.go:29] interesting pod/e2e-shared-b-w4-a72754-openshift-default-6dbf8c96d5-c4xcb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.162:15021/healthz/ready\": dial tcp 10.134.0.162:15021: connect: connection refused" start-of-body=
Oct 31 01:12:46.717579 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:12:46.717553    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w4-a72754-openshift-default-6dbf8c96d5-c4xcb" podUID="e6cf830a-f9d2-4d0c-90de-672a81d3a441" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.162:15021/healthz/ready\": dial tcp 10.134.0.162:15021: connect: connection refused"
Oct 31 01:12:46.737486 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:12:46.737457    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-3be6fa-5db9dc4847-46thz"]
Oct 31 01:12:46.738160 ip-10-0-132-173 kubenswrapper[4284]: W1031 01:12:46.738126    4284 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode38c82a4_98f4_48a6_b888_f95877335bd1.slice/crio-972b3a4deb45dbe9851879f9d2ecb4f2e6fd45e5f1725d50df6471902aa467d4 WatchSource:0}: Error finding container 972b3a4deb45dbe9851879f9d2ecb4f2e6fd45e5f1725d50df6471902aa467d4: Status 404 returned error can't find the container with id 972b3a4deb45dbe9851879f9d2ecb4f2e6fd45e5f1725d50df6471902aa467d4
Oct 31 01:12:47.038508 ip-10-0-132-173 kubenswrapper[4284]: E1031 01:12:47.038419    4284 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fd8sp" containerName="extractor"
Oct 31 01:12:47.550589 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:12:47.550541    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-3be6fa-5db9dc4847-46thz" event={"ID":"e38c82a4-98f4-48a6-b888-f95877335bd1","Type":"ContainerStarted","Data":"972b3a4deb45dbe9851879f9d2ecb4f2e6fd45e5f1725d50df6471902aa467d4"}
Oct 31 01:12:47.717848 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:12:47.717800    4284 patch_prober.go:29] interesting pod/e2e-shared-b-w4-a72754-openshift-default-6dbf8c96d5-c4xcb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.162:15021/healthz/ready\": dial tcp 10.134.0.162:15021: connect: connection refused" start-of-body=
Oct 31 01:12:47.718023 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:12:47.717870    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w4-a72754-openshift-default-6dbf8c96d5-c4xcb" podUID="e6cf830a-f9d2-4d0c-90de-672a81d3a441" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.162:15021/healthz/ready\": dial tcp 10.134.0.162:15021: connect: connection refused"
Oct 31 01:12:48.558953 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:12:48.558900    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-3be6fa-5db9dc4847-46thz" event={"ID":"e38c82a4-98f4-48a6-b888-f95877335bd1","Type":"ContainerStarted","Data":"cbd2003858448580b240b709299c93bf893fd47219bd3c1409101c48ee7603eb"}
Oct 31 01:12:48.559489 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:12:48.558976    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-3be6fa-5db9dc4847-46thz"
Oct 31 01:12:48.673916 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:12:48.673852    4284 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-3be6fa-5db9dc4847-46thz" podStartSLOduration=2.7477216909999997 podStartE2EDuration="3.67383142s" totalImagesPullingTime="926.109729ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 01:12:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 01:12:48.672075799 +0000 UTC m=+2682.817082717" watchObservedRunningTime="2026-10-31 01:12:48.67383142 +0000 UTC m=+2682.818838337"
Oct 31 01:12:48.724252 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:12:48.724187    4284 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w4-a72754-openshift-default-6dbf8c96d5-c4xcb"
Oct 31 01:12:48.724588 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:12:48.724570    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w4-a72754-openshift-default-6dbf8c96d5-c4xcb"
Oct 31 01:12:48.725283 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:12:48.725262    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w4-a72754-openshift-default-6dbf8c96d5-c4xcb"
Oct 31 01:12:50.699104 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:12:50.699068    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:12:54.571150 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:12:54.571123    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-3be6fa-5db9dc4847-46thz"
Oct 31 01:12:55.463943 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:12:55.463899    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:13:00.201949 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:13:00.201916    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:13:04.015050 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:13:04.015015    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:13:04.642201 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:13:04.642167    4284 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-f9925a-7b8b48d8d7-z2n6r"]
Oct 31 01:13:04.645291 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:13:04.645272    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-f9925a-7b8b48d8d7-z2n6r"
Oct 31 01:13:04.697724 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:13:04.697678    4284 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-w2-f9925a-serving-cert\""
Oct 31 01:13:04.701648 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:13:04.701624    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-f9925a-7b8b48d8d7-z2n6r"]
Oct 31 01:13:04.768926 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:13:04.768897    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r4mds\" (UniqueName: \"kubernetes.io/projected/f763b240-930c-4044-8c59-02cc2bd614d7-kube-api-access-r4mds\") pod \"maas-api-e2e-shared-b-w2-f9925a-7b8b48d8d7-z2n6r\" (UID: \"f763b240-930c-4044-8c59-02cc2bd614d7\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-f9925a-7b8b48d8d7-z2n6r"
Oct 31 01:13:04.769123 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:13:04.769068    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f763b240-930c-4044-8c59-02cc2bd614d7-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-f9925a-7b8b48d8d7-z2n6r\" (UID: \"f763b240-930c-4044-8c59-02cc2bd614d7\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-f9925a-7b8b48d8d7-z2n6r"
Oct 31 01:13:04.769184 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:13:04.769129    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f763b240-930c-4044-8c59-02cc2bd614d7-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-f9925a-7b8b48d8d7-z2n6r\" (UID: \"f763b240-930c-4044-8c59-02cc2bd614d7\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-f9925a-7b8b48d8d7-z2n6r"
Oct 31 01:13:04.769254 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:13:04.769199    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f763b240-930c-4044-8c59-02cc2bd614d7-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-f9925a-7b8b48d8d7-z2n6r\" (UID: \"f763b240-930c-4044-8c59-02cc2bd614d7\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-f9925a-7b8b48d8d7-z2n6r"
Oct 31 01:13:04.836745 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:13:04.836703    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:13:04.870398 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:13:04.870355    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f763b240-930c-4044-8c59-02cc2bd614d7-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-f9925a-7b8b48d8d7-z2n6r\" (UID: \"f763b240-930c-4044-8c59-02cc2bd614d7\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-f9925a-7b8b48d8d7-z2n6r"
Oct 31 01:13:04.870598 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:13:04.870432    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f763b240-930c-4044-8c59-02cc2bd614d7-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-f9925a-7b8b48d8d7-z2n6r\" (UID: \"f763b240-930c-4044-8c59-02cc2bd614d7\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-f9925a-7b8b48d8d7-z2n6r"
Oct 31 01:13:04.870598 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:13:04.870519    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f763b240-930c-4044-8c59-02cc2bd614d7-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-f9925a-7b8b48d8d7-z2n6r\" (UID: \"f763b240-930c-4044-8c59-02cc2bd614d7\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-f9925a-7b8b48d8d7-z2n6r"
Oct 31 01:13:04.870598 ip-10-0-132-173 kubenswrapper[4284]: E1031 01:13:04.870533    4284 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-f9925a-serving-cert: secret "maas-api-e2e-shared-b-w2-f9925a-serving-cert" not found
Oct 31 01:13:04.870598 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:13:04.870587    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r4mds\" (UniqueName: \"kubernetes.io/projected/f763b240-930c-4044-8c59-02cc2bd614d7-kube-api-access-r4mds\") pod \"maas-api-e2e-shared-b-w2-f9925a-7b8b48d8d7-z2n6r\" (UID: \"f763b240-930c-4044-8c59-02cc2bd614d7\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-f9925a-7b8b48d8d7-z2n6r"
Oct 31 01:13:04.870822 ip-10-0-132-173 kubenswrapper[4284]: E1031 01:13:04.870625    4284 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f763b240-930c-4044-8c59-02cc2bd614d7-maas-api-tls podName:f763b240-930c-4044-8c59-02cc2bd614d7 nodeName:}" failed. No retries permitted until 2026-10-31 01:13:05.370606921 +0000 UTC m=+2699.515613832 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/f763b240-930c-4044-8c59-02cc2bd614d7-maas-api-tls") pod "maas-api-e2e-shared-b-w2-f9925a-7b8b48d8d7-z2n6r" (UID: "f763b240-930c-4044-8c59-02cc2bd614d7") : secret "maas-api-e2e-shared-b-w2-f9925a-serving-cert" not found
Oct 31 01:13:04.871373 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:13:04.871164    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f763b240-930c-4044-8c59-02cc2bd614d7-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-f9925a-7b8b48d8d7-z2n6r\" (UID: \"f763b240-930c-4044-8c59-02cc2bd614d7\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-f9925a-7b8b48d8d7-z2n6r"
Oct 31 01:13:04.873199 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:13:04.873173    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f763b240-930c-4044-8c59-02cc2bd614d7-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-f9925a-7b8b48d8d7-z2n6r\" (UID: \"f763b240-930c-4044-8c59-02cc2bd614d7\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-f9925a-7b8b48d8d7-z2n6r"
Oct 31 01:13:04.939360 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:13:04.939323    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r4mds\" (UniqueName: \"kubernetes.io/projected/f763b240-930c-4044-8c59-02cc2bd614d7-kube-api-access-r4mds\") pod \"maas-api-e2e-shared-b-w2-f9925a-7b8b48d8d7-z2n6r\" (UID: \"f763b240-930c-4044-8c59-02cc2bd614d7\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-f9925a-7b8b48d8d7-z2n6r"
Oct 31 01:13:04.984992 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:13:04.984941    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:13:05.130202 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:13:05.130167    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:13:05.378143 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:13:05.378043    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f763b240-930c-4044-8c59-02cc2bd614d7-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-f9925a-7b8b48d8d7-z2n6r\" (UID: \"f763b240-930c-4044-8c59-02cc2bd614d7\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-f9925a-7b8b48d8d7-z2n6r"
Oct 31 01:13:05.380834 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:13:05.380809    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f763b240-930c-4044-8c59-02cc2bd614d7-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-f9925a-7b8b48d8d7-z2n6r\" (UID: \"f763b240-930c-4044-8c59-02cc2bd614d7\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-f9925a-7b8b48d8d7-z2n6r"
Oct 31 01:13:05.557759 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:13:05.557725    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-f9925a-7b8b48d8d7-z2n6r"
Oct 31 01:13:05.930048 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:13:05.930024    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-f9925a-7b8b48d8d7-z2n6r"]
Oct 31 01:13:05.931536 ip-10-0-132-173 kubenswrapper[4284]: W1031 01:13:05.931487    4284 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf763b240_930c_4044_8c59_02cc2bd614d7.slice/crio-5710bcb191778bb8aa957cf698fa86e83024bb7a4e34d2380f9a01e412373502 WatchSource:0}: Error finding container 5710bcb191778bb8aa957cf698fa86e83024bb7a4e34d2380f9a01e412373502: Status 404 returned error can't find the container with id 5710bcb191778bb8aa957cf698fa86e83024bb7a4e34d2380f9a01e412373502
Oct 31 01:13:05.933137 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:13:05.933114    4284 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 31 01:13:06.697070 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:13:06.697033    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-f9925a-7b8b48d8d7-z2n6r" event={"ID":"f763b240-930c-4044-8c59-02cc2bd614d7","Type":"ContainerStarted","Data":"5710bcb191778bb8aa957cf698fa86e83024bb7a4e34d2380f9a01e412373502"}
Oct 31 01:13:07.750848 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:13:07.750809    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-f9925a-7b8b48d8d7-z2n6r" event={"ID":"f763b240-930c-4044-8c59-02cc2bd614d7","Type":"ContainerStarted","Data":"4e35d3294b9cb82929171716d4ec2d5f43a1cd3db233576e758f80fd189d631b"}
Oct 31 01:13:07.752598 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:13:07.752575    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-f9925a-7b8b48d8d7-z2n6r"
Oct 31 01:13:07.866530 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:13:07.866476    4284 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-f9925a-7b8b48d8d7-z2n6r" podStartSLOduration=3.016363795 podStartE2EDuration="3.866463708s" totalImagesPullingTime="850.099913ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 01:13:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 01:13:07.866067494 +0000 UTC m=+2702.011074411" watchObservedRunningTime="2026-10-31 01:13:07.866463708 +0000 UTC m=+2702.011470625"
Oct 31 01:13:10.116005 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:13:10.115954    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:13:13.498751 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:13:13.498703    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:13:13.775441 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:13:13.774427    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-f9925a-7b8b48d8d7-z2n6r"
Oct 31 01:13:13.950411 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:13:13.950306    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:13:16.624142 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:13:16.624099    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:13:17.014853 ip-10-0-132-173 kubenswrapper[4284]: E1031 01:13:17.014816    4284 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fd8sp" containerName="extractor"
Oct 31 01:13:21.379404 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:13:21.379364    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:13:26.120075 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:13:26.120039    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:13:29.710365 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:13:29.710327    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:13:30.862276 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:13:30.862240    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:13:35.983383 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:13:35.983347    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:13:36.383491 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:13:36.383399    4284 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-55fbbbf9dc-j9vgx_5fa890d5-a2f9-495a-972b-06b8db658c88/payload-processing/0.log"
Oct 31 01:13:36.657482 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:13:36.657449    4284 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-55fbbbf9dc-j9vgx_5fa890d5-a2f9-495a-972b-06b8db658c88/payload-processing/0.log"
Oct 31 01:13:37.245592 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:13:37.245544    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:13:39.083427 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:13:39.083386    4284 patch_prober.go:29] interesting pod/e2e-shared-b-w2-f9925a-openshift-default-6d99c6d9c-nfzmj container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.164:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 31 01:13:39.083898 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:13:39.083473    4284 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-w2-f9925a-openshift-default-6d99c6d9c-nfzmj" podUID="ed93ba84-d871-4811-ade6-5581127ca3dc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.164:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 31 01:13:39.966072 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:13:39.965989    4284 generic.go:356] "Generic (PLEG): container finished" podID="ed93ba84-d871-4811-ade6-5581127ca3dc" containerID="1e11a386dd30b7efa3351e22ec346f09bfc78fcefd6e2657c82f6c7e3a9d8dff" exitCode=0
Oct 31 01:13:39.966432 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:13:39.966408    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-f9925a-openshift-default-6d99c6d9c-nfzmj" event={"ID":"ed93ba84-d871-4811-ade6-5581127ca3dc","Type":"ContainerDied","Data":"1e11a386dd30b7efa3351e22ec346f09bfc78fcefd6e2657c82f6c7e3a9d8dff"}
Oct 31 01:13:39.966817 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:13:39.966799    4284 scope.go:122] "RemoveContainer" containerID="1e11a386dd30b7efa3351e22ec346f09bfc78fcefd6e2657c82f6c7e3a9d8dff"
Oct 31 01:13:39.968335 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:13:39.968312    4284 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 01:13:39.968407 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:13:39.968375    4284 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 01:13:39.968455 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:13:39.968409    4284 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 01:13:40.974552 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:13:40.974515    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-f9925a-openshift-default-6d99c6d9c-nfzmj" event={"ID":"ed93ba84-d871-4811-ade6-5581127ca3dc","Type":"ContainerStarted","Data":"e229ba4a03c1cf9651c6d521874f8615a689cf88eeeaf9c4dc279abbd70d4673"}
Oct 31 01:13:41.071236 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:13:41.071177    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:13:41.305661 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:13:41.305570    4284 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w2-f9925a-openshift-default-6d99c6d9c-nfzmj"
Oct 31 01:13:41.310833 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:13:41.310791    4284 patch_prober.go:29] interesting pod/e2e-shared-b-w2-f9925a-openshift-default-6d99c6d9c-nfzmj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.164:15021/healthz/ready\": dial tcp 10.134.0.164:15021: connect: connection refused" start-of-body=
Oct 31 01:13:41.311006 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:13:41.310859    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-f9925a-openshift-default-6d99c6d9c-nfzmj" podUID="ed93ba84-d871-4811-ade6-5581127ca3dc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.164:15021/healthz/ready\": dial tcp 10.134.0.164:15021: connect: connection refused"
Oct 31 01:13:42.309372 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:13:42.309333    4284 patch_prober.go:29] interesting pod/e2e-shared-b-w2-f9925a-openshift-default-6d99c6d9c-nfzmj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.164:15021/healthz/ready\": dial tcp 10.134.0.164:15021: connect: connection refused" start-of-body=
Oct 31 01:13:42.309786 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:13:42.309401    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-f9925a-openshift-default-6d99c6d9c-nfzmj" podUID="ed93ba84-d871-4811-ade6-5581127ca3dc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.164:15021/healthz/ready\": dial tcp 10.134.0.164:15021: connect: connection refused"
Oct 31 01:13:43.135779 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:13:43.135740    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:13:43.308407 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:13:43.308372    4284 patch_prober.go:29] interesting pod/e2e-shared-b-w2-f9925a-openshift-default-6d99c6d9c-nfzmj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.164:15021/healthz/ready\": dial tcp 10.134.0.164:15021: connect: connection refused" start-of-body=
Oct 31 01:13:43.308927 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:13:43.308899    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-f9925a-openshift-default-6d99c6d9c-nfzmj" podUID="ed93ba84-d871-4811-ade6-5581127ca3dc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.164:15021/healthz/ready\": dial tcp 10.134.0.164:15021: connect: connection refused"
Oct 31 01:13:44.310659 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:13:44.310610    4284 patch_prober.go:29] interesting pod/e2e-shared-b-w2-f9925a-openshift-default-6d99c6d9c-nfzmj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.164:15021/healthz/ready\": dial tcp 10.134.0.164:15021: connect: connection refused" start-of-body=
Oct 31 01:13:44.311149 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:13:44.310692    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-f9925a-openshift-default-6d99c6d9c-nfzmj" podUID="ed93ba84-d871-4811-ade6-5581127ca3dc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.164:15021/healthz/ready\": dial tcp 10.134.0.164:15021: connect: connection refused"
Oct 31 01:13:44.651737 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:13:44.651649    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:13:45.308236 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:13:45.307296    4284 patch_prober.go:29] interesting pod/e2e-shared-b-w2-f9925a-openshift-default-6d99c6d9c-nfzmj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.164:15021/healthz/ready\": dial tcp 10.134.0.164:15021: connect: connection refused" start-of-body=
Oct 31 01:13:45.308236 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:13:45.307356    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-f9925a-openshift-default-6d99c6d9c-nfzmj" podUID="ed93ba84-d871-4811-ade6-5581127ca3dc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.164:15021/healthz/ready\": dial tcp 10.134.0.164:15021: connect: connection refused"
Oct 31 01:13:46.309478 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:13:46.309441    4284 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w2-f9925a-openshift-default-6d99c6d9c-nfzmj"
Oct 31 01:13:46.309918 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:13:46.309738    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w2-f9925a-openshift-default-6d99c6d9c-nfzmj"
Oct 31 01:13:46.310375 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:13:46.310358    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w2-f9925a-openshift-default-6d99c6d9c-nfzmj"
Oct 31 01:13:47.015258 ip-10-0-132-173 kubenswrapper[4284]: E1031 01:13:47.015195    4284 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fd8sp" containerName="extractor"
Oct 31 01:13:47.867833 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:13:47.867773    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:13:52.671586 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:13:52.671543    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:13:57.131065 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:13:57.131029    4284 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-7f8596-79fb4467bd-qxqqj"]
Oct 31 01:13:57.134993 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:13:57.134974    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-7f8596-79fb4467bd-qxqqj"
Oct 31 01:13:57.213236 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:13:57.213188    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6dcf3d48-1abd-4417-8933-d3dabc39b565-metrics-certs\") pod \"maas-api-e2e-shared-b-w0-7f8596-79fb4467bd-qxqqj\" (UID: \"6dcf3d48-1abd-4417-8933-d3dabc39b565\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-7f8596-79fb4467bd-qxqqj"
Oct 31 01:13:57.213463 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:13:57.213252    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jrtk7\" (UniqueName: \"kubernetes.io/projected/6dcf3d48-1abd-4417-8933-d3dabc39b565-kube-api-access-jrtk7\") pod \"maas-api-e2e-shared-b-w0-7f8596-79fb4467bd-qxqqj\" (UID: \"6dcf3d48-1abd-4417-8933-d3dabc39b565\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-7f8596-79fb4467bd-qxqqj"
Oct 31 01:13:57.213463 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:13:57.213296    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6dcf3d48-1abd-4417-8933-d3dabc39b565-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w0-7f8596-79fb4467bd-qxqqj\" (UID: \"6dcf3d48-1abd-4417-8933-d3dabc39b565\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-7f8596-79fb4467bd-qxqqj"
Oct 31 01:13:57.213463 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:13:57.213455    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6dcf3d48-1abd-4417-8933-d3dabc39b565-maas-api-tls\") pod \"maas-api-e2e-shared-b-w0-7f8596-79fb4467bd-qxqqj\" (UID: \"6dcf3d48-1abd-4417-8933-d3dabc39b565\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-7f8596-79fb4467bd-qxqqj"
Oct 31 01:13:57.235036 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:13:57.234994    4284 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-w0-7f8596-serving-cert\""
Oct 31 01:13:57.240582 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:13:57.240550    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-7f8596-79fb4467bd-qxqqj"]
Oct 31 01:13:57.314924 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:13:57.314892    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6dcf3d48-1abd-4417-8933-d3dabc39b565-metrics-certs\") pod \"maas-api-e2e-shared-b-w0-7f8596-79fb4467bd-qxqqj\" (UID: \"6dcf3d48-1abd-4417-8933-d3dabc39b565\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-7f8596-79fb4467bd-qxqqj"
Oct 31 01:13:57.314924 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:13:57.314928    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jrtk7\" (UniqueName: \"kubernetes.io/projected/6dcf3d48-1abd-4417-8933-d3dabc39b565-kube-api-access-jrtk7\") pod \"maas-api-e2e-shared-b-w0-7f8596-79fb4467bd-qxqqj\" (UID: \"6dcf3d48-1abd-4417-8933-d3dabc39b565\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-7f8596-79fb4467bd-qxqqj"
Oct 31 01:13:57.315167 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:13:57.315066    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6dcf3d48-1abd-4417-8933-d3dabc39b565-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w0-7f8596-79fb4467bd-qxqqj\" (UID: \"6dcf3d48-1abd-4417-8933-d3dabc39b565\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-7f8596-79fb4467bd-qxqqj"
Oct 31 01:13:57.315251 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:13:57.315167    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6dcf3d48-1abd-4417-8933-d3dabc39b565-maas-api-tls\") pod \"maas-api-e2e-shared-b-w0-7f8596-79fb4467bd-qxqqj\" (UID: \"6dcf3d48-1abd-4417-8933-d3dabc39b565\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-7f8596-79fb4467bd-qxqqj"
Oct 31 01:13:57.315331 ip-10-0-132-173 kubenswrapper[4284]: E1031 01:13:57.315307    4284 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-7f8596-serving-cert: secret "maas-api-e2e-shared-b-w0-7f8596-serving-cert" not found
Oct 31 01:13:57.315406 ip-10-0-132-173 kubenswrapper[4284]: E1031 01:13:57.315394    4284 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6dcf3d48-1abd-4417-8933-d3dabc39b565-maas-api-tls podName:6dcf3d48-1abd-4417-8933-d3dabc39b565 nodeName:}" failed. No retries permitted until 2026-10-31 01:13:57.81537781 +0000 UTC m=+2751.960384719 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/6dcf3d48-1abd-4417-8933-d3dabc39b565-maas-api-tls") pod "maas-api-e2e-shared-b-w0-7f8596-79fb4467bd-qxqqj" (UID: "6dcf3d48-1abd-4417-8933-d3dabc39b565") : secret "maas-api-e2e-shared-b-w0-7f8596-serving-cert" not found
Oct 31 01:13:57.315819 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:13:57.315797    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6dcf3d48-1abd-4417-8933-d3dabc39b565-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w0-7f8596-79fb4467bd-qxqqj\" (UID: \"6dcf3d48-1abd-4417-8933-d3dabc39b565\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-7f8596-79fb4467bd-qxqqj"
Oct 31 01:13:57.317586 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:13:57.317561    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6dcf3d48-1abd-4417-8933-d3dabc39b565-metrics-certs\") pod \"maas-api-e2e-shared-b-w0-7f8596-79fb4467bd-qxqqj\" (UID: \"6dcf3d48-1abd-4417-8933-d3dabc39b565\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-7f8596-79fb4467bd-qxqqj"
Oct 31 01:13:57.345665 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:13:57.345634    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:13:57.412498 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:13:57.412309    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jrtk7\" (UniqueName: \"kubernetes.io/projected/6dcf3d48-1abd-4417-8933-d3dabc39b565-kube-api-access-jrtk7\") pod \"maas-api-e2e-shared-b-w0-7f8596-79fb4467bd-qxqqj\" (UID: \"6dcf3d48-1abd-4417-8933-d3dabc39b565\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-7f8596-79fb4467bd-qxqqj"
Oct 31 01:13:57.552480 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:13:57.552449    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:13:57.824811 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:13:57.824721    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6dcf3d48-1abd-4417-8933-d3dabc39b565-maas-api-tls\") pod \"maas-api-e2e-shared-b-w0-7f8596-79fb4467bd-qxqqj\" (UID: \"6dcf3d48-1abd-4417-8933-d3dabc39b565\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-7f8596-79fb4467bd-qxqqj"
Oct 31 01:13:57.827319 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:13:57.827295    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6dcf3d48-1abd-4417-8933-d3dabc39b565-maas-api-tls\") pod \"maas-api-e2e-shared-b-w0-7f8596-79fb4467bd-qxqqj\" (UID: \"6dcf3d48-1abd-4417-8933-d3dabc39b565\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-7f8596-79fb4467bd-qxqqj"
Oct 31 01:13:58.047073 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:13:58.047026    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-7f8596-79fb4467bd-qxqqj"
Oct 31 01:13:58.416915 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:13:58.416885    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-7f8596-79fb4467bd-qxqqj"]
Oct 31 01:13:58.417457 ip-10-0-132-173 kubenswrapper[4284]: W1031 01:13:58.417434    4284 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6dcf3d48_1abd_4417_8933_d3dabc39b565.slice/crio-81da6da65e0ac9a067d1fc2a0d9a3689a32a57b23babed8789efe2f77d15c979 WatchSource:0}: Error finding container 81da6da65e0ac9a067d1fc2a0d9a3689a32a57b23babed8789efe2f77d15c979: Status 404 returned error can't find the container with id 81da6da65e0ac9a067d1fc2a0d9a3689a32a57b23babed8789efe2f77d15c979
Oct 31 01:13:59.111861 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:13:59.111827    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-7f8596-79fb4467bd-qxqqj" event={"ID":"6dcf3d48-1abd-4417-8933-d3dabc39b565","Type":"ContainerStarted","Data":"81da6da65e0ac9a067d1fc2a0d9a3689a32a57b23babed8789efe2f77d15c979"}
Oct 31 01:14:00.139347 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:14:00.139292    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-7f8596-79fb4467bd-qxqqj" event={"ID":"6dcf3d48-1abd-4417-8933-d3dabc39b565","Type":"ContainerStarted","Data":"93313ebf6cee996cf4194248cf36e7d77a47314ac706cb40e0a9ff2b7216f0e0"}
Oct 31 01:14:00.140410 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:14:00.140385    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-7f8596-79fb4467bd-qxqqj"
Oct 31 01:14:00.282409 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:14:00.282356    4284 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-7f8596-79fb4467bd-qxqqj" podStartSLOduration=2.371466133 podStartE2EDuration="3.282342478s" totalImagesPullingTime="910.876345ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 01:13:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 01:14:00.280253371 +0000 UTC m=+2754.425260288" watchObservedRunningTime="2026-10-31 01:14:00.282342478 +0000 UTC m=+2754.427349435"
Oct 31 01:14:02.323865 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:14:02.323825    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:14:04.324269 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:14:04.324231    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:14:06.094959 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:14:06.094924    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:14:06.960581 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:14:06.960547    4284 scope.go:122] "RemoveContainer" containerID="8de2fa6e33514da4668b19e7e0e460ade06b9f1e09687e6210be4a72ea2f671d"
Oct 31 01:14:07.081890 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:14:07.081859    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:14:07.181750 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:14:07.181715    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-7f8596-79fb4467bd-qxqqj"
Oct 31 01:14:11.982756 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:14:11.982700    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:14:14.206395 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:14:14.206356    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:14:17.014616 ip-10-0-132-173 kubenswrapper[4284]: E1031 01:14:17.014578    4284 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fd8sp" containerName="extractor"
Oct 31 01:14:17.272290 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:14:17.264148    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:14:21.973579 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:14:21.973543    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:14:26.688770 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:14:26.688733    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:14:31.761763 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:14:31.761725    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:14:34.163377 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:14:34.163344    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:14:38.284551 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:14:38.284512    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:14:41.894352 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:14:41.894299    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:14:43.078290 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:14:43.078259    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:14:47.016417 ip-10-0-132-173 kubenswrapper[4284]: E1031 01:14:47.016382    4284 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fd8sp" containerName="extractor"
Oct 31 01:14:47.809971 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:14:47.809927    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:14:52.525564 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:14:52.525527    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:14:56.715599 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:14:56.715558    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-f1d081-5b8f78496b-xz9lw"]
Oct 31 01:14:56.716192 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:14:56.715816    4284 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-f1d081-5b8f78496b-xz9lw" podUID="dda5d4ad-a454-43fb-96ff-664f19acc3b4" containerName="maas-api" containerID="cri-o://797debadf5ad6801c24fdc7dda62a61e235faf745451f7499c35a815f593a599" gracePeriod=30
Oct 31 01:14:56.813276 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:14:56.813240    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:14:57.065383 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:14:57.065358    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-f1d081-5b8f78496b-xz9lw"
Oct 31 01:14:57.104343 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:14:57.098669    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/dda5d4ad-a454-43fb-96ff-664f19acc3b4-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/dda5d4ad-a454-43fb-96ff-664f19acc3b4-openshift-service-ca\") pod \"dda5d4ad-a454-43fb-96ff-664f19acc3b4\" (UID: \"dda5d4ad-a454-43fb-96ff-664f19acc3b4\") "
Oct 31 01:14:57.104343 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:14:57.098738    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/dda5d4ad-a454-43fb-96ff-664f19acc3b4-kube-api-access-m76pv\" (UniqueName: \"kubernetes.io/projected/dda5d4ad-a454-43fb-96ff-664f19acc3b4-kube-api-access-m76pv\") pod \"dda5d4ad-a454-43fb-96ff-664f19acc3b4\" (UID: \"dda5d4ad-a454-43fb-96ff-664f19acc3b4\") "
Oct 31 01:14:57.104343 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:14:57.098892    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/dda5d4ad-a454-43fb-96ff-664f19acc3b4-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dda5d4ad-a454-43fb-96ff-664f19acc3b4-maas-api-tls\") pod \"dda5d4ad-a454-43fb-96ff-664f19acc3b4\" (UID: \"dda5d4ad-a454-43fb-96ff-664f19acc3b4\") "
Oct 31 01:14:57.104343 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:14:57.099004    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/dda5d4ad-a454-43fb-96ff-664f19acc3b4-metrics-certs\" (UniqueName: \"kubernetes.io/secret/dda5d4ad-a454-43fb-96ff-664f19acc3b4-metrics-certs\") pod \"dda5d4ad-a454-43fb-96ff-664f19acc3b4\" (UID: \"dda5d4ad-a454-43fb-96ff-664f19acc3b4\") "
Oct 31 01:14:57.129494 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:14:57.129064    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dda5d4ad-a454-43fb-96ff-664f19acc3b4-kube-api-access-m76pv" pod "dda5d4ad-a454-43fb-96ff-664f19acc3b4" (UID: "dda5d4ad-a454-43fb-96ff-664f19acc3b4"). InnerVolumeSpecName "kube-api-access-m76pv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:14:57.133468 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:14:57.132312    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dda5d4ad-a454-43fb-96ff-664f19acc3b4-maas-api-tls" pod "dda5d4ad-a454-43fb-96ff-664f19acc3b4" (UID: "dda5d4ad-a454-43fb-96ff-664f19acc3b4"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 01:14:57.137942 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:14:57.137906    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dda5d4ad-a454-43fb-96ff-664f19acc3b4-metrics-certs" pod "dda5d4ad-a454-43fb-96ff-664f19acc3b4" (UID: "dda5d4ad-a454-43fb-96ff-664f19acc3b4"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 01:14:57.138500 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:14:57.138477    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dda5d4ad-a454-43fb-96ff-664f19acc3b4-openshift-service-ca" pod "dda5d4ad-a454-43fb-96ff-664f19acc3b4" (UID: "dda5d4ad-a454-43fb-96ff-664f19acc3b4"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 01:14:57.170240 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:14:57.164718    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-f1d081-8bc8c89c8-j7s55"]
Oct 31 01:14:57.170240 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:14:57.165040    4284 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-a-f1d081-8bc8c89c8-j7s55" podUID="6a061135-19b6-49e7-8917-b3a9a4274488" containerName="payload-processing" containerID="cri-o://312431d8e47135034e52b59c2e1580ad8f5f326404a208ce1394d5f3c7fd1b9e" gracePeriod=30
Oct 31 01:14:57.200931 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:14:57.200900    4284 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dda5d4ad-a454-43fb-96ff-664f19acc3b4-metrics-certs\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:14:57.200931 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:14:57.200932    4284 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/dda5d4ad-a454-43fb-96ff-664f19acc3b4-openshift-service-ca\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:14:57.201434 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:14:57.200942    4284 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m76pv\" (UniqueName: \"kubernetes.io/projected/dda5d4ad-a454-43fb-96ff-664f19acc3b4-kube-api-access-m76pv\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:14:57.201434 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:14:57.200952    4284 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dda5d4ad-a454-43fb-96ff-664f19acc3b4-maas-api-tls\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:14:57.201825 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:14:57.201795    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-f1d081-5c766b5c87-46qzr"]
Oct 31 01:14:57.202031 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:14:57.202005    4284 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-f1d081-5c766b5c87-46qzr" podUID="2d3da89a-1e5f-41af-95b5-6910bdd24aad" containerName="payload-pre-processing" containerID="cri-o://388a516c0044a468282de7173c84f5f0fffb739e3f990814333f9a4eca30fe85" gracePeriod=30
Oct 31 01:14:57.335168 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:14:57.335127    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:14:57.493184 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:14:57.493158    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-f1d081-8bc8c89c8-j7s55"
Oct 31 01:14:57.494450 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:14:57.494430    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-f1d081-5c766b5c87-46qzr"
Oct 31 01:14:57.526594 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:14:57.526547    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:14:57.554936 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:14:57.554908    4284 generic.go:356] "Generic (PLEG): container finished" podID="dda5d4ad-a454-43fb-96ff-664f19acc3b4" containerID="797debadf5ad6801c24fdc7dda62a61e235faf745451f7499c35a815f593a599" exitCode=0
Oct 31 01:14:57.555126 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:14:57.554968    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-f1d081-5b8f78496b-xz9lw"
Oct 31 01:14:57.555126 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:14:57.554998    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-f1d081-5b8f78496b-xz9lw" event={"ID":"dda5d4ad-a454-43fb-96ff-664f19acc3b4","Type":"ContainerDied","Data":"797debadf5ad6801c24fdc7dda62a61e235faf745451f7499c35a815f593a599"}
Oct 31 01:14:57.555126 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:14:57.555062    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-f1d081-5b8f78496b-xz9lw" event={"ID":"dda5d4ad-a454-43fb-96ff-664f19acc3b4","Type":"ContainerDied","Data":"0eab682be83e63c0c5a61dbacc6d581ba0fe586ddf4251eadcb3adbddd11b490"}
Oct 31 01:14:57.555126 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:14:57.555090    4284 scope.go:122] "RemoveContainer" containerID="797debadf5ad6801c24fdc7dda62a61e235faf745451f7499c35a815f593a599"
Oct 31 01:14:57.556406 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:14:57.556376    4284 generic.go:356] "Generic (PLEG): container finished" podID="6a061135-19b6-49e7-8917-b3a9a4274488" containerID="312431d8e47135034e52b59c2e1580ad8f5f326404a208ce1394d5f3c7fd1b9e" exitCode=0
Oct 31 01:14:57.556497 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:14:57.556416    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-f1d081-8bc8c89c8-j7s55"
Oct 31 01:14:57.556497 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:14:57.556458    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-f1d081-8bc8c89c8-j7s55" event={"ID":"6a061135-19b6-49e7-8917-b3a9a4274488","Type":"ContainerDied","Data":"312431d8e47135034e52b59c2e1580ad8f5f326404a208ce1394d5f3c7fd1b9e"}
Oct 31 01:14:57.556613 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:14:57.556508    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-f1d081-8bc8c89c8-j7s55" event={"ID":"6a061135-19b6-49e7-8917-b3a9a4274488","Type":"ContainerDied","Data":"9329510bf7ac884c3337da0964c9f4348d2dbf402fd09c97c934c8f1b9ef6743"}
Oct 31 01:14:57.558042 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:14:57.558018    4284 generic.go:356] "Generic (PLEG): container finished" podID="2d3da89a-1e5f-41af-95b5-6910bdd24aad" containerID="388a516c0044a468282de7173c84f5f0fffb739e3f990814333f9a4eca30fe85" exitCode=0
Oct 31 01:14:57.558155 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:14:57.558046    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-f1d081-5c766b5c87-46qzr"
Oct 31 01:14:57.558155 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:14:57.558103    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-f1d081-5c766b5c87-46qzr" event={"ID":"2d3da89a-1e5f-41af-95b5-6910bdd24aad","Type":"ContainerDied","Data":"388a516c0044a468282de7173c84f5f0fffb739e3f990814333f9a4eca30fe85"}
Oct 31 01:14:57.558155 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:14:57.558148    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-f1d081-5c766b5c87-46qzr" event={"ID":"2d3da89a-1e5f-41af-95b5-6910bdd24aad","Type":"ContainerDied","Data":"a5107c91b37aaf29b4f9d8f73af7be9513b1b273989139a99522c27d53965cb5"}
Oct 31 01:14:57.566037 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:14:57.566014    4284 scope.go:122] "RemoveContainer" containerID="797debadf5ad6801c24fdc7dda62a61e235faf745451f7499c35a815f593a599"
Oct 31 01:14:57.566357 ip-10-0-132-173 kubenswrapper[4284]: E1031 01:14:57.566334    4284 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"797debadf5ad6801c24fdc7dda62a61e235faf745451f7499c35a815f593a599\": container with ID starting with 797debadf5ad6801c24fdc7dda62a61e235faf745451f7499c35a815f593a599 not found: ID does not exist" containerID="797debadf5ad6801c24fdc7dda62a61e235faf745451f7499c35a815f593a599"
Oct 31 01:14:57.566448 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:14:57.566370    4284 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"797debadf5ad6801c24fdc7dda62a61e235faf745451f7499c35a815f593a599"} err="failed to get container status \"797debadf5ad6801c24fdc7dda62a61e235faf745451f7499c35a815f593a599\": rpc error: code = NotFound desc = could not find container \"797debadf5ad6801c24fdc7dda62a61e235faf745451f7499c35a815f593a599\": container with ID starting with 797debadf5ad6801c24fdc7dda62a61e235faf745451f7499c35a815f593a599 not found: ID does not exist"
Oct 31 01:14:57.566448 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:14:57.566394    4284 scope.go:122] "RemoveContainer" containerID="312431d8e47135034e52b59c2e1580ad8f5f326404a208ce1394d5f3c7fd1b9e"
Oct 31 01:14:57.575619 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:14:57.575601    4284 scope.go:122] "RemoveContainer" containerID="312431d8e47135034e52b59c2e1580ad8f5f326404a208ce1394d5f3c7fd1b9e"
Oct 31 01:14:57.575890 ip-10-0-132-173 kubenswrapper[4284]: E1031 01:14:57.575873    4284 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"312431d8e47135034e52b59c2e1580ad8f5f326404a208ce1394d5f3c7fd1b9e\": container with ID starting with 312431d8e47135034e52b59c2e1580ad8f5f326404a208ce1394d5f3c7fd1b9e not found: ID does not exist" containerID="312431d8e47135034e52b59c2e1580ad8f5f326404a208ce1394d5f3c7fd1b9e"
Oct 31 01:14:57.575937 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:14:57.575911    4284 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"312431d8e47135034e52b59c2e1580ad8f5f326404a208ce1394d5f3c7fd1b9e"} err="failed to get container status \"312431d8e47135034e52b59c2e1580ad8f5f326404a208ce1394d5f3c7fd1b9e\": rpc error: code = NotFound desc = could not find container \"312431d8e47135034e52b59c2e1580ad8f5f326404a208ce1394d5f3c7fd1b9e\": container with ID starting with 312431d8e47135034e52b59c2e1580ad8f5f326404a208ce1394d5f3c7fd1b9e not found: ID does not exist"
Oct 31 01:14:57.575937 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:14:57.575928    4284 scope.go:122] "RemoveContainer" containerID="388a516c0044a468282de7173c84f5f0fffb739e3f990814333f9a4eca30fe85"
Oct 31 01:14:57.584635 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:14:57.584620    4284 scope.go:122] "RemoveContainer" containerID="388a516c0044a468282de7173c84f5f0fffb739e3f990814333f9a4eca30fe85"
Oct 31 01:14:57.584879 ip-10-0-132-173 kubenswrapper[4284]: E1031 01:14:57.584860    4284 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"388a516c0044a468282de7173c84f5f0fffb739e3f990814333f9a4eca30fe85\": container with ID starting with 388a516c0044a468282de7173c84f5f0fffb739e3f990814333f9a4eca30fe85 not found: ID does not exist" containerID="388a516c0044a468282de7173c84f5f0fffb739e3f990814333f9a4eca30fe85"
Oct 31 01:14:57.584925 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:14:57.584887    4284 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"388a516c0044a468282de7173c84f5f0fffb739e3f990814333f9a4eca30fe85"} err="failed to get container status \"388a516c0044a468282de7173c84f5f0fffb739e3f990814333f9a4eca30fe85\": rpc error: code = NotFound desc = could not find container \"388a516c0044a468282de7173c84f5f0fffb739e3f990814333f9a4eca30fe85\": container with ID starting with 388a516c0044a468282de7173c84f5f0fffb739e3f990814333f9a4eca30fe85 not found: ID does not exist"
Oct 31 01:14:57.606662 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:14:57.606627    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6a061135-19b6-49e7-8917-b3a9a4274488-kube-api-access-wh5hk\" (UniqueName: \"kubernetes.io/projected/6a061135-19b6-49e7-8917-b3a9a4274488-kube-api-access-wh5hk\") pod \"6a061135-19b6-49e7-8917-b3a9a4274488\" (UID: \"6a061135-19b6-49e7-8917-b3a9a4274488\") "
Oct 31 01:14:57.606810 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:14:57.606744    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/6a061135-19b6-49e7-8917-b3a9a4274488-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6a061135-19b6-49e7-8917-b3a9a4274488-plugins-config-volume\") pod \"6a061135-19b6-49e7-8917-b3a9a4274488\" (UID: \"6a061135-19b6-49e7-8917-b3a9a4274488\") "
Oct 31 01:14:57.606810 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:14:57.606798    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/2d3da89a-1e5f-41af-95b5-6910bdd24aad-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2d3da89a-1e5f-41af-95b5-6910bdd24aad-plugins-config-volume\") pod \"2d3da89a-1e5f-41af-95b5-6910bdd24aad\" (UID: \"2d3da89a-1e5f-41af-95b5-6910bdd24aad\") "
Oct 31 01:14:57.606937 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:14:57.606865    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2d3da89a-1e5f-41af-95b5-6910bdd24aad-kube-api-access-7f9jj\" (UniqueName: \"kubernetes.io/projected/2d3da89a-1e5f-41af-95b5-6910bdd24aad-kube-api-access-7f9jj\") pod \"2d3da89a-1e5f-41af-95b5-6910bdd24aad\" (UID: \"2d3da89a-1e5f-41af-95b5-6910bdd24aad\") "
Oct 31 01:14:57.607243 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:14:57.607133    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6a061135-19b6-49e7-8917-b3a9a4274488-plugins-config-volume" pod "6a061135-19b6-49e7-8917-b3a9a4274488" (UID: "6a061135-19b6-49e7-8917-b3a9a4274488"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 01:14:57.607352 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:14:57.607329    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2d3da89a-1e5f-41af-95b5-6910bdd24aad-plugins-config-volume" pod "2d3da89a-1e5f-41af-95b5-6910bdd24aad" (UID: "2d3da89a-1e5f-41af-95b5-6910bdd24aad"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 01:14:57.607457 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:14:57.607440    4284 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6a061135-19b6-49e7-8917-b3a9a4274488-plugins-config-volume\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:14:57.607530 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:14:57.607464    4284 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2d3da89a-1e5f-41af-95b5-6910bdd24aad-plugins-config-volume\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:14:57.609590 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:14:57.609569    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2d3da89a-1e5f-41af-95b5-6910bdd24aad-kube-api-access-7f9jj" pod "2d3da89a-1e5f-41af-95b5-6910bdd24aad" (UID: "2d3da89a-1e5f-41af-95b5-6910bdd24aad"). InnerVolumeSpecName "kube-api-access-7f9jj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:14:57.609687 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:14:57.609639    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6a061135-19b6-49e7-8917-b3a9a4274488-kube-api-access-wh5hk" pod "6a061135-19b6-49e7-8917-b3a9a4274488" (UID: "6a061135-19b6-49e7-8917-b3a9a4274488"). InnerVolumeSpecName "kube-api-access-wh5hk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:14:57.750142 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:14:57.742298    4284 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wh5hk\" (UniqueName: \"kubernetes.io/projected/6a061135-19b6-49e7-8917-b3a9a4274488-kube-api-access-wh5hk\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:14:57.750142 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:14:57.742328    4284 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7f9jj\" (UniqueName: \"kubernetes.io/projected/2d3da89a-1e5f-41af-95b5-6910bdd24aad-kube-api-access-7f9jj\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:14:57.797706 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:14:57.797661    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-f1d081-5b8f78496b-xz9lw"]
Oct 31 01:14:57.800390 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:14:57.800364    4284 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-f1d081-5b8f78496b-xz9lw"]
Oct 31 01:14:57.964857 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:14:57.964824    4284 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="dda5d4ad-a454-43fb-96ff-664f19acc3b4" path="/var/lib/kubelet/pods/dda5d4ad-a454-43fb-96ff-664f19acc3b4/volumes"
Oct 31 01:14:58.026705 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:14:58.026676    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-f1d081-5c766b5c87-46qzr"]
Oct 31 01:14:58.034591 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:14:58.034565    4284 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-f1d081-5c766b5c87-46qzr"]
Oct 31 01:14:58.150680 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:14:58.150580    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-f1d081-8bc8c89c8-j7s55"]
Oct 31 01:14:58.153783 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:14:58.153758    4284 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-f1d081-8bc8c89c8-j7s55"]
Oct 31 01:14:58.323466 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:14:58.323433    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:14:58.599608 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:14:58.599579    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:14:58.742795 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:14:58.742755    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:14:59.366434 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:14:59.366396    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:14:59.948113 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:14:59.948067    4284 prober.go:122] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-f1d081-5b8f78496b-xz9lw" podUID="dda5d4ad-a454-43fb-96ff-664f19acc3b4" containerName="maas-api" probeResult="failure" output="Get \"https://10.134.0.149:8443/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 01:14:59.963865 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:14:59.963830    4284 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2d3da89a-1e5f-41af-95b5-6910bdd24aad" path="/var/lib/kubelet/pods/2d3da89a-1e5f-41af-95b5-6910bdd24aad/volumes"
Oct 31 01:14:59.964388 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:14:59.964367    4284 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6a061135-19b6-49e7-8917-b3a9a4274488" path="/var/lib/kubelet/pods/6a061135-19b6-49e7-8917-b3a9a4274488/volumes"
Oct 31 01:15:00.050790 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:00.050752    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-rr5k6"]
Oct 31 01:15:00.050974 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:00.050954    4284 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-rr5k6" podUID="27f8cf21-0f47-4397-831a-bf716741dc67" containerName="istio-proxy" containerID="cri-o://35067cce7b522d2302a926596827c13d7fb4d828f291d22318b7fbcccaa5c162" gracePeriod=30
Oct 31 01:15:00.153407 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:00.153368    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-b63c24-openshift-default-d8c64855f-wfrh9"]
Oct 31 01:15:00.153652 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:00.153623    4284 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w1-b63c24-openshift-default-d8c64855f-wfrh9" podUID="11338a78-9c55-4df1-8562-360ce3c79c04" containerName="istio-proxy" containerID="cri-o://0cb7e4f8dc40d7a3c46db72e795056052bfe8ad45080151dd5ad4e3b69ba4aa1" gracePeriod=30
Oct 31 01:15:00.275306 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:00.275201    4284 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w0-7f8596-29890155-lzdg7"]
Oct 31 01:15:00.275799 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:00.275786    4284 cpu_manager.go:414] "containerMap: removing container" podUID="dda5d4ad-a454-43fb-96ff-664f19acc3b4" containerName="maas-api"
Oct 31 01:15:00.275861 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:00.275801    4284 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="dda5d4ad-a454-43fb-96ff-664f19acc3b4" containerName="maas-api"
Oct 31 01:15:00.275861 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:00.275824    4284 cpu_manager.go:414] "containerMap: removing container" podUID="6a061135-19b6-49e7-8917-b3a9a4274488" containerName="payload-processing"
Oct 31 01:15:00.275861 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:00.275829    4284 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6a061135-19b6-49e7-8917-b3a9a4274488" containerName="payload-processing"
Oct 31 01:15:00.275861 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:00.275851    4284 cpu_manager.go:414] "containerMap: removing container" podUID="2d3da89a-1e5f-41af-95b5-6910bdd24aad" containerName="payload-pre-processing"
Oct 31 01:15:00.275861 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:00.275856    4284 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2d3da89a-1e5f-41af-95b5-6910bdd24aad" containerName="payload-pre-processing"
Oct 31 01:15:00.276052 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:00.275936    4284 memory_manager.go:365] "RemoveStaleState removing state" podUID="dda5d4ad-a454-43fb-96ff-664f19acc3b4" containerName="maas-api"
Oct 31 01:15:00.276052 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:00.275945    4284 memory_manager.go:365] "RemoveStaleState removing state" podUID="2d3da89a-1e5f-41af-95b5-6910bdd24aad" containerName="payload-pre-processing"
Oct 31 01:15:00.276052 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:00.275952    4284 memory_manager.go:365] "RemoveStaleState removing state" podUID="6a061135-19b6-49e7-8917-b3a9a4274488" containerName="payload-processing"
Oct 31 01:15:00.279457 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:00.279438    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w0-7f8596-29890155-lzdg7"
Oct 31 01:15:00.301024 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:00.300995    4284 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-f9925a-29890155-hj42v"]
Oct 31 01:15:00.304553 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:00.304532    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-f9925a-29890155-hj42v"
Oct 31 01:15:00.344173 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:00.344141    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w0-7f8596-29890155-lzdg7"]
Oct 31 01:15:00.370529 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:00.370495    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-f9925a-29890155-hj42v"]
Oct 31 01:15:00.371708 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:00.371689    4284 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-8pzqp\""
Oct 31 01:15:00.384301 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:00.383638    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pshdh\" (UniqueName: \"kubernetes.io/projected/abe955f6-a7a7-4001-a592-c9c69b71b025-kube-api-access-pshdh\") pod \"maas-api-key-cleanup-e2e-shared-b-w2-f9925a-29890155-hj42v\" (UID: \"abe955f6-a7a7-4001-a592-c9c69b71b025\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-f9925a-29890155-hj42v"
Oct 31 01:15:00.384301 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:00.383733    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dfdcd040-7c98-4a93-83b3-13532fda03a8-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-shared-b-w0-7f8596-29890155-lzdg7\" (UID: \"dfdcd040-7c98-4a93-83b3-13532fda03a8\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w0-7f8596-29890155-lzdg7"
Oct 31 01:15:00.384301 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:00.383834    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lxrw2\" (UniqueName: \"kubernetes.io/projected/dfdcd040-7c98-4a93-83b3-13532fda03a8-kube-api-access-lxrw2\") pod \"maas-api-key-cleanup-e2e-shared-b-w0-7f8596-29890155-lzdg7\" (UID: \"dfdcd040-7c98-4a93-83b3-13532fda03a8\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w0-7f8596-29890155-lzdg7"
Oct 31 01:15:00.384301 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:00.383982    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/abe955f6-a7a7-4001-a592-c9c69b71b025-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-shared-b-w2-f9925a-29890155-hj42v\" (UID: \"abe955f6-a7a7-4001-a592-c9c69b71b025\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-f9925a-29890155-hj42v"
Oct 31 01:15:00.485604 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:00.485566    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dfdcd040-7c98-4a93-83b3-13532fda03a8-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-shared-b-w0-7f8596-29890155-lzdg7\" (UID: \"dfdcd040-7c98-4a93-83b3-13532fda03a8\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w0-7f8596-29890155-lzdg7"
Oct 31 01:15:00.485863 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:00.485691    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lxrw2\" (UniqueName: \"kubernetes.io/projected/dfdcd040-7c98-4a93-83b3-13532fda03a8-kube-api-access-lxrw2\") pod \"maas-api-key-cleanup-e2e-shared-b-w0-7f8596-29890155-lzdg7\" (UID: \"dfdcd040-7c98-4a93-83b3-13532fda03a8\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w0-7f8596-29890155-lzdg7"
Oct 31 01:15:00.485863 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:00.485817    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/abe955f6-a7a7-4001-a592-c9c69b71b025-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-shared-b-w2-f9925a-29890155-hj42v\" (UID: \"abe955f6-a7a7-4001-a592-c9c69b71b025\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-f9925a-29890155-hj42v"
Oct 31 01:15:00.486089 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:00.485874    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pshdh\" (UniqueName: \"kubernetes.io/projected/abe955f6-a7a7-4001-a592-c9c69b71b025-kube-api-access-pshdh\") pod \"maas-api-key-cleanup-e2e-shared-b-w2-f9925a-29890155-hj42v\" (UID: \"abe955f6-a7a7-4001-a592-c9c69b71b025\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-f9925a-29890155-hj42v"
Oct 31 01:15:00.486373 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:00.486355    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dfdcd040-7c98-4a93-83b3-13532fda03a8-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-shared-b-w0-7f8596-29890155-lzdg7\" (UID: \"dfdcd040-7c98-4a93-83b3-13532fda03a8\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w0-7f8596-29890155-lzdg7"
Oct 31 01:15:00.486518 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:00.486499    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/abe955f6-a7a7-4001-a592-c9c69b71b025-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-shared-b-w2-f9925a-29890155-hj42v\" (UID: \"abe955f6-a7a7-4001-a592-c9c69b71b025\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-f9925a-29890155-hj42v"
Oct 31 01:15:00.550695 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:00.550624    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lxrw2\" (UniqueName: \"kubernetes.io/projected/dfdcd040-7c98-4a93-83b3-13532fda03a8-kube-api-access-lxrw2\") pod \"maas-api-key-cleanup-e2e-shared-b-w0-7f8596-29890155-lzdg7\" (UID: \"dfdcd040-7c98-4a93-83b3-13532fda03a8\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w0-7f8596-29890155-lzdg7"
Oct 31 01:15:00.555165 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:00.555138    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pshdh\" (UniqueName: \"kubernetes.io/projected/abe955f6-a7a7-4001-a592-c9c69b71b025-kube-api-access-pshdh\") pod \"maas-api-key-cleanup-e2e-shared-b-w2-f9925a-29890155-hj42v\" (UID: \"abe955f6-a7a7-4001-a592-c9c69b71b025\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-f9925a-29890155-hj42v"
Oct 31 01:15:00.593567 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:00.593541    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w0-7f8596-29890155-lzdg7"
Oct 31 01:15:00.634884 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:00.634847    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-f9925a-29890155-hj42v"
Oct 31 01:15:00.967549 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:00.967523    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w0-7f8596-29890155-lzdg7"]
Oct 31 01:15:00.967687 ip-10-0-132-173 kubenswrapper[4284]: W1031 01:15:00.967576    4284 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddfdcd040_7c98_4a93_83b3_13532fda03a8.slice/crio-a65975fe921d1867220716acad5f9dddf9ac89e2fb9e2518b0eba843446425e7 WatchSource:0}: Error finding container a65975fe921d1867220716acad5f9dddf9ac89e2fb9e2518b0eba843446425e7: Status 404 returned error can't find the container with id a65975fe921d1867220716acad5f9dddf9ac89e2fb9e2518b0eba843446425e7
Oct 31 01:15:01.013301 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:01.013270    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-f9925a-29890155-hj42v"]
Oct 31 01:15:01.014256 ip-10-0-132-173 kubenswrapper[4284]: W1031 01:15:01.014229    4284 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podabe955f6_a7a7_4001_a592_c9c69b71b025.slice/crio-fa07a88a67867b6fbbef7d339463506b033c938d78babb54e9ea86a6d4ab5a73 WatchSource:0}: Error finding container fa07a88a67867b6fbbef7d339463506b033c938d78babb54e9ea86a6d4ab5a73: Status 404 returned error can't find the container with id fa07a88a67867b6fbbef7d339463506b033c938d78babb54e9ea86a6d4ab5a73
Oct 31 01:15:01.588162 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:01.588068    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-f9925a-29890155-hj42v" event={"ID":"abe955f6-a7a7-4001-a592-c9c69b71b025","Type":"ContainerStarted","Data":"ec126db6a3a5eefcef180baf1aa8a765abc27da3fec057190039d2db2549a46a"}
Oct 31 01:15:01.588162 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:01.588129    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-f9925a-29890155-hj42v" event={"ID":"abe955f6-a7a7-4001-a592-c9c69b71b025","Type":"ContainerStarted","Data":"fa07a88a67867b6fbbef7d339463506b033c938d78babb54e9ea86a6d4ab5a73"}
Oct 31 01:15:01.589803 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:01.589777    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w0-7f8596-29890155-lzdg7" event={"ID":"dfdcd040-7c98-4a93-83b3-13532fda03a8","Type":"ContainerStarted","Data":"9ee009ff717cacc37f59b62468d0b1d4ced5f89334d4738f0b864138feb41afe"}
Oct 31 01:15:01.589908 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:01.589826    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w0-7f8596-29890155-lzdg7" event={"ID":"dfdcd040-7c98-4a93-83b3-13532fda03a8","Type":"ContainerStarted","Data":"a65975fe921d1867220716acad5f9dddf9ac89e2fb9e2518b0eba843446425e7"}
Oct 31 01:15:01.703652 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:01.703598    4284 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-f9925a-29890155-hj42v" podStartSLOduration=1.7035858689999999 podStartE2EDuration="1.703585869s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 01:15:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 01:15:01.700948115 +0000 UTC m=+2815.845955031" watchObservedRunningTime="2026-10-31 01:15:01.703585869 +0000 UTC m=+2815.848592826"
Oct 31 01:15:01.812458 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:01.812403    4284 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w0-7f8596-29890155-lzdg7" podStartSLOduration=1.812388892 podStartE2EDuration="1.812388892s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 01:15:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 01:15:01.81210077 +0000 UTC m=+2815.957107687" watchObservedRunningTime="2026-10-31 01:15:01.812388892 +0000 UTC m=+2815.957395807"
Oct 31 01:15:02.600263 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:02.600227    4284 generic.go:356] "Generic (PLEG): container finished" podID="abe955f6-a7a7-4001-a592-c9c69b71b025" containerID="ec126db6a3a5eefcef180baf1aa8a765abc27da3fec057190039d2db2549a46a" exitCode=60
Oct 31 01:15:02.600791 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:02.600263    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-f9925a-29890155-hj42v" event={"ID":"abe955f6-a7a7-4001-a592-c9c69b71b025","Type":"ContainerDied","Data":"ec126db6a3a5eefcef180baf1aa8a765abc27da3fec057190039d2db2549a46a"}
Oct 31 01:15:02.600791 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:02.600672    4284 scope.go:122] "RemoveContainer" containerID="ec126db6a3a5eefcef180baf1aa8a765abc27da3fec057190039d2db2549a46a"
Oct 31 01:15:02.602189 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:02.602160    4284 generic.go:356] "Generic (PLEG): container finished" podID="dfdcd040-7c98-4a93-83b3-13532fda03a8" containerID="9ee009ff717cacc37f59b62468d0b1d4ced5f89334d4738f0b864138feb41afe" exitCode=60
Oct 31 01:15:02.602344 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:02.602206    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w0-7f8596-29890155-lzdg7" event={"ID":"dfdcd040-7c98-4a93-83b3-13532fda03a8","Type":"ContainerDied","Data":"9ee009ff717cacc37f59b62468d0b1d4ced5f89334d4738f0b864138feb41afe"}
Oct 31 01:15:02.602516 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:02.602501    4284 scope.go:122] "RemoveContainer" containerID="9ee009ff717cacc37f59b62468d0b1d4ced5f89334d4738f0b864138feb41afe"
Oct 31 01:15:03.554611 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:03.554563    4284 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6cc458d887-rr5k6 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 31 01:15:03.554787 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:03.554640    4284 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-rr5k6" podUID="27f8cf21-0f47-4397-831a-bf716741dc67" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 31 01:15:03.615264 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:03.615234    4284 generic.go:356] "Generic (PLEG): container finished" podID="abe955f6-a7a7-4001-a592-c9c69b71b025" containerID="b10b3c880403d71b7663f080e14777168876efe5ece0537e57addcc7d0b494d6" exitCode=60
Oct 31 01:15:03.615764 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:03.615312    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-f9925a-29890155-hj42v" event={"ID":"abe955f6-a7a7-4001-a592-c9c69b71b025","Type":"ContainerDied","Data":"b10b3c880403d71b7663f080e14777168876efe5ece0537e57addcc7d0b494d6"}
Oct 31 01:15:03.615764 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:03.615370    4284 scope.go:122] "RemoveContainer" containerID="ec126db6a3a5eefcef180baf1aa8a765abc27da3fec057190039d2db2549a46a"
Oct 31 01:15:03.615764 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:03.615692    4284 scope.go:122] "RemoveContainer" containerID="b10b3c880403d71b7663f080e14777168876efe5ece0537e57addcc7d0b494d6"
Oct 31 01:15:03.616084 ip-10-0-132-173 kubenswrapper[4284]: E1031 01:15:03.616060    4284 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-shared-b-w2-f9925a-29890155-hj42v_odh-ai-gateway-infra(abe955f6-a7a7-4001-a592-c9c69b71b025)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-f9925a-29890155-hj42v" podUID="abe955f6-a7a7-4001-a592-c9c69b71b025"
Oct 31 01:15:03.617252 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:03.617228    4284 generic.go:356] "Generic (PLEG): container finished" podID="dfdcd040-7c98-4a93-83b3-13532fda03a8" containerID="de267f22d67394a7349acd4ce0d82485823243fd243e2737ee1ef58fcec418a4" exitCode=60
Oct 31 01:15:03.617349 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:03.617257    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w0-7f8596-29890155-lzdg7" event={"ID":"dfdcd040-7c98-4a93-83b3-13532fda03a8","Type":"ContainerDied","Data":"de267f22d67394a7349acd4ce0d82485823243fd243e2737ee1ef58fcec418a4"}
Oct 31 01:15:03.617552 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:03.617540    4284 scope.go:122] "RemoveContainer" containerID="de267f22d67394a7349acd4ce0d82485823243fd243e2737ee1ef58fcec418a4"
Oct 31 01:15:03.617811 ip-10-0-132-173 kubenswrapper[4284]: E1031 01:15:03.617790    4284 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-shared-b-w0-7f8596-29890155-lzdg7_odh-ai-gateway-infra(dfdcd040-7c98-4a93-83b3-13532fda03a8)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w0-7f8596-29890155-lzdg7" podUID="dfdcd040-7c98-4a93-83b3-13532fda03a8"
Oct 31 01:15:03.703701 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:03.703674    4284 scope.go:122] "RemoveContainer" containerID="9ee009ff717cacc37f59b62468d0b1d4ced5f89334d4738f0b864138feb41afe"
Oct 31 01:15:04.010541 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:04.010506    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:15:04.626396 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:04.626365    4284 scope.go:122] "RemoveContainer" containerID="b10b3c880403d71b7663f080e14777168876efe5ece0537e57addcc7d0b494d6"
Oct 31 01:15:04.626903 ip-10-0-132-173 kubenswrapper[4284]: E1031 01:15:04.626713    4284 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-shared-b-w2-f9925a-29890155-hj42v_odh-ai-gateway-infra(abe955f6-a7a7-4001-a592-c9c69b71b025)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-f9925a-29890155-hj42v" podUID="abe955f6-a7a7-4001-a592-c9c69b71b025"
Oct 31 01:15:04.627908 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:04.627885    4284 scope.go:122] "RemoveContainer" containerID="de267f22d67394a7349acd4ce0d82485823243fd243e2737ee1ef58fcec418a4"
Oct 31 01:15:04.628229 ip-10-0-132-173 kubenswrapper[4284]: E1031 01:15:04.628189    4284 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-shared-b-w0-7f8596-29890155-lzdg7_odh-ai-gateway-infra(dfdcd040-7c98-4a93-83b3-13532fda03a8)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w0-7f8596-29890155-lzdg7" podUID="dfdcd040-7c98-4a93-83b3-13532fda03a8"
Oct 31 01:15:04.652859 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:04.652822    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:15:05.331625 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:05.331599    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-rr5k6"
Oct 31 01:15:05.434127 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:05.434101    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-b63c24-openshift-default-d8c64855f-wfrh9"
Oct 31 01:15:05.490344 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:05.490311    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/27f8cf21-0f47-4397-831a-bf716741dc67-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/27f8cf21-0f47-4397-831a-bf716741dc67-workload-certs\") pod \"27f8cf21-0f47-4397-831a-bf716741dc67\" (UID: \"27f8cf21-0f47-4397-831a-bf716741dc67\") "
Oct 31 01:15:05.490532 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:05.490353    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/27f8cf21-0f47-4397-831a-bf716741dc67-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/27f8cf21-0f47-4397-831a-bf716741dc67-istio-envoy\") pod \"27f8cf21-0f47-4397-831a-bf716741dc67\" (UID: \"27f8cf21-0f47-4397-831a-bf716741dc67\") "
Oct 31 01:15:05.490532 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:05.490441    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/27f8cf21-0f47-4397-831a-bf716741dc67-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/27f8cf21-0f47-4397-831a-bf716741dc67-istiod-ca-cert\") pod \"27f8cf21-0f47-4397-831a-bf716741dc67\" (UID: \"27f8cf21-0f47-4397-831a-bf716741dc67\") "
Oct 31 01:15:05.490532 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:05.490495    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/27f8cf21-0f47-4397-831a-bf716741dc67-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/27f8cf21-0f47-4397-831a-bf716741dc67-istio-data\") pod \"27f8cf21-0f47-4397-831a-bf716741dc67\" (UID: \"27f8cf21-0f47-4397-831a-bf716741dc67\") "
Oct 31 01:15:05.490704 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:05.490544    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/27f8cf21-0f47-4397-831a-bf716741dc67-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/27f8cf21-0f47-4397-831a-bf716741dc67-workload-socket\") pod \"27f8cf21-0f47-4397-831a-bf716741dc67\" (UID: \"27f8cf21-0f47-4397-831a-bf716741dc67\") "
Oct 31 01:15:05.490704 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:05.490629    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/27f8cf21-0f47-4397-831a-bf716741dc67-kube-api-access-slzp9\" (UniqueName: \"kubernetes.io/projected/27f8cf21-0f47-4397-831a-bf716741dc67-kube-api-access-slzp9\") pod \"27f8cf21-0f47-4397-831a-bf716741dc67\" (UID: \"27f8cf21-0f47-4397-831a-bf716741dc67\") "
Oct 31 01:15:05.490704 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:05.490655    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/27f8cf21-0f47-4397-831a-bf716741dc67-workload-certs" pod "27f8cf21-0f47-4397-831a-bf716741dc67" (UID: "27f8cf21-0f47-4397-831a-bf716741dc67"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:15:05.490704 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:05.490695    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/27f8cf21-0f47-4397-831a-bf716741dc67-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/27f8cf21-0f47-4397-831a-bf716741dc67-istio-podinfo\") pod \"27f8cf21-0f47-4397-831a-bf716741dc67\" (UID: \"27f8cf21-0f47-4397-831a-bf716741dc67\") "
Oct 31 01:15:05.490931 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:05.490829    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/27f8cf21-0f47-4397-831a-bf716741dc67-istio-token\" (UniqueName: \"kubernetes.io/projected/27f8cf21-0f47-4397-831a-bf716741dc67-istio-token\") pod \"27f8cf21-0f47-4397-831a-bf716741dc67\" (UID: \"27f8cf21-0f47-4397-831a-bf716741dc67\") "
Oct 31 01:15:05.490931 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:05.490850    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/27f8cf21-0f47-4397-831a-bf716741dc67-workload-socket" pod "27f8cf21-0f47-4397-831a-bf716741dc67" (UID: "27f8cf21-0f47-4397-831a-bf716741dc67"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:15:05.490931 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:05.490857    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/27f8cf21-0f47-4397-831a-bf716741dc67-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/27f8cf21-0f47-4397-831a-bf716741dc67-credential-socket\") pod \"27f8cf21-0f47-4397-831a-bf716741dc67\" (UID: \"27f8cf21-0f47-4397-831a-bf716741dc67\") "
Oct 31 01:15:05.491121 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:05.490929    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/27f8cf21-0f47-4397-831a-bf716741dc67-istiod-ca-cert" pod "27f8cf21-0f47-4397-831a-bf716741dc67" (UID: "27f8cf21-0f47-4397-831a-bf716741dc67"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 01:15:05.491185 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:05.491119    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/27f8cf21-0f47-4397-831a-bf716741dc67-istio-data" pod "27f8cf21-0f47-4397-831a-bf716741dc67" (UID: "27f8cf21-0f47-4397-831a-bf716741dc67"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:15:05.491379 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:05.491356    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/27f8cf21-0f47-4397-831a-bf716741dc67-credential-socket" pod "27f8cf21-0f47-4397-831a-bf716741dc67" (UID: "27f8cf21-0f47-4397-831a-bf716741dc67"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:15:05.491641 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:05.491584    4284 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/27f8cf21-0f47-4397-831a-bf716741dc67-istiod-ca-cert\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:15:05.491641 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:05.491614    4284 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/27f8cf21-0f47-4397-831a-bf716741dc67-istio-data\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:15:05.491641 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:05.491628    4284 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/27f8cf21-0f47-4397-831a-bf716741dc67-workload-socket\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:15:05.491641 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:05.491642    4284 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/27f8cf21-0f47-4397-831a-bf716741dc67-credential-socket\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:15:05.491641 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:05.491655    4284 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/27f8cf21-0f47-4397-831a-bf716741dc67-workload-certs\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:15:05.493267 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:05.493243    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/27f8cf21-0f47-4397-831a-bf716741dc67-istio-token" pod "27f8cf21-0f47-4397-831a-bf716741dc67" (UID: "27f8cf21-0f47-4397-831a-bf716741dc67"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:15:05.493756 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:05.493738    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/27f8cf21-0f47-4397-831a-bf716741dc67-istio-podinfo" pod "27f8cf21-0f47-4397-831a-bf716741dc67" (UID: "27f8cf21-0f47-4397-831a-bf716741dc67"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 01:15:05.493970 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:05.493954    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/27f8cf21-0f47-4397-831a-bf716741dc67-istio-envoy" pod "27f8cf21-0f47-4397-831a-bf716741dc67" (UID: "27f8cf21-0f47-4397-831a-bf716741dc67"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:15:05.494191 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:05.494174    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/27f8cf21-0f47-4397-831a-bf716741dc67-kube-api-access-slzp9" pod "27f8cf21-0f47-4397-831a-bf716741dc67" (UID: "27f8cf21-0f47-4397-831a-bf716741dc67"). InnerVolumeSpecName "kube-api-access-slzp9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:15:05.592674 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:05.592582    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/11338a78-9c55-4df1-8562-360ce3c79c04-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/11338a78-9c55-4df1-8562-360ce3c79c04-istio-podinfo\") pod \"11338a78-9c55-4df1-8562-360ce3c79c04\" (UID: \"11338a78-9c55-4df1-8562-360ce3c79c04\") "
Oct 31 01:15:05.592674 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:05.592648    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/11338a78-9c55-4df1-8562-360ce3c79c04-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/11338a78-9c55-4df1-8562-360ce3c79c04-workload-certs\") pod \"11338a78-9c55-4df1-8562-360ce3c79c04\" (UID: \"11338a78-9c55-4df1-8562-360ce3c79c04\") "
Oct 31 01:15:05.592937 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:05.592760    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/11338a78-9c55-4df1-8562-360ce3c79c04-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/11338a78-9c55-4df1-8562-360ce3c79c04-credential-socket\") pod \"11338a78-9c55-4df1-8562-360ce3c79c04\" (UID: \"11338a78-9c55-4df1-8562-360ce3c79c04\") "
Oct 31 01:15:05.592937 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:05.592829    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/11338a78-9c55-4df1-8562-360ce3c79c04-istio-token\" (UniqueName: \"kubernetes.io/projected/11338a78-9c55-4df1-8562-360ce3c79c04-istio-token\") pod \"11338a78-9c55-4df1-8562-360ce3c79c04\" (UID: \"11338a78-9c55-4df1-8562-360ce3c79c04\") "
Oct 31 01:15:05.592937 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:05.592862    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/11338a78-9c55-4df1-8562-360ce3c79c04-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/11338a78-9c55-4df1-8562-360ce3c79c04-istiod-ca-cert\") pod \"11338a78-9c55-4df1-8562-360ce3c79c04\" (UID: \"11338a78-9c55-4df1-8562-360ce3c79c04\") "
Oct 31 01:15:05.592937 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:05.592904    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/11338a78-9c55-4df1-8562-360ce3c79c04-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/11338a78-9c55-4df1-8562-360ce3c79c04-istio-data\") pod \"11338a78-9c55-4df1-8562-360ce3c79c04\" (UID: \"11338a78-9c55-4df1-8562-360ce3c79c04\") "
Oct 31 01:15:05.593172 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:05.592968    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/11338a78-9c55-4df1-8562-360ce3c79c04-kube-api-access-bgfdb\" (UniqueName: \"kubernetes.io/projected/11338a78-9c55-4df1-8562-360ce3c79c04-kube-api-access-bgfdb\") pod \"11338a78-9c55-4df1-8562-360ce3c79c04\" (UID: \"11338a78-9c55-4df1-8562-360ce3c79c04\") "
Oct 31 01:15:05.593172 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:05.593017    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/11338a78-9c55-4df1-8562-360ce3c79c04-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/11338a78-9c55-4df1-8562-360ce3c79c04-istio-envoy\") pod \"11338a78-9c55-4df1-8562-360ce3c79c04\" (UID: \"11338a78-9c55-4df1-8562-360ce3c79c04\") "
Oct 31 01:15:05.593172 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:05.593089    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/11338a78-9c55-4df1-8562-360ce3c79c04-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/11338a78-9c55-4df1-8562-360ce3c79c04-workload-socket\") pod \"11338a78-9c55-4df1-8562-360ce3c79c04\" (UID: \"11338a78-9c55-4df1-8562-360ce3c79c04\") "
Oct 31 01:15:05.593172 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:05.593139    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/11338a78-9c55-4df1-8562-360ce3c79c04-workload-certs" pod "11338a78-9c55-4df1-8562-360ce3c79c04" (UID: "11338a78-9c55-4df1-8562-360ce3c79c04"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:15:05.593172 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:05.593166    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/11338a78-9c55-4df1-8562-360ce3c79c04-credential-socket" pod "11338a78-9c55-4df1-8562-360ce3c79c04" (UID: "11338a78-9c55-4df1-8562-360ce3c79c04"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:15:05.593916 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:05.593892    4284 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/27f8cf21-0f47-4397-831a-bf716741dc67-istio-token\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:15:05.594069 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:05.593922    4284 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/27f8cf21-0f47-4397-831a-bf716741dc67-istio-envoy\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:15:05.594069 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:05.593941    4284 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-slzp9\" (UniqueName: \"kubernetes.io/projected/27f8cf21-0f47-4397-831a-bf716741dc67-kube-api-access-slzp9\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:15:05.594069 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:05.593958    4284 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/11338a78-9c55-4df1-8562-360ce3c79c04-workload-certs\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:15:05.594069 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:05.593971    4284 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/27f8cf21-0f47-4397-831a-bf716741dc67-istio-podinfo\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:15:05.594069 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:05.593986    4284 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/11338a78-9c55-4df1-8562-360ce3c79c04-credential-socket\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:15:05.595139 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:05.595099    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/11338a78-9c55-4df1-8562-360ce3c79c04-istio-podinfo" pod "11338a78-9c55-4df1-8562-360ce3c79c04" (UID: "11338a78-9c55-4df1-8562-360ce3c79c04"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 01:15:05.595354 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:05.595330    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/11338a78-9c55-4df1-8562-360ce3c79c04-istio-data" pod "11338a78-9c55-4df1-8562-360ce3c79c04" (UID: "11338a78-9c55-4df1-8562-360ce3c79c04"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:15:05.595354 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:05.595338    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/11338a78-9c55-4df1-8562-360ce3c79c04-workload-socket" pod "11338a78-9c55-4df1-8562-360ce3c79c04" (UID: "11338a78-9c55-4df1-8562-360ce3c79c04"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:15:05.595494 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:05.595425    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/11338a78-9c55-4df1-8562-360ce3c79c04-istiod-ca-cert" pod "11338a78-9c55-4df1-8562-360ce3c79c04" (UID: "11338a78-9c55-4df1-8562-360ce3c79c04"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 01:15:05.597105 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:05.597082    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/11338a78-9c55-4df1-8562-360ce3c79c04-istio-token" pod "11338a78-9c55-4df1-8562-360ce3c79c04" (UID: "11338a78-9c55-4df1-8562-360ce3c79c04"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:15:05.597235 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:05.597152    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/11338a78-9c55-4df1-8562-360ce3c79c04-istio-envoy" pod "11338a78-9c55-4df1-8562-360ce3c79c04" (UID: "11338a78-9c55-4df1-8562-360ce3c79c04"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:15:05.597833 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:05.597810    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/11338a78-9c55-4df1-8562-360ce3c79c04-kube-api-access-bgfdb" pod "11338a78-9c55-4df1-8562-360ce3c79c04" (UID: "11338a78-9c55-4df1-8562-360ce3c79c04"). InnerVolumeSpecName "kube-api-access-bgfdb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:15:05.635341 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:05.635300    4284 generic.go:356] "Generic (PLEG): container finished" podID="11338a78-9c55-4df1-8562-360ce3c79c04" containerID="0cb7e4f8dc40d7a3c46db72e795056052bfe8ad45080151dd5ad4e3b69ba4aa1" exitCode=0
Oct 31 01:15:05.635841 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:05.635345    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-b63c24-openshift-default-d8c64855f-wfrh9"
Oct 31 01:15:05.635841 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:05.635385    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-b63c24-openshift-default-d8c64855f-wfrh9" event={"ID":"11338a78-9c55-4df1-8562-360ce3c79c04","Type":"ContainerDied","Data":"0cb7e4f8dc40d7a3c46db72e795056052bfe8ad45080151dd5ad4e3b69ba4aa1"}
Oct 31 01:15:05.635841 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:05.635440    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-b63c24-openshift-default-d8c64855f-wfrh9" event={"ID":"11338a78-9c55-4df1-8562-360ce3c79c04","Type":"ContainerDied","Data":"5f2f3930971db7a7c7584df3491d76f888ee81e7106283391fee77c3f5234ff0"}
Oct 31 01:15:05.635841 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:05.635462    4284 scope.go:122] "RemoveContainer" containerID="0cb7e4f8dc40d7a3c46db72e795056052bfe8ad45080151dd5ad4e3b69ba4aa1"
Oct 31 01:15:05.637157 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:05.637133    4284 generic.go:356] "Generic (PLEG): container finished" podID="27f8cf21-0f47-4397-831a-bf716741dc67" containerID="35067cce7b522d2302a926596827c13d7fb4d828f291d22318b7fbcccaa5c162" exitCode=0
Oct 31 01:15:05.637321 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:05.637195    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-rr5k6" event={"ID":"27f8cf21-0f47-4397-831a-bf716741dc67","Type":"ContainerDied","Data":"35067cce7b522d2302a926596827c13d7fb4d828f291d22318b7fbcccaa5c162"}
Oct 31 01:15:05.637321 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:05.637203    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-rr5k6"
Oct 31 01:15:05.637321 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:05.637270    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-rr5k6" event={"ID":"27f8cf21-0f47-4397-831a-bf716741dc67","Type":"ContainerDied","Data":"0e1c9de255ad075e1e2b93bb23ee5c402a2286a895ec280c3a83f08725e8ad4d"}
Oct 31 01:15:05.646112 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:05.646089    4284 scope.go:122] "RemoveContainer" containerID="0cb7e4f8dc40d7a3c46db72e795056052bfe8ad45080151dd5ad4e3b69ba4aa1"
Oct 31 01:15:05.646490 ip-10-0-132-173 kubenswrapper[4284]: E1031 01:15:05.646461    4284 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0cb7e4f8dc40d7a3c46db72e795056052bfe8ad45080151dd5ad4e3b69ba4aa1\": container with ID starting with 0cb7e4f8dc40d7a3c46db72e795056052bfe8ad45080151dd5ad4e3b69ba4aa1 not found: ID does not exist" containerID="0cb7e4f8dc40d7a3c46db72e795056052bfe8ad45080151dd5ad4e3b69ba4aa1"
Oct 31 01:15:05.646583 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:05.646504    4284 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0cb7e4f8dc40d7a3c46db72e795056052bfe8ad45080151dd5ad4e3b69ba4aa1"} err="failed to get container status \"0cb7e4f8dc40d7a3c46db72e795056052bfe8ad45080151dd5ad4e3b69ba4aa1\": rpc error: code = NotFound desc = could not find container \"0cb7e4f8dc40d7a3c46db72e795056052bfe8ad45080151dd5ad4e3b69ba4aa1\": container with ID starting with 0cb7e4f8dc40d7a3c46db72e795056052bfe8ad45080151dd5ad4e3b69ba4aa1 not found: ID does not exist"
Oct 31 01:15:05.646583 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:05.646529    4284 scope.go:122] "RemoveContainer" containerID="35067cce7b522d2302a926596827c13d7fb4d828f291d22318b7fbcccaa5c162"
Oct 31 01:15:05.656804 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:05.656785    4284 scope.go:122] "RemoveContainer" containerID="35067cce7b522d2302a926596827c13d7fb4d828f291d22318b7fbcccaa5c162"
Oct 31 01:15:05.657145 ip-10-0-132-173 kubenswrapper[4284]: E1031 01:15:05.657126    4284 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"35067cce7b522d2302a926596827c13d7fb4d828f291d22318b7fbcccaa5c162\": container with ID starting with 35067cce7b522d2302a926596827c13d7fb4d828f291d22318b7fbcccaa5c162 not found: ID does not exist" containerID="35067cce7b522d2302a926596827c13d7fb4d828f291d22318b7fbcccaa5c162"
Oct 31 01:15:05.657251 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:05.657155    4284 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"35067cce7b522d2302a926596827c13d7fb4d828f291d22318b7fbcccaa5c162"} err="failed to get container status \"35067cce7b522d2302a926596827c13d7fb4d828f291d22318b7fbcccaa5c162\": rpc error: code = NotFound desc = could not find container \"35067cce7b522d2302a926596827c13d7fb4d828f291d22318b7fbcccaa5c162\": container with ID starting with 35067cce7b522d2302a926596827c13d7fb4d828f291d22318b7fbcccaa5c162 not found: ID does not exist"
Oct 31 01:15:05.695311 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:05.695270    4284 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bgfdb\" (UniqueName: \"kubernetes.io/projected/11338a78-9c55-4df1-8562-360ce3c79c04-kube-api-access-bgfdb\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:15:05.695311 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:05.695304    4284 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/11338a78-9c55-4df1-8562-360ce3c79c04-istio-envoy\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:15:05.695311 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:05.695315    4284 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/11338a78-9c55-4df1-8562-360ce3c79c04-workload-socket\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:15:05.695311 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:05.695323    4284 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/11338a78-9c55-4df1-8562-360ce3c79c04-istio-podinfo\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:15:05.695605 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:05.695332    4284 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/11338a78-9c55-4df1-8562-360ce3c79c04-istio-token\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:15:05.695605 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:05.695340    4284 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/11338a78-9c55-4df1-8562-360ce3c79c04-istiod-ca-cert\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:15:05.695605 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:05.695348    4284 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/11338a78-9c55-4df1-8562-360ce3c79c04-istio-data\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:15:05.794205 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:05.794167    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-b63c24-openshift-default-d8c64855f-wfrh9"]
Oct 31 01:15:05.798449 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:05.798424    4284 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w1-b63c24-openshift-default-d8c64855f-wfrh9"]
Oct 31 01:15:05.903830 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:05.903797    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-rr5k6"]
Oct 31 01:15:05.907773 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:05.907743    4284 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-rr5k6"]
Oct 31 01:15:05.964086 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:05.964053    4284 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="11338a78-9c55-4df1-8562-360ce3c79c04" path="/var/lib/kubelet/pods/11338a78-9c55-4df1-8562-360ce3c79c04/volumes"
Oct 31 01:15:05.964565 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:05.964548    4284 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="27f8cf21-0f47-4397-831a-bf716741dc67" path="/var/lib/kubelet/pods/27f8cf21-0f47-4397-831a-bf716741dc67/volumes"
Oct 31 01:15:06.663010 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:06.662976    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:15:06.854406 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:06.854373    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:15:07.005406 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:07.005376    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-f1d081-openshift-default-56c48f6487-9v42v"]
Oct 31 01:15:07.005666 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:07.005636    4284 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-a-f1d081-openshift-default-56c48f6487-9v42v" podUID="94cb0276-7866-47b5-aa67-1d50bc2250ae" containerName="istio-proxy" containerID="cri-o://ff6967fc6c9d453f8da6d968a270ae6f08dfb8c3cfb5277b9288e2d345d40bfe" gracePeriod=30
Oct 31 01:15:07.014578 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:07.014550    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:15:07.817273 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:07.817232    4284 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-66e263-nkkx8"]
Oct 31 01:15:07.817890 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:07.817872    4284 cpu_manager.go:414] "containerMap: removing container" podUID="11338a78-9c55-4df1-8562-360ce3c79c04" containerName="istio-proxy"
Oct 31 01:15:07.817973 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:07.817890    4284 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="11338a78-9c55-4df1-8562-360ce3c79c04" containerName="istio-proxy"
Oct 31 01:15:07.817973 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:07.817930    4284 cpu_manager.go:414] "containerMap: removing container" podUID="27f8cf21-0f47-4397-831a-bf716741dc67" containerName="istio-proxy"
Oct 31 01:15:07.817973 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:07.817938    4284 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="27f8cf21-0f47-4397-831a-bf716741dc67" containerName="istio-proxy"
Oct 31 01:15:07.818137 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:07.818044    4284 memory_manager.go:365] "RemoveStaleState removing state" podUID="11338a78-9c55-4df1-8562-360ce3c79c04" containerName="istio-proxy"
Oct 31 01:15:07.818137 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:07.818059    4284 memory_manager.go:365] "RemoveStaleState removing state" podUID="27f8cf21-0f47-4397-831a-bf716741dc67" containerName="istio-proxy"
Oct 31 01:15:07.822193 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:07.822173    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-66e263-nkkx8"
Oct 31 01:15:07.839756 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:07.839732    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/02d26a30-b873-49b6-ab71-74986412ddf1-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-66e263-nkkx8\" (UID: \"02d26a30-b873-49b6-ab71-74986412ddf1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-66e263-nkkx8"
Oct 31 01:15:07.839887 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:07.839805    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-crt95\" (UniqueName: \"kubernetes.io/projected/02d26a30-b873-49b6-ab71-74986412ddf1-kube-api-access-crt95\") pod \"maas-api-revoke-keys-e2e-ipp-b-66e263-nkkx8\" (UID: \"02d26a30-b873-49b6-ab71-74986412ddf1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-66e263-nkkx8"
Oct 31 01:15:07.839941 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:07.839921    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/02d26a30-b873-49b6-ab71-74986412ddf1-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-66e263-nkkx8\" (UID: \"02d26a30-b873-49b6-ab71-74986412ddf1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-66e263-nkkx8"
Oct 31 01:15:07.875896 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:07.875861    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-66e263-nkkx8"]
Oct 31 01:15:07.919308 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:07.919260    4284 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 31 01:15:07.940715 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:07.940673    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/02d26a30-b873-49b6-ab71-74986412ddf1-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-66e263-nkkx8\" (UID: \"02d26a30-b873-49b6-ab71-74986412ddf1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-66e263-nkkx8"
Oct 31 01:15:07.940901 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:07.940758    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-crt95\" (UniqueName: \"kubernetes.io/projected/02d26a30-b873-49b6-ab71-74986412ddf1-kube-api-access-crt95\") pod \"maas-api-revoke-keys-e2e-ipp-b-66e263-nkkx8\" (UID: \"02d26a30-b873-49b6-ab71-74986412ddf1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-66e263-nkkx8"
Oct 31 01:15:07.940901 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:07.940819    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/02d26a30-b873-49b6-ab71-74986412ddf1-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-66e263-nkkx8\" (UID: \"02d26a30-b873-49b6-ab71-74986412ddf1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-66e263-nkkx8"
Oct 31 01:15:07.941385 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:07.941365    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/02d26a30-b873-49b6-ab71-74986412ddf1-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-66e263-nkkx8\" (UID: \"02d26a30-b873-49b6-ab71-74986412ddf1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-66e263-nkkx8"
Oct 31 01:15:07.943152 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:07.943130    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/02d26a30-b873-49b6-ab71-74986412ddf1-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-66e263-nkkx8\" (UID: \"02d26a30-b873-49b6-ab71-74986412ddf1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-66e263-nkkx8"
Oct 31 01:15:07.999662 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:07.999626    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-crt95\" (UniqueName: \"kubernetes.io/projected/02d26a30-b873-49b6-ab71-74986412ddf1-kube-api-access-crt95\") pod \"maas-api-revoke-keys-e2e-ipp-b-66e263-nkkx8\" (UID: \"02d26a30-b873-49b6-ab71-74986412ddf1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-66e263-nkkx8"
Oct 31 01:15:08.135193 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:08.135107    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-66e263-nkkx8"
Oct 31 01:15:08.523565 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:08.523527    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-66e263-nkkx8"]
Oct 31 01:15:08.523956 ip-10-0-132-173 kubenswrapper[4284]: W1031 01:15:08.523937    4284 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod02d26a30_b873_49b6_ab71_74986412ddf1.slice/crio-f6112a3c1e44f2f738fa0f01827b0a588ec721f86a96b53dc9e26d07e192b91e WatchSource:0}: Error finding container f6112a3c1e44f2f738fa0f01827b0a588ec721f86a96b53dc9e26d07e192b91e: Status 404 returned error can't find the container with id f6112a3c1e44f2f738fa0f01827b0a588ec721f86a96b53dc9e26d07e192b91e
Oct 31 01:15:08.659258 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:08.659204    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-66e263-nkkx8" event={"ID":"02d26a30-b873-49b6-ab71-74986412ddf1","Type":"ContainerStarted","Data":"f6112a3c1e44f2f738fa0f01827b0a588ec721f86a96b53dc9e26d07e192b91e"}
Oct 31 01:15:09.044249 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:09.044124    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:15:09.666835 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:09.666801    4284 generic.go:356] "Generic (PLEG): container finished" podID="02d26a30-b873-49b6-ab71-74986412ddf1" containerID="580ec896304e3f1366a874654ee80a193ae88148c7d7418ea5509b5801370547" exitCode=0
Oct 31 01:15:09.667012 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:09.666849    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-66e263-nkkx8" event={"ID":"02d26a30-b873-49b6-ab71-74986412ddf1","Type":"ContainerDied","Data":"580ec896304e3f1366a874654ee80a193ae88148c7d7418ea5509b5801370547"}
Oct 31 01:15:10.816349 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:10.816323    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-66e263-nkkx8"
Oct 31 01:15:10.880905 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:10.880875    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/02d26a30-b873-49b6-ab71-74986412ddf1-kube-api-access-crt95\" (UniqueName: \"kubernetes.io/projected/02d26a30-b873-49b6-ab71-74986412ddf1-kube-api-access-crt95\") pod \"02d26a30-b873-49b6-ab71-74986412ddf1\" (UID: \"02d26a30-b873-49b6-ab71-74986412ddf1\") "
Oct 31 01:15:10.881124 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:10.880977    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/02d26a30-b873-49b6-ab71-74986412ddf1-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/02d26a30-b873-49b6-ab71-74986412ddf1-maas-api-openshift-ca\") pod \"02d26a30-b873-49b6-ab71-74986412ddf1\" (UID: \"02d26a30-b873-49b6-ab71-74986412ddf1\") "
Oct 31 01:15:10.881124 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:10.881089    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/02d26a30-b873-49b6-ab71-74986412ddf1-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/02d26a30-b873-49b6-ab71-74986412ddf1-maas-api-xks-ca\") pod \"02d26a30-b873-49b6-ab71-74986412ddf1\" (UID: \"02d26a30-b873-49b6-ab71-74986412ddf1\") "
Oct 31 01:15:10.881319 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:10.881293    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/02d26a30-b873-49b6-ab71-74986412ddf1-maas-api-openshift-ca" pod "02d26a30-b873-49b6-ab71-74986412ddf1" (UID: "02d26a30-b873-49b6-ab71-74986412ddf1"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 01:15:10.881855 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:10.881833    4284 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/02d26a30-b873-49b6-ab71-74986412ddf1-maas-api-openshift-ca\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:15:10.883260 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:10.883198    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/02d26a30-b873-49b6-ab71-74986412ddf1-maas-api-xks-ca" pod "02d26a30-b873-49b6-ab71-74986412ddf1" (UID: "02d26a30-b873-49b6-ab71-74986412ddf1"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 01:15:10.883376 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:10.883335    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/02d26a30-b873-49b6-ab71-74986412ddf1-kube-api-access-crt95" pod "02d26a30-b873-49b6-ab71-74986412ddf1" (UID: "02d26a30-b873-49b6-ab71-74986412ddf1"). InnerVolumeSpecName "kube-api-access-crt95". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:15:10.983859 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:10.983762    4284 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-crt95\" (UniqueName: \"kubernetes.io/projected/02d26a30-b873-49b6-ab71-74986412ddf1-kube-api-access-crt95\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:15:10.983859 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:10.983796    4284 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/02d26a30-b873-49b6-ab71-74986412ddf1-maas-api-xks-ca\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:15:11.680127 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:11.680094    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-66e263-nkkx8"
Oct 31 01:15:11.680353 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:11.680126    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-66e263-nkkx8" event={"ID":"02d26a30-b873-49b6-ab71-74986412ddf1","Type":"ContainerDied","Data":"f6112a3c1e44f2f738fa0f01827b0a588ec721f86a96b53dc9e26d07e192b91e"}
Oct 31 01:15:11.680353 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:11.680189    4284 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f6112a3c1e44f2f738fa0f01827b0a588ec721f86a96b53dc9e26d07e192b91e"
Oct 31 01:15:12.270982 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:12.270923    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-f1d081-openshift-default-56c48f6487-9v42v"
Oct 31 01:15:12.402333 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:12.402293    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/94cb0276-7866-47b5-aa67-1d50bc2250ae-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/94cb0276-7866-47b5-aa67-1d50bc2250ae-istio-podinfo\") pod \"94cb0276-7866-47b5-aa67-1d50bc2250ae\" (UID: \"94cb0276-7866-47b5-aa67-1d50bc2250ae\") "
Oct 31 01:15:12.402507 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:12.402338    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/94cb0276-7866-47b5-aa67-1d50bc2250ae-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/94cb0276-7866-47b5-aa67-1d50bc2250ae-workload-certs\") pod \"94cb0276-7866-47b5-aa67-1d50bc2250ae\" (UID: \"94cb0276-7866-47b5-aa67-1d50bc2250ae\") "
Oct 31 01:15:12.402507 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:12.402395    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/94cb0276-7866-47b5-aa67-1d50bc2250ae-kube-api-access-b7wqx\" (UniqueName: \"kubernetes.io/projected/94cb0276-7866-47b5-aa67-1d50bc2250ae-kube-api-access-b7wqx\") pod \"94cb0276-7866-47b5-aa67-1d50bc2250ae\" (UID: \"94cb0276-7866-47b5-aa67-1d50bc2250ae\") "
Oct 31 01:15:12.402507 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:12.402435    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/94cb0276-7866-47b5-aa67-1d50bc2250ae-istio-token\" (UniqueName: \"kubernetes.io/projected/94cb0276-7866-47b5-aa67-1d50bc2250ae-istio-token\") pod \"94cb0276-7866-47b5-aa67-1d50bc2250ae\" (UID: \"94cb0276-7866-47b5-aa67-1d50bc2250ae\") "
Oct 31 01:15:12.402507 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:12.402469    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/94cb0276-7866-47b5-aa67-1d50bc2250ae-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/94cb0276-7866-47b5-aa67-1d50bc2250ae-istio-data\") pod \"94cb0276-7866-47b5-aa67-1d50bc2250ae\" (UID: \"94cb0276-7866-47b5-aa67-1d50bc2250ae\") "
Oct 31 01:15:12.402507 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:12.402503    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/94cb0276-7866-47b5-aa67-1d50bc2250ae-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/94cb0276-7866-47b5-aa67-1d50bc2250ae-istiod-ca-cert\") pod \"94cb0276-7866-47b5-aa67-1d50bc2250ae\" (UID: \"94cb0276-7866-47b5-aa67-1d50bc2250ae\") "
Oct 31 01:15:12.402806 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:12.402580    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/94cb0276-7866-47b5-aa67-1d50bc2250ae-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/94cb0276-7866-47b5-aa67-1d50bc2250ae-credential-socket\") pod \"94cb0276-7866-47b5-aa67-1d50bc2250ae\" (UID: \"94cb0276-7866-47b5-aa67-1d50bc2250ae\") "
Oct 31 01:15:12.402806 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:12.402695    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/94cb0276-7866-47b5-aa67-1d50bc2250ae-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/94cb0276-7866-47b5-aa67-1d50bc2250ae-istio-envoy\") pod \"94cb0276-7866-47b5-aa67-1d50bc2250ae\" (UID: \"94cb0276-7866-47b5-aa67-1d50bc2250ae\") "
Oct 31 01:15:12.402806 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:12.402710    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/94cb0276-7866-47b5-aa67-1d50bc2250ae-workload-certs" pod "94cb0276-7866-47b5-aa67-1d50bc2250ae" (UID: "94cb0276-7866-47b5-aa67-1d50bc2250ae"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:15:12.402806 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:12.402782    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/94cb0276-7866-47b5-aa67-1d50bc2250ae-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/94cb0276-7866-47b5-aa67-1d50bc2250ae-workload-socket\") pod \"94cb0276-7866-47b5-aa67-1d50bc2250ae\" (UID: \"94cb0276-7866-47b5-aa67-1d50bc2250ae\") "
Oct 31 01:15:12.404054 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:12.403149    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/94cb0276-7866-47b5-aa67-1d50bc2250ae-istiod-ca-cert" pod "94cb0276-7866-47b5-aa67-1d50bc2250ae" (UID: "94cb0276-7866-47b5-aa67-1d50bc2250ae"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 01:15:12.404054 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:12.403253    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/94cb0276-7866-47b5-aa67-1d50bc2250ae-credential-socket" pod "94cb0276-7866-47b5-aa67-1d50bc2250ae" (UID: "94cb0276-7866-47b5-aa67-1d50bc2250ae"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:15:12.404054 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:12.403486    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/94cb0276-7866-47b5-aa67-1d50bc2250ae-workload-socket" pod "94cb0276-7866-47b5-aa67-1d50bc2250ae" (UID: "94cb0276-7866-47b5-aa67-1d50bc2250ae"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:15:12.404054 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:12.403966    4284 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/94cb0276-7866-47b5-aa67-1d50bc2250ae-workload-certs\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:15:12.404054 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:12.403994    4284 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/94cb0276-7866-47b5-aa67-1d50bc2250ae-istiod-ca-cert\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:15:12.404054 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:12.404010    4284 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/94cb0276-7866-47b5-aa67-1d50bc2250ae-credential-socket\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:15:12.404054 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:12.404024    4284 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/94cb0276-7866-47b5-aa67-1d50bc2250ae-workload-socket\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:15:12.404541 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:12.404256    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/94cb0276-7866-47b5-aa67-1d50bc2250ae-istio-data" pod "94cb0276-7866-47b5-aa67-1d50bc2250ae" (UID: "94cb0276-7866-47b5-aa67-1d50bc2250ae"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:15:12.405550 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:12.405528    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/94cb0276-7866-47b5-aa67-1d50bc2250ae-istio-envoy" pod "94cb0276-7866-47b5-aa67-1d50bc2250ae" (UID: "94cb0276-7866-47b5-aa67-1d50bc2250ae"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:15:12.405651 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:12.405558    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/94cb0276-7866-47b5-aa67-1d50bc2250ae-istio-token" pod "94cb0276-7866-47b5-aa67-1d50bc2250ae" (UID: "94cb0276-7866-47b5-aa67-1d50bc2250ae"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:15:12.405651 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:12.405623    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/94cb0276-7866-47b5-aa67-1d50bc2250ae-istio-podinfo" pod "94cb0276-7866-47b5-aa67-1d50bc2250ae" (UID: "94cb0276-7866-47b5-aa67-1d50bc2250ae"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 01:15:12.405767 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:12.405638    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/94cb0276-7866-47b5-aa67-1d50bc2250ae-kube-api-access-b7wqx" pod "94cb0276-7866-47b5-aa67-1d50bc2250ae" (UID: "94cb0276-7866-47b5-aa67-1d50bc2250ae"). InnerVolumeSpecName "kube-api-access-b7wqx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:15:12.505440 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:12.505400    4284 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/94cb0276-7866-47b5-aa67-1d50bc2250ae-istio-podinfo\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:15:12.505440 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:12.505435    4284 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b7wqx\" (UniqueName: \"kubernetes.io/projected/94cb0276-7866-47b5-aa67-1d50bc2250ae-kube-api-access-b7wqx\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:15:12.505440 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:12.505449    4284 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/94cb0276-7866-47b5-aa67-1d50bc2250ae-istio-token\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:15:12.505694 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:12.505462    4284 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/94cb0276-7866-47b5-aa67-1d50bc2250ae-istio-data\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:15:12.505694 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:12.505473    4284 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/94cb0276-7866-47b5-aa67-1d50bc2250ae-istio-envoy\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:15:12.689199 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:12.689164    4284 generic.go:356] "Generic (PLEG): container finished" podID="94cb0276-7866-47b5-aa67-1d50bc2250ae" containerID="ff6967fc6c9d453f8da6d968a270ae6f08dfb8c3cfb5277b9288e2d345d40bfe" exitCode=0
Oct 31 01:15:12.689394 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:12.689239    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-f1d081-openshift-default-56c48f6487-9v42v"
Oct 31 01:15:12.689394 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:12.689250    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-f1d081-openshift-default-56c48f6487-9v42v" event={"ID":"94cb0276-7866-47b5-aa67-1d50bc2250ae","Type":"ContainerDied","Data":"ff6967fc6c9d453f8da6d968a270ae6f08dfb8c3cfb5277b9288e2d345d40bfe"}
Oct 31 01:15:12.689394 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:12.689298    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-f1d081-openshift-default-56c48f6487-9v42v" event={"ID":"94cb0276-7866-47b5-aa67-1d50bc2250ae","Type":"ContainerDied","Data":"c127fd92094bd4e3f14c6127c98ef4ddadfbb72ff721b884e748d7dd1b367f61"}
Oct 31 01:15:12.689394 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:12.689328    4284 scope.go:122] "RemoveContainer" containerID="ff6967fc6c9d453f8da6d968a270ae6f08dfb8c3cfb5277b9288e2d345d40bfe"
Oct 31 01:15:12.704674 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:12.704648    4284 scope.go:122] "RemoveContainer" containerID="b2af715ca7ffe62865aece4e7996799523c55d84f84beebc977f60f086a2333a"
Oct 31 01:15:12.714644 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:12.714589    4284 scope.go:122] "RemoveContainer" containerID="ff6967fc6c9d453f8da6d968a270ae6f08dfb8c3cfb5277b9288e2d345d40bfe"
Oct 31 01:15:12.715107 ip-10-0-132-173 kubenswrapper[4284]: E1031 01:15:12.715070    4284 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ff6967fc6c9d453f8da6d968a270ae6f08dfb8c3cfb5277b9288e2d345d40bfe\": container with ID starting with ff6967fc6c9d453f8da6d968a270ae6f08dfb8c3cfb5277b9288e2d345d40bfe not found: ID does not exist" containerID="ff6967fc6c9d453f8da6d968a270ae6f08dfb8c3cfb5277b9288e2d345d40bfe"
Oct 31 01:15:12.715191 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:12.715122    4284 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ff6967fc6c9d453f8da6d968a270ae6f08dfb8c3cfb5277b9288e2d345d40bfe"} err="failed to get container status \"ff6967fc6c9d453f8da6d968a270ae6f08dfb8c3cfb5277b9288e2d345d40bfe\": rpc error: code = NotFound desc = could not find container \"ff6967fc6c9d453f8da6d968a270ae6f08dfb8c3cfb5277b9288e2d345d40bfe\": container with ID starting with ff6967fc6c9d453f8da6d968a270ae6f08dfb8c3cfb5277b9288e2d345d40bfe not found: ID does not exist"
Oct 31 01:15:12.715191 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:12.715152    4284 scope.go:122] "RemoveContainer" containerID="b2af715ca7ffe62865aece4e7996799523c55d84f84beebc977f60f086a2333a"
Oct 31 01:15:12.715493 ip-10-0-132-173 kubenswrapper[4284]: E1031 01:15:12.715471    4284 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b2af715ca7ffe62865aece4e7996799523c55d84f84beebc977f60f086a2333a\": container with ID starting with b2af715ca7ffe62865aece4e7996799523c55d84f84beebc977f60f086a2333a not found: ID does not exist" containerID="b2af715ca7ffe62865aece4e7996799523c55d84f84beebc977f60f086a2333a"
Oct 31 01:15:12.715546 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:12.715507    4284 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b2af715ca7ffe62865aece4e7996799523c55d84f84beebc977f60f086a2333a"} err="failed to get container status \"b2af715ca7ffe62865aece4e7996799523c55d84f84beebc977f60f086a2333a\": rpc error: code = NotFound desc = could not find container \"b2af715ca7ffe62865aece4e7996799523c55d84f84beebc977f60f086a2333a\": container with ID starting with b2af715ca7ffe62865aece4e7996799523c55d84f84beebc977f60f086a2333a not found: ID does not exist"
Oct 31 01:15:12.848063 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:12.848026    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-f1d081-openshift-default-56c48f6487-9v42v"]
Oct 31 01:15:12.896699 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:12.896665    4284 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-a-f1d081-openshift-default-56c48f6487-9v42v"]
Oct 31 01:15:12.981332 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:12.981259    4284 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-w1-ba34eb/auth-test-model-ba34eb-kserve-55f46b9477-rd7dt"]
Oct 31 01:15:12.981841 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:12.981828    4284 cpu_manager.go:414] "containerMap: removing container" podUID="94cb0276-7866-47b5-aa67-1d50bc2250ae" containerName="istio-proxy"
Oct 31 01:15:12.981880 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:12.981842    4284 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="94cb0276-7866-47b5-aa67-1d50bc2250ae" containerName="istio-proxy"
Oct 31 01:15:12.981880 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:12.981872    4284 cpu_manager.go:414] "containerMap: removing container" podUID="94cb0276-7866-47b5-aa67-1d50bc2250ae" containerName="istio-proxy"
Oct 31 01:15:12.981880 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:12.981877    4284 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="94cb0276-7866-47b5-aa67-1d50bc2250ae" containerName="istio-proxy"
Oct 31 01:15:12.981979 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:12.981894    4284 cpu_manager.go:414] "containerMap: removing container" podUID="02d26a30-b873-49b6-ab71-74986412ddf1" containerName="revoke-keys"
Oct 31 01:15:12.981979 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:12.981899    4284 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="02d26a30-b873-49b6-ab71-74986412ddf1" containerName="revoke-keys"
Oct 31 01:15:12.981979 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:12.981969    4284 memory_manager.go:365] "RemoveStaleState removing state" podUID="94cb0276-7866-47b5-aa67-1d50bc2250ae" containerName="istio-proxy"
Oct 31 01:15:12.982082 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:12.981981    4284 memory_manager.go:365] "RemoveStaleState removing state" podUID="02d26a30-b873-49b6-ab71-74986412ddf1" containerName="revoke-keys"
Oct 31 01:15:12.985571 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:12.985552    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w1-ba34eb/auth-test-model-ba34eb-kserve-55f46b9477-rd7dt"
Oct 31 01:15:13.037413 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:13.037000    4284 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w1-ba34eb\"/\"openshift-service-ca.crt\""
Oct 31 01:15:13.037413 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:13.037198    4284 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w1-ba34eb\"/\"kube-root-ca.crt\""
Oct 31 01:15:13.037673 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:13.037611    4284 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w1-ba34eb\"/\"default-dockercfg-26wts\""
Oct 31 01:15:13.037673 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:13.037641    4284 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w1-ba34eb\"/\"auth-test-model-ba34eb-kserve-self-signed-certs\""
Oct 31 01:15:13.041204 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:13.041180    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w1-ba34eb/auth-test-model-ba34eb-kserve-55f46b9477-rd7dt"]
Oct 31 01:15:13.068245 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:13.067565    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:15:13.116417 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:13.114145    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4192aa2f-a5a8-44fe-9ff6-6bdc48478c39-model-cache\") pod \"auth-test-model-ba34eb-kserve-55f46b9477-rd7dt\" (UID: \"4192aa2f-a5a8-44fe-9ff6-6bdc48478c39\") " pod="ai-tenant-e2e-shared-a-w1-ba34eb/auth-test-model-ba34eb-kserve-55f46b9477-rd7dt"
Oct 31 01:15:13.116417 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:13.114250    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4192aa2f-a5a8-44fe-9ff6-6bdc48478c39-dshm\") pod \"auth-test-model-ba34eb-kserve-55f46b9477-rd7dt\" (UID: \"4192aa2f-a5a8-44fe-9ff6-6bdc48478c39\") " pod="ai-tenant-e2e-shared-a-w1-ba34eb/auth-test-model-ba34eb-kserve-55f46b9477-rd7dt"
Oct 31 01:15:13.116417 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:13.114338    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4192aa2f-a5a8-44fe-9ff6-6bdc48478c39-tmp-dir\") pod \"auth-test-model-ba34eb-kserve-55f46b9477-rd7dt\" (UID: \"4192aa2f-a5a8-44fe-9ff6-6bdc48478c39\") " pod="ai-tenant-e2e-shared-a-w1-ba34eb/auth-test-model-ba34eb-kserve-55f46b9477-rd7dt"
Oct 31 01:15:13.116417 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:13.114471    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4192aa2f-a5a8-44fe-9ff6-6bdc48478c39-tls-certs\") pod \"auth-test-model-ba34eb-kserve-55f46b9477-rd7dt\" (UID: \"4192aa2f-a5a8-44fe-9ff6-6bdc48478c39\") " pod="ai-tenant-e2e-shared-a-w1-ba34eb/auth-test-model-ba34eb-kserve-55f46b9477-rd7dt"
Oct 31 01:15:13.116417 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:13.114522    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4192aa2f-a5a8-44fe-9ff6-6bdc48478c39-home\") pod \"auth-test-model-ba34eb-kserve-55f46b9477-rd7dt\" (UID: \"4192aa2f-a5a8-44fe-9ff6-6bdc48478c39\") " pod="ai-tenant-e2e-shared-a-w1-ba34eb/auth-test-model-ba34eb-kserve-55f46b9477-rd7dt"
Oct 31 01:15:13.116417 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:13.114571    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j7f66\" (UniqueName: \"kubernetes.io/projected/4192aa2f-a5a8-44fe-9ff6-6bdc48478c39-kube-api-access-j7f66\") pod \"auth-test-model-ba34eb-kserve-55f46b9477-rd7dt\" (UID: \"4192aa2f-a5a8-44fe-9ff6-6bdc48478c39\") " pod="ai-tenant-e2e-shared-a-w1-ba34eb/auth-test-model-ba34eb-kserve-55f46b9477-rd7dt"
Oct 31 01:15:13.215647 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:13.215606    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4192aa2f-a5a8-44fe-9ff6-6bdc48478c39-tls-certs\") pod \"auth-test-model-ba34eb-kserve-55f46b9477-rd7dt\" (UID: \"4192aa2f-a5a8-44fe-9ff6-6bdc48478c39\") " pod="ai-tenant-e2e-shared-a-w1-ba34eb/auth-test-model-ba34eb-kserve-55f46b9477-rd7dt"
Oct 31 01:15:13.215806 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:13.215668    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4192aa2f-a5a8-44fe-9ff6-6bdc48478c39-home\") pod \"auth-test-model-ba34eb-kserve-55f46b9477-rd7dt\" (UID: \"4192aa2f-a5a8-44fe-9ff6-6bdc48478c39\") " pod="ai-tenant-e2e-shared-a-w1-ba34eb/auth-test-model-ba34eb-kserve-55f46b9477-rd7dt"
Oct 31 01:15:13.215806 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:13.215716    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j7f66\" (UniqueName: \"kubernetes.io/projected/4192aa2f-a5a8-44fe-9ff6-6bdc48478c39-kube-api-access-j7f66\") pod \"auth-test-model-ba34eb-kserve-55f46b9477-rd7dt\" (UID: \"4192aa2f-a5a8-44fe-9ff6-6bdc48478c39\") " pod="ai-tenant-e2e-shared-a-w1-ba34eb/auth-test-model-ba34eb-kserve-55f46b9477-rd7dt"
Oct 31 01:15:13.215806 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:13.215772    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4192aa2f-a5a8-44fe-9ff6-6bdc48478c39-model-cache\") pod \"auth-test-model-ba34eb-kserve-55f46b9477-rd7dt\" (UID: \"4192aa2f-a5a8-44fe-9ff6-6bdc48478c39\") " pod="ai-tenant-e2e-shared-a-w1-ba34eb/auth-test-model-ba34eb-kserve-55f46b9477-rd7dt"
Oct 31 01:15:13.215991 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:13.215814    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4192aa2f-a5a8-44fe-9ff6-6bdc48478c39-dshm\") pod \"auth-test-model-ba34eb-kserve-55f46b9477-rd7dt\" (UID: \"4192aa2f-a5a8-44fe-9ff6-6bdc48478c39\") " pod="ai-tenant-e2e-shared-a-w1-ba34eb/auth-test-model-ba34eb-kserve-55f46b9477-rd7dt"
Oct 31 01:15:13.215991 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:13.215890    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4192aa2f-a5a8-44fe-9ff6-6bdc48478c39-tmp-dir\") pod \"auth-test-model-ba34eb-kserve-55f46b9477-rd7dt\" (UID: \"4192aa2f-a5a8-44fe-9ff6-6bdc48478c39\") " pod="ai-tenant-e2e-shared-a-w1-ba34eb/auth-test-model-ba34eb-kserve-55f46b9477-rd7dt"
Oct 31 01:15:13.216251 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:13.216199    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4192aa2f-a5a8-44fe-9ff6-6bdc48478c39-home\") pod \"auth-test-model-ba34eb-kserve-55f46b9477-rd7dt\" (UID: \"4192aa2f-a5a8-44fe-9ff6-6bdc48478c39\") " pod="ai-tenant-e2e-shared-a-w1-ba34eb/auth-test-model-ba34eb-kserve-55f46b9477-rd7dt"
Oct 31 01:15:13.216347 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:13.216246    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4192aa2f-a5a8-44fe-9ff6-6bdc48478c39-model-cache\") pod \"auth-test-model-ba34eb-kserve-55f46b9477-rd7dt\" (UID: \"4192aa2f-a5a8-44fe-9ff6-6bdc48478c39\") " pod="ai-tenant-e2e-shared-a-w1-ba34eb/auth-test-model-ba34eb-kserve-55f46b9477-rd7dt"
Oct 31 01:15:13.216347 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:13.216272    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4192aa2f-a5a8-44fe-9ff6-6bdc48478c39-tmp-dir\") pod \"auth-test-model-ba34eb-kserve-55f46b9477-rd7dt\" (UID: \"4192aa2f-a5a8-44fe-9ff6-6bdc48478c39\") " pod="ai-tenant-e2e-shared-a-w1-ba34eb/auth-test-model-ba34eb-kserve-55f46b9477-rd7dt"
Oct 31 01:15:13.218280 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:13.218249    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4192aa2f-a5a8-44fe-9ff6-6bdc48478c39-dshm\") pod \"auth-test-model-ba34eb-kserve-55f46b9477-rd7dt\" (UID: \"4192aa2f-a5a8-44fe-9ff6-6bdc48478c39\") " pod="ai-tenant-e2e-shared-a-w1-ba34eb/auth-test-model-ba34eb-kserve-55f46b9477-rd7dt"
Oct 31 01:15:13.218475 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:13.218460    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4192aa2f-a5a8-44fe-9ff6-6bdc48478c39-tls-certs\") pod \"auth-test-model-ba34eb-kserve-55f46b9477-rd7dt\" (UID: \"4192aa2f-a5a8-44fe-9ff6-6bdc48478c39\") " pod="ai-tenant-e2e-shared-a-w1-ba34eb/auth-test-model-ba34eb-kserve-55f46b9477-rd7dt"
Oct 31 01:15:13.275068 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:13.274993    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j7f66\" (UniqueName: \"kubernetes.io/projected/4192aa2f-a5a8-44fe-9ff6-6bdc48478c39-kube-api-access-j7f66\") pod \"auth-test-model-ba34eb-kserve-55f46b9477-rd7dt\" (UID: \"4192aa2f-a5a8-44fe-9ff6-6bdc48478c39\") " pod="ai-tenant-e2e-shared-a-w1-ba34eb/auth-test-model-ba34eb-kserve-55f46b9477-rd7dt"
Oct 31 01:15:13.297246 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:13.297189    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w1-ba34eb/auth-test-model-ba34eb-kserve-55f46b9477-rd7dt"
Oct 31 01:15:13.675046 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:13.675015    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w1-ba34eb/auth-test-model-ba34eb-kserve-55f46b9477-rd7dt"]
Oct 31 01:15:13.675161 ip-10-0-132-173 kubenswrapper[4284]: W1031 01:15:13.675103    4284 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4192aa2f_a5a8_44fe_9ff6_6bdc48478c39.slice/crio-a53df7541d6a3eb1470ab2bf410f43910f178f7edb2ecc30195c2587c31f542a WatchSource:0}: Error finding container a53df7541d6a3eb1470ab2bf410f43910f178f7edb2ecc30195c2587c31f542a: Status 404 returned error can't find the container with id a53df7541d6a3eb1470ab2bf410f43910f178f7edb2ecc30195c2587c31f542a
Oct 31 01:15:13.699545 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:13.699514    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w1-ba34eb/auth-test-model-ba34eb-kserve-55f46b9477-rd7dt" event={"ID":"4192aa2f-a5a8-44fe-9ff6-6bdc48478c39","Type":"ContainerStarted","Data":"a53df7541d6a3eb1470ab2bf410f43910f178f7edb2ecc30195c2587c31f542a"}
Oct 31 01:15:13.967614 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:13.967534    4284 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="94cb0276-7866-47b5-aa67-1d50bc2250ae" path="/var/lib/kubelet/pods/94cb0276-7866-47b5-aa67-1d50bc2250ae/volumes"
Oct 31 01:15:14.185108 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:14.185069    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:15:14.708600 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:14.708562    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w1-ba34eb/auth-test-model-ba34eb-kserve-55f46b9477-rd7dt" event={"ID":"4192aa2f-a5a8-44fe-9ff6-6bdc48478c39","Type":"ContainerStarted","Data":"a33fcc9a8164acef4386d77c5b98358ccdd1f0ff1ec945b0b49be6b00389fe00"}
Oct 31 01:15:14.821680 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:14.821627    4284 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-w1-ba34eb/auth-test-model-ba34eb-kserve-55f46b9477-rd7dt" podStartSLOduration=2.821612938 podStartE2EDuration="2.821612938s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 01:15:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 01:15:14.820200244 +0000 UTC m=+2828.965207159" watchObservedRunningTime="2026-10-31 01:15:14.821612938 +0000 UTC m=+2828.966619854"
Oct 31 01:15:15.297908 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:15.297873    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-w1-ba34eb/auth-test-model-ba34eb-kserve-55f46b9477-rd7dt"
Oct 31 01:15:16.956705 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:16.956629    4284 scope.go:122] "RemoveContainer" containerID="b10b3c880403d71b7663f080e14777168876efe5ece0537e57addcc7d0b494d6"
Oct 31 01:15:17.019442 ip-10-0-132-173 kubenswrapper[4284]: E1031 01:15:17.019408    4284 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fd8sp" containerName="extractor"
Oct 31 01:15:17.728888 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:17.728855    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-f9925a-29890155-hj42v" event={"ID":"abe955f6-a7a7-4001-a592-c9c69b71b025","Type":"ContainerStarted","Data":"436ba11744b7bc77a8ea095ec404c291b25f648e1eb0459940049c3dd17cae58"}
Oct 31 01:15:17.956507 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:17.956479    4284 scope.go:122] "RemoveContainer" containerID="de267f22d67394a7349acd4ce0d82485823243fd243e2737ee1ef58fcec418a4"
Oct 31 01:15:18.737446 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:18.737415    4284 generic.go:356] "Generic (PLEG): container finished" podID="abe955f6-a7a7-4001-a592-c9c69b71b025" containerID="436ba11744b7bc77a8ea095ec404c291b25f648e1eb0459940049c3dd17cae58" exitCode=60
Oct 31 01:15:18.737974 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:18.737487    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-f9925a-29890155-hj42v" event={"ID":"abe955f6-a7a7-4001-a592-c9c69b71b025","Type":"ContainerDied","Data":"436ba11744b7bc77a8ea095ec404c291b25f648e1eb0459940049c3dd17cae58"}
Oct 31 01:15:18.737974 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:18.737561    4284 scope.go:122] "RemoveContainer" containerID="b10b3c880403d71b7663f080e14777168876efe5ece0537e57addcc7d0b494d6"
Oct 31 01:15:18.738062 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:18.737973    4284 scope.go:122] "RemoveContainer" containerID="436ba11744b7bc77a8ea095ec404c291b25f648e1eb0459940049c3dd17cae58"
Oct 31 01:15:18.738352 ip-10-0-132-173 kubenswrapper[4284]: E1031 01:15:18.738325    4284 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-shared-b-w2-f9925a-29890155-hj42v_odh-ai-gateway-infra(abe955f6-a7a7-4001-a592-c9c69b71b025)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-f9925a-29890155-hj42v" podUID="abe955f6-a7a7-4001-a592-c9c69b71b025"
Oct 31 01:15:18.739589 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:18.739568    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w0-7f8596-29890155-lzdg7" event={"ID":"dfdcd040-7c98-4a93-83b3-13532fda03a8","Type":"ContainerStarted","Data":"6a5be7eb445113fc30eb5a739857247aefa33eb4a1370b2c98c448bd90f37cb1"}
Oct 31 01:15:18.860450 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:18.860415    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-f9925a-29890155-hj42v"]
Oct 31 01:15:19.081388 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:19.081293    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:15:19.748540 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:19.748509    4284 generic.go:356] "Generic (PLEG): container finished" podID="dfdcd040-7c98-4a93-83b3-13532fda03a8" containerID="6a5be7eb445113fc30eb5a739857247aefa33eb4a1370b2c98c448bd90f37cb1" exitCode=60
Oct 31 01:15:19.748917 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:19.748547    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w0-7f8596-29890155-lzdg7" event={"ID":"dfdcd040-7c98-4a93-83b3-13532fda03a8","Type":"ContainerDied","Data":"6a5be7eb445113fc30eb5a739857247aefa33eb4a1370b2c98c448bd90f37cb1"}
Oct 31 01:15:19.748917 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:19.748621    4284 scope.go:122] "RemoveContainer" containerID="de267f22d67394a7349acd4ce0d82485823243fd243e2737ee1ef58fcec418a4"
Oct 31 01:15:19.748917 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:19.748906    4284 scope.go:122] "RemoveContainer" containerID="6a5be7eb445113fc30eb5a739857247aefa33eb4a1370b2c98c448bd90f37cb1"
Oct 31 01:15:19.749301 ip-10-0-132-173 kubenswrapper[4284]: E1031 01:15:19.749238    4284 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-shared-b-w0-7f8596-29890155-lzdg7_odh-ai-gateway-infra(dfdcd040-7c98-4a93-83b3-13532fda03a8)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w0-7f8596-29890155-lzdg7" podUID="dfdcd040-7c98-4a93-83b3-13532fda03a8"
Oct 31 01:15:19.914451 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:19.914424    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-f9925a-29890155-hj42v"
Oct 31 01:15:19.969494 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:19.969461    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w0-7f8596-29890155-lzdg7"]
Oct 31 01:15:20.030646 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:20.030563    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/abe955f6-a7a7-4001-a592-c9c69b71b025-kube-api-access-pshdh\" (UniqueName: \"kubernetes.io/projected/abe955f6-a7a7-4001-a592-c9c69b71b025-kube-api-access-pshdh\") pod \"abe955f6-a7a7-4001-a592-c9c69b71b025\" (UID: \"abe955f6-a7a7-4001-a592-c9c69b71b025\") "
Oct 31 01:15:20.030815 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:20.030656    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/abe955f6-a7a7-4001-a592-c9c69b71b025-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/abe955f6-a7a7-4001-a592-c9c69b71b025-maas-api-service-ca\") pod \"abe955f6-a7a7-4001-a592-c9c69b71b025\" (UID: \"abe955f6-a7a7-4001-a592-c9c69b71b025\") "
Oct 31 01:15:20.031034 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:20.031010    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/abe955f6-a7a7-4001-a592-c9c69b71b025-maas-api-service-ca" pod "abe955f6-a7a7-4001-a592-c9c69b71b025" (UID: "abe955f6-a7a7-4001-a592-c9c69b71b025"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 01:15:20.032759 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:20.032740    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/abe955f6-a7a7-4001-a592-c9c69b71b025-kube-api-access-pshdh" pod "abe955f6-a7a7-4001-a592-c9c69b71b025" (UID: "abe955f6-a7a7-4001-a592-c9c69b71b025"). InnerVolumeSpecName "kube-api-access-pshdh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:15:20.132775 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:20.132739    4284 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/abe955f6-a7a7-4001-a592-c9c69b71b025-maas-api-service-ca\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:15:20.132775 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:20.132775    4284 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pshdh\" (UniqueName: \"kubernetes.io/projected/abe955f6-a7a7-4001-a592-c9c69b71b025-kube-api-access-pshdh\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:15:20.374361 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:20.374271    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:15:20.774828 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:20.774351    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-f9925a-29890155-hj42v" event={"ID":"abe955f6-a7a7-4001-a592-c9c69b71b025","Type":"ContainerDied","Data":"fa07a88a67867b6fbbef7d339463506b033c938d78babb54e9ea86a6d4ab5a73"}
Oct 31 01:15:20.774828 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:20.774382    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-f9925a-29890155-hj42v"
Oct 31 01:15:20.774828 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:20.774467    4284 scope.go:122] "RemoveContainer" containerID="436ba11744b7bc77a8ea095ec404c291b25f648e1eb0459940049c3dd17cae58"
Oct 31 01:15:20.933516 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:20.933487    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-f9925a-29890155-hj42v"]
Oct 31 01:15:20.941677 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:20.941654    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w0-7f8596-29890155-lzdg7"
Oct 31 01:15:20.944498 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:20.944471    4284 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-f9925a-29890155-hj42v"]
Oct 31 01:15:21.048562 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:21.048471    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/dfdcd040-7c98-4a93-83b3-13532fda03a8-kube-api-access-lxrw2\" (UniqueName: \"kubernetes.io/projected/dfdcd040-7c98-4a93-83b3-13532fda03a8-kube-api-access-lxrw2\") pod \"dfdcd040-7c98-4a93-83b3-13532fda03a8\" (UID: \"dfdcd040-7c98-4a93-83b3-13532fda03a8\") "
Oct 31 01:15:21.048724 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:21.048613    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/dfdcd040-7c98-4a93-83b3-13532fda03a8-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dfdcd040-7c98-4a93-83b3-13532fda03a8-maas-api-service-ca\") pod \"dfdcd040-7c98-4a93-83b3-13532fda03a8\" (UID: \"dfdcd040-7c98-4a93-83b3-13532fda03a8\") "
Oct 31 01:15:21.048950 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:21.048928    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dfdcd040-7c98-4a93-83b3-13532fda03a8-maas-api-service-ca" pod "dfdcd040-7c98-4a93-83b3-13532fda03a8" (UID: "dfdcd040-7c98-4a93-83b3-13532fda03a8"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 01:15:21.049104 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:21.049086    4284 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dfdcd040-7c98-4a93-83b3-13532fda03a8-maas-api-service-ca\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:15:21.050908 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:21.050885    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dfdcd040-7c98-4a93-83b3-13532fda03a8-kube-api-access-lxrw2" pod "dfdcd040-7c98-4a93-83b3-13532fda03a8" (UID: "dfdcd040-7c98-4a93-83b3-13532fda03a8"). InnerVolumeSpecName "kube-api-access-lxrw2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:15:21.150555 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:21.150522    4284 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lxrw2\" (UniqueName: \"kubernetes.io/projected/dfdcd040-7c98-4a93-83b3-13532fda03a8-kube-api-access-lxrw2\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:15:21.232872 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:21.232840    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-4jv2z"]
Oct 31 01:15:21.233067 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:21.233038    4284 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-4jv2z" podUID="205f3d84-b321-479b-8931-7584aaf21b1a" containerName="istio-proxy" containerID="cri-o://cfdefd97e1de84f7778a864c751225e5c57abd9ba93244648c72e318acf28a88" gracePeriod=30
Oct 31 01:15:21.235922 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:21.235880    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-7c849"]
Oct 31 01:15:21.236129 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:21.236104    4284 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-7c849" podUID="28346ae6-eb40-4e6f-a0b2-71fe57c02119" containerName="istio-proxy" containerID="cri-o://7ae25bd0ad3d46d71424cce5798ae47bda06e3a920c80ba595dd7a6e547bf70b" gracePeriod=30
Oct 31 01:15:21.798755 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:21.797785    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w0-7f8596-29890155-lzdg7" event={"ID":"dfdcd040-7c98-4a93-83b3-13532fda03a8","Type":"ContainerDied","Data":"a65975fe921d1867220716acad5f9dddf9ac89e2fb9e2518b0eba843446425e7"}
Oct 31 01:15:21.798755 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:21.797856    4284 scope.go:122] "RemoveContainer" containerID="6a5be7eb445113fc30eb5a739857247aefa33eb4a1370b2c98c448bd90f37cb1"
Oct 31 01:15:21.798755 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:21.797967    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w0-7f8596-29890155-lzdg7"
Oct 31 01:15:21.925992 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:21.925953    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:15:21.984588 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:21.984553    4284 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="abe955f6-a7a7-4001-a592-c9c69b71b025" path="/var/lib/kubelet/pods/abe955f6-a7a7-4001-a592-c9c69b71b025/volumes"
Oct 31 01:15:21.985337 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:21.985313    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w0-7f8596-29890155-lzdg7"]
Oct 31 01:15:21.985545 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:21.985529    4284 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w0-7f8596-29890155-lzdg7"]
Oct 31 01:15:23.297890 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:23.297855    4284 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-w1-ba34eb/auth-test-model-ba34eb-kserve-55f46b9477-rd7dt"
Oct 31 01:15:23.312246 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:23.312201    4284 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-w1-ba34eb/auth-test-model-ba34eb-kserve-55f46b9477-rd7dt"
Oct 31 01:15:23.828497 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:23.828463    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-w1-ba34eb/auth-test-model-ba34eb-kserve-55f46b9477-rd7dt"
Oct 31 01:15:23.963791 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:23.963748    4284 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="dfdcd040-7c98-4a93-83b3-13532fda03a8" path="/var/lib/kubelet/pods/dfdcd040-7c98-4a93-83b3-13532fda03a8/volumes"
Oct 31 01:15:24.270690 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:24.270656    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:15:26.385858 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:26.385800    4284 patch_prober.go:29] interesting pod/e2e-worker-w4-ef2572-openshift-default-7649dd87-7c849 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.150:15021/healthz/ready\": dial tcp 10.134.0.150:15021: connect: connection refused" start-of-body=
Oct 31 01:15:26.386275 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:26.385891    4284 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-7c849" podUID="28346ae6-eb40-4e6f-a0b2-71fe57c02119" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.150:15021/healthz/ready\": dial tcp 10.134.0.150:15021: connect: connection refused"
Oct 31 01:15:26.523723 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:26.523699    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-4jv2z"
Oct 31 01:15:26.545944 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:26.545920    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-7c849"
Oct 31 01:15:26.629190 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:26.629156    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/205f3d84-b321-479b-8931-7584aaf21b1a-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/205f3d84-b321-479b-8931-7584aaf21b1a-istiod-ca-cert\") pod \"205f3d84-b321-479b-8931-7584aaf21b1a\" (UID: \"205f3d84-b321-479b-8931-7584aaf21b1a\") "
Oct 31 01:15:26.629394 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:26.629240    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/28346ae6-eb40-4e6f-a0b2-71fe57c02119-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/28346ae6-eb40-4e6f-a0b2-71fe57c02119-workload-certs\") pod \"28346ae6-eb40-4e6f-a0b2-71fe57c02119\" (UID: \"28346ae6-eb40-4e6f-a0b2-71fe57c02119\") "
Oct 31 01:15:26.629394 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:26.629323    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/205f3d84-b321-479b-8931-7584aaf21b1a-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/205f3d84-b321-479b-8931-7584aaf21b1a-credential-socket\") pod \"205f3d84-b321-479b-8931-7584aaf21b1a\" (UID: \"205f3d84-b321-479b-8931-7584aaf21b1a\") "
Oct 31 01:15:26.629394 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:26.629374    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/205f3d84-b321-479b-8931-7584aaf21b1a-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/205f3d84-b321-479b-8931-7584aaf21b1a-workload-socket\") pod \"205f3d84-b321-479b-8931-7584aaf21b1a\" (UID: \"205f3d84-b321-479b-8931-7584aaf21b1a\") "
Oct 31 01:15:26.629538 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:26.629436    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/28346ae6-eb40-4e6f-a0b2-71fe57c02119-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/28346ae6-eb40-4e6f-a0b2-71fe57c02119-workload-socket\") pod \"28346ae6-eb40-4e6f-a0b2-71fe57c02119\" (UID: \"28346ae6-eb40-4e6f-a0b2-71fe57c02119\") "
Oct 31 01:15:26.629538 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:26.629487    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/28346ae6-eb40-4e6f-a0b2-71fe57c02119-kube-api-access-cczxw\" (UniqueName: \"kubernetes.io/projected/28346ae6-eb40-4e6f-a0b2-71fe57c02119-kube-api-access-cczxw\") pod \"28346ae6-eb40-4e6f-a0b2-71fe57c02119\" (UID: \"28346ae6-eb40-4e6f-a0b2-71fe57c02119\") "
Oct 31 01:15:26.629538 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:26.629525    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/28346ae6-eb40-4e6f-a0b2-71fe57c02119-istio-token\" (UniqueName: \"kubernetes.io/projected/28346ae6-eb40-4e6f-a0b2-71fe57c02119-istio-token\") pod \"28346ae6-eb40-4e6f-a0b2-71fe57c02119\" (UID: \"28346ae6-eb40-4e6f-a0b2-71fe57c02119\") "
Oct 31 01:15:26.629680 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:26.629536    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/28346ae6-eb40-4e6f-a0b2-71fe57c02119-workload-certs" pod "28346ae6-eb40-4e6f-a0b2-71fe57c02119" (UID: "28346ae6-eb40-4e6f-a0b2-71fe57c02119"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:15:26.629680 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:26.629554    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/28346ae6-eb40-4e6f-a0b2-71fe57c02119-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/28346ae6-eb40-4e6f-a0b2-71fe57c02119-istio-data\") pod \"28346ae6-eb40-4e6f-a0b2-71fe57c02119\" (UID: \"28346ae6-eb40-4e6f-a0b2-71fe57c02119\") "
Oct 31 01:15:26.629680 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:26.629654    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/205f3d84-b321-479b-8931-7584aaf21b1a-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/205f3d84-b321-479b-8931-7584aaf21b1a-workload-certs\") pod \"205f3d84-b321-479b-8931-7584aaf21b1a\" (UID: \"205f3d84-b321-479b-8931-7584aaf21b1a\") "
Oct 31 01:15:26.629680 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:26.629662    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/205f3d84-b321-479b-8931-7584aaf21b1a-credential-socket" pod "205f3d84-b321-479b-8931-7584aaf21b1a" (UID: "205f3d84-b321-479b-8931-7584aaf21b1a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:15:26.629952 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:26.629670    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/205f3d84-b321-479b-8931-7584aaf21b1a-istiod-ca-cert" pod "205f3d84-b321-479b-8931-7584aaf21b1a" (UID: "205f3d84-b321-479b-8931-7584aaf21b1a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 01:15:26.629952 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:26.629729    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/28346ae6-eb40-4e6f-a0b2-71fe57c02119-workload-socket" pod "28346ae6-eb40-4e6f-a0b2-71fe57c02119" (UID: "28346ae6-eb40-4e6f-a0b2-71fe57c02119"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:15:26.629952 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:26.629744    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/28346ae6-eb40-4e6f-a0b2-71fe57c02119-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/28346ae6-eb40-4e6f-a0b2-71fe57c02119-credential-socket\") pod \"28346ae6-eb40-4e6f-a0b2-71fe57c02119\" (UID: \"28346ae6-eb40-4e6f-a0b2-71fe57c02119\") "
Oct 31 01:15:26.629952 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:26.629781    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/205f3d84-b321-479b-8931-7584aaf21b1a-workload-socket" pod "205f3d84-b321-479b-8931-7584aaf21b1a" (UID: "205f3d84-b321-479b-8931-7584aaf21b1a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:15:26.629952 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:26.629823    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/205f3d84-b321-479b-8931-7584aaf21b1a-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/205f3d84-b321-479b-8931-7584aaf21b1a-istio-data\") pod \"205f3d84-b321-479b-8931-7584aaf21b1a\" (UID: \"205f3d84-b321-479b-8931-7584aaf21b1a\") "
Oct 31 01:15:26.629952 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:26.629877    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/28346ae6-eb40-4e6f-a0b2-71fe57c02119-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/28346ae6-eb40-4e6f-a0b2-71fe57c02119-istiod-ca-cert\") pod \"28346ae6-eb40-4e6f-a0b2-71fe57c02119\" (UID: \"28346ae6-eb40-4e6f-a0b2-71fe57c02119\") "
Oct 31 01:15:26.629952 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:26.629910    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/28346ae6-eb40-4e6f-a0b2-71fe57c02119-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/28346ae6-eb40-4e6f-a0b2-71fe57c02119-istio-podinfo\") pod \"28346ae6-eb40-4e6f-a0b2-71fe57c02119\" (UID: \"28346ae6-eb40-4e6f-a0b2-71fe57c02119\") "
Oct 31 01:15:26.629952 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:26.629913    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/28346ae6-eb40-4e6f-a0b2-71fe57c02119-istio-data" pod "28346ae6-eb40-4e6f-a0b2-71fe57c02119" (UID: "28346ae6-eb40-4e6f-a0b2-71fe57c02119"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:15:26.629952 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:26.629935    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/205f3d84-b321-479b-8931-7584aaf21b1a-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/205f3d84-b321-479b-8931-7584aaf21b1a-istio-podinfo\") pod \"205f3d84-b321-479b-8931-7584aaf21b1a\" (UID: \"205f3d84-b321-479b-8931-7584aaf21b1a\") "
Oct 31 01:15:26.629952 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:26.629935    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/205f3d84-b321-479b-8931-7584aaf21b1a-workload-certs" pod "205f3d84-b321-479b-8931-7584aaf21b1a" (UID: "205f3d84-b321-479b-8931-7584aaf21b1a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:15:26.630520 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:26.630074    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/205f3d84-b321-479b-8931-7584aaf21b1a-kube-api-access-n9kws\" (UniqueName: \"kubernetes.io/projected/205f3d84-b321-479b-8931-7584aaf21b1a-kube-api-access-n9kws\") pod \"205f3d84-b321-479b-8931-7584aaf21b1a\" (UID: \"205f3d84-b321-479b-8931-7584aaf21b1a\") "
Oct 31 01:15:26.630520 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:26.630075    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/205f3d84-b321-479b-8931-7584aaf21b1a-istio-data" pod "205f3d84-b321-479b-8931-7584aaf21b1a" (UID: "205f3d84-b321-479b-8931-7584aaf21b1a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:15:26.630520 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:26.630120    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/205f3d84-b321-479b-8931-7584aaf21b1a-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/205f3d84-b321-479b-8931-7584aaf21b1a-istio-envoy\") pod \"205f3d84-b321-479b-8931-7584aaf21b1a\" (UID: \"205f3d84-b321-479b-8931-7584aaf21b1a\") "
Oct 31 01:15:26.630520 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:26.630155    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/28346ae6-eb40-4e6f-a0b2-71fe57c02119-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/28346ae6-eb40-4e6f-a0b2-71fe57c02119-istio-envoy\") pod \"28346ae6-eb40-4e6f-a0b2-71fe57c02119\" (UID: \"28346ae6-eb40-4e6f-a0b2-71fe57c02119\") "
Oct 31 01:15:26.630520 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:26.630183    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/205f3d84-b321-479b-8931-7584aaf21b1a-istio-token\" (UniqueName: \"kubernetes.io/projected/205f3d84-b321-479b-8931-7584aaf21b1a-istio-token\") pod \"205f3d84-b321-479b-8931-7584aaf21b1a\" (UID: \"205f3d84-b321-479b-8931-7584aaf21b1a\") "
Oct 31 01:15:26.630520 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:26.630245    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/28346ae6-eb40-4e6f-a0b2-71fe57c02119-istiod-ca-cert" pod "28346ae6-eb40-4e6f-a0b2-71fe57c02119" (UID: "28346ae6-eb40-4e6f-a0b2-71fe57c02119"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 01:15:26.631042 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:26.630986    4284 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/28346ae6-eb40-4e6f-a0b2-71fe57c02119-istio-data\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:15:26.631042 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:26.631017    4284 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/205f3d84-b321-479b-8931-7584aaf21b1a-workload-certs\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:15:26.631042 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:26.631032    4284 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/205f3d84-b321-479b-8931-7584aaf21b1a-istio-data\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:15:26.631042 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:26.631048    4284 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/28346ae6-eb40-4e6f-a0b2-71fe57c02119-istiod-ca-cert\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:15:26.631042 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:26.631062    4284 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/205f3d84-b321-479b-8931-7584aaf21b1a-istiod-ca-cert\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:15:26.631042 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:26.631076    4284 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/28346ae6-eb40-4e6f-a0b2-71fe57c02119-workload-certs\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:15:26.631460 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:26.631089    4284 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/205f3d84-b321-479b-8931-7584aaf21b1a-credential-socket\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:15:26.631460 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:26.631102    4284 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/205f3d84-b321-479b-8931-7584aaf21b1a-workload-socket\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:15:26.631460 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:26.631119    4284 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/28346ae6-eb40-4e6f-a0b2-71fe57c02119-workload-socket\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:15:26.631460 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:26.631180    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/28346ae6-eb40-4e6f-a0b2-71fe57c02119-credential-socket" pod "28346ae6-eb40-4e6f-a0b2-71fe57c02119" (UID: "28346ae6-eb40-4e6f-a0b2-71fe57c02119"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:15:26.632876 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:26.632847    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/28346ae6-eb40-4e6f-a0b2-71fe57c02119-istio-podinfo" pod "28346ae6-eb40-4e6f-a0b2-71fe57c02119" (UID: "28346ae6-eb40-4e6f-a0b2-71fe57c02119"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 01:15:26.632992 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:26.632882    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/205f3d84-b321-479b-8931-7584aaf21b1a-istio-envoy" pod "205f3d84-b321-479b-8931-7584aaf21b1a" (UID: "205f3d84-b321-479b-8931-7584aaf21b1a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:15:26.632992 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:26.632925    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/28346ae6-eb40-4e6f-a0b2-71fe57c02119-kube-api-access-cczxw" pod "28346ae6-eb40-4e6f-a0b2-71fe57c02119" (UID: "28346ae6-eb40-4e6f-a0b2-71fe57c02119"). InnerVolumeSpecName "kube-api-access-cczxw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:15:26.632992 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:26.632978    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/205f3d84-b321-479b-8931-7584aaf21b1a-istio-podinfo" pod "205f3d84-b321-479b-8931-7584aaf21b1a" (UID: "205f3d84-b321-479b-8931-7584aaf21b1a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 01:15:26.633092 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:26.633010    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/28346ae6-eb40-4e6f-a0b2-71fe57c02119-istio-token" pod "28346ae6-eb40-4e6f-a0b2-71fe57c02119" (UID: "28346ae6-eb40-4e6f-a0b2-71fe57c02119"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:15:26.633262 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:26.633238    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/28346ae6-eb40-4e6f-a0b2-71fe57c02119-istio-envoy" pod "28346ae6-eb40-4e6f-a0b2-71fe57c02119" (UID: "28346ae6-eb40-4e6f-a0b2-71fe57c02119"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:15:26.633409 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:26.633394    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/205f3d84-b321-479b-8931-7584aaf21b1a-istio-token" pod "205f3d84-b321-479b-8931-7584aaf21b1a" (UID: "205f3d84-b321-479b-8931-7584aaf21b1a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:15:26.633765 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:26.633751    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/205f3d84-b321-479b-8931-7584aaf21b1a-kube-api-access-n9kws" pod "205f3d84-b321-479b-8931-7584aaf21b1a" (UID: "205f3d84-b321-479b-8931-7584aaf21b1a"). InnerVolumeSpecName "kube-api-access-n9kws". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:15:26.732118 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:26.732079    4284 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cczxw\" (UniqueName: \"kubernetes.io/projected/28346ae6-eb40-4e6f-a0b2-71fe57c02119-kube-api-access-cczxw\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:15:26.732118 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:26.732113    4284 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/28346ae6-eb40-4e6f-a0b2-71fe57c02119-istio-token\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:15:26.732118 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:26.732124    4284 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/28346ae6-eb40-4e6f-a0b2-71fe57c02119-credential-socket\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:15:26.732428 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:26.732132    4284 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/28346ae6-eb40-4e6f-a0b2-71fe57c02119-istio-podinfo\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:15:26.732428 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:26.732144    4284 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/205f3d84-b321-479b-8931-7584aaf21b1a-istio-podinfo\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:15:26.732428 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:26.732152    4284 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n9kws\" (UniqueName: \"kubernetes.io/projected/205f3d84-b321-479b-8931-7584aaf21b1a-kube-api-access-n9kws\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:15:26.732428 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:26.732161    4284 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/205f3d84-b321-479b-8931-7584aaf21b1a-istio-envoy\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:15:26.732428 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:26.732169    4284 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/28346ae6-eb40-4e6f-a0b2-71fe57c02119-istio-envoy\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:15:26.732428 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:26.732176    4284 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/205f3d84-b321-479b-8931-7584aaf21b1a-istio-token\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:15:26.837367 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:26.837272    4284 generic.go:356] "Generic (PLEG): container finished" podID="28346ae6-eb40-4e6f-a0b2-71fe57c02119" containerID="7ae25bd0ad3d46d71424cce5798ae47bda06e3a920c80ba595dd7a6e547bf70b" exitCode=0
Oct 31 01:15:26.837367 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:26.837341    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-7c849"
Oct 31 01:15:26.837590 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:26.837360    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-7c849" event={"ID":"28346ae6-eb40-4e6f-a0b2-71fe57c02119","Type":"ContainerDied","Data":"7ae25bd0ad3d46d71424cce5798ae47bda06e3a920c80ba595dd7a6e547bf70b"}
Oct 31 01:15:26.837590 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:26.837430    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-7c849" event={"ID":"28346ae6-eb40-4e6f-a0b2-71fe57c02119","Type":"ContainerDied","Data":"d95e129d0e4544e645ebf7358ebcdc2beac50b2b278d5f6c6221648e927a95cf"}
Oct 31 01:15:26.837590 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:26.837457    4284 scope.go:122] "RemoveContainer" containerID="7ae25bd0ad3d46d71424cce5798ae47bda06e3a920c80ba595dd7a6e547bf70b"
Oct 31 01:15:26.843570 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:26.843541    4284 generic.go:356] "Generic (PLEG): container finished" podID="205f3d84-b321-479b-8931-7584aaf21b1a" containerID="cfdefd97e1de84f7778a864c751225e5c57abd9ba93244648c72e318acf28a88" exitCode=0
Oct 31 01:15:26.843710 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:26.843597    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-4jv2z" event={"ID":"205f3d84-b321-479b-8931-7584aaf21b1a","Type":"ContainerDied","Data":"cfdefd97e1de84f7778a864c751225e5c57abd9ba93244648c72e318acf28a88"}
Oct 31 01:15:26.843710 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:26.843614    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-4jv2z"
Oct 31 01:15:26.843710 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:26.843647    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-4jv2z" event={"ID":"205f3d84-b321-479b-8931-7584aaf21b1a","Type":"ContainerDied","Data":"dfc1d9e7d3eb9bec88f003af5c061230a971449fb94a6b75caa46dcdf8178ccc"}
Oct 31 01:15:26.848970 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:26.848949    4284 scope.go:122] "RemoveContainer" containerID="7ae25bd0ad3d46d71424cce5798ae47bda06e3a920c80ba595dd7a6e547bf70b"
Oct 31 01:15:26.849295 ip-10-0-132-173 kubenswrapper[4284]: E1031 01:15:26.849275    4284 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7ae25bd0ad3d46d71424cce5798ae47bda06e3a920c80ba595dd7a6e547bf70b\": container with ID starting with 7ae25bd0ad3d46d71424cce5798ae47bda06e3a920c80ba595dd7a6e547bf70b not found: ID does not exist" containerID="7ae25bd0ad3d46d71424cce5798ae47bda06e3a920c80ba595dd7a6e547bf70b"
Oct 31 01:15:26.849374 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:26.849302    4284 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7ae25bd0ad3d46d71424cce5798ae47bda06e3a920c80ba595dd7a6e547bf70b"} err="failed to get container status \"7ae25bd0ad3d46d71424cce5798ae47bda06e3a920c80ba595dd7a6e547bf70b\": rpc error: code = NotFound desc = could not find container \"7ae25bd0ad3d46d71424cce5798ae47bda06e3a920c80ba595dd7a6e547bf70b\": container with ID starting with 7ae25bd0ad3d46d71424cce5798ae47bda06e3a920c80ba595dd7a6e547bf70b not found: ID does not exist"
Oct 31 01:15:26.849374 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:26.849318    4284 scope.go:122] "RemoveContainer" containerID="cfdefd97e1de84f7778a864c751225e5c57abd9ba93244648c72e318acf28a88"
Oct 31 01:15:26.860899 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:26.860875    4284 scope.go:122] "RemoveContainer" containerID="cfdefd97e1de84f7778a864c751225e5c57abd9ba93244648c72e318acf28a88"
Oct 31 01:15:26.861203 ip-10-0-132-173 kubenswrapper[4284]: E1031 01:15:26.861185    4284 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cfdefd97e1de84f7778a864c751225e5c57abd9ba93244648c72e318acf28a88\": container with ID starting with cfdefd97e1de84f7778a864c751225e5c57abd9ba93244648c72e318acf28a88 not found: ID does not exist" containerID="cfdefd97e1de84f7778a864c751225e5c57abd9ba93244648c72e318acf28a88"
Oct 31 01:15:26.861482 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:26.861461    4284 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cfdefd97e1de84f7778a864c751225e5c57abd9ba93244648c72e318acf28a88"} err="failed to get container status \"cfdefd97e1de84f7778a864c751225e5c57abd9ba93244648c72e318acf28a88\": rpc error: code = NotFound desc = could not find container \"cfdefd97e1de84f7778a864c751225e5c57abd9ba93244648c72e318acf28a88\": container with ID starting with cfdefd97e1de84f7778a864c751225e5c57abd9ba93244648c72e318acf28a88 not found: ID does not exist"
Oct 31 01:15:26.999418 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:26.999385    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-7c849"]
Oct 31 01:15:27.003873 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:27.003846    4284 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-7c849"]
Oct 31 01:15:27.109823 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:27.109732    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-4jv2z"]
Oct 31 01:15:27.113372 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:27.113346    4284 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-ef2572-openshift-default-7649dd87-4jv2z"]
Oct 31 01:15:27.964206 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:27.964167    4284 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="205f3d84-b321-479b-8931-7584aaf21b1a" path="/var/lib/kubelet/pods/205f3d84-b321-479b-8931-7584aaf21b1a/volumes"
Oct 31 01:15:27.964779 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:27.964762    4284 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="28346ae6-eb40-4e6f-a0b2-71fe57c02119" path="/var/lib/kubelet/pods/28346ae6-eb40-4e6f-a0b2-71fe57c02119/volumes"
Oct 31 01:15:29.015706 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:29.015638    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:15:33.804894 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:33.804851    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:15:38.513777 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:38.513727    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:15:43.261694 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:43.261659    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:15:47.013970 ip-10-0-132-173 kubenswrapper[4284]: E1031 01:15:47.013923    4284 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fd8sp" containerName="extractor"
Oct 31 01:15:47.467312 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:47.467277    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:15:47.968146 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:47.968108    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:15:48.155011 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:48.154970    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:15:49.003641 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:49.003600    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:15:49.270955 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:49.270857    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:15:49.455530 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:49.455475    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:15:49.542374 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:49.542288    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-66e263-nkkx8"]
Oct 31 01:15:49.589942 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:49.589906    4284 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-66e263-nkkx8"]
Oct 31 01:15:49.962959 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:49.962924    4284 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="02d26a30-b873-49b6-ab71-74986412ddf1" path="/var/lib/kubelet/pods/02d26a30-b873-49b6-ab71-74986412ddf1/volumes"
Oct 31 01:15:49.970122 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:49.970091    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:15:52.718869 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:52.718825    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:15:54.284694 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:54.284658    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:15:56.383346 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:56.383306    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:15:56.583780 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:56.583740    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:15:56.725618 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:56.725581    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:15:59.052728 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:15:59.052697    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:16:03.762897 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:16:03.762856    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:16:04.985106 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:16:04.985070    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:16:08.481893 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:16:08.481853    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:16:08.669513 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:16:08.669476    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:16:09.094666 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:16:09.094620    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:16:09.259238 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:16:09.253765    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:16:13.515867 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:16:13.515817    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:16:13.668441 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:16:13.668408    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:16:13.812046 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:16:13.811958    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:16:16.766057 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:16:16.765995    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:16:16.955287 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:16:16.955246    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:16:17.014986 ip-10-0-132-173 kubenswrapper[4284]: E1031 01:16:17.014952    4284 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fd8sp" containerName="extractor"
Oct 31 01:16:17.245861 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:16:17.245825    4284 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-8ddd2c-69b95b5867-pfjc2"]
Oct 31 01:16:17.246935 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:16:17.246909    4284 cpu_manager.go:414] "containerMap: removing container" podUID="dfdcd040-7c98-4a93-83b3-13532fda03a8" containerName="cleanup"
Oct 31 01:16:17.247094 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:16:17.247080    4284 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="dfdcd040-7c98-4a93-83b3-13532fda03a8" containerName="cleanup"
Oct 31 01:16:17.247192 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:16:17.247181    4284 cpu_manager.go:414] "containerMap: removing container" podUID="28346ae6-eb40-4e6f-a0b2-71fe57c02119" containerName="istio-proxy"
Oct 31 01:16:17.247304 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:16:17.247290    4284 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="28346ae6-eb40-4e6f-a0b2-71fe57c02119" containerName="istio-proxy"
Oct 31 01:16:17.247401 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:16:17.247391    4284 cpu_manager.go:414] "containerMap: removing container" podUID="abe955f6-a7a7-4001-a592-c9c69b71b025" containerName="cleanup"
Oct 31 01:16:17.247485 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:16:17.247474    4284 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="abe955f6-a7a7-4001-a592-c9c69b71b025" containerName="cleanup"
Oct 31 01:16:17.247589 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:16:17.247578    4284 cpu_manager.go:414] "containerMap: removing container" podUID="dfdcd040-7c98-4a93-83b3-13532fda03a8" containerName="cleanup"
Oct 31 01:16:17.247684 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:16:17.247673    4284 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="dfdcd040-7c98-4a93-83b3-13532fda03a8" containerName="cleanup"
Oct 31 01:16:17.247776 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:16:17.247765    4284 cpu_manager.go:414] "containerMap: removing container" podUID="dfdcd040-7c98-4a93-83b3-13532fda03a8" containerName="cleanup"
Oct 31 01:16:17.247867 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:16:17.247855    4284 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="dfdcd040-7c98-4a93-83b3-13532fda03a8" containerName="cleanup"
Oct 31 01:16:17.247993 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:16:17.247980    4284 cpu_manager.go:414] "containerMap: removing container" podUID="abe955f6-a7a7-4001-a592-c9c69b71b025" containerName="cleanup"
Oct 31 01:16:17.248095 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:16:17.248076    4284 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="abe955f6-a7a7-4001-a592-c9c69b71b025" containerName="cleanup"
Oct 31 01:16:17.248225 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:16:17.248196    4284 cpu_manager.go:414] "containerMap: removing container" podUID="205f3d84-b321-479b-8931-7584aaf21b1a" containerName="istio-proxy"
Oct 31 01:16:17.248339 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:16:17.248325    4284 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="205f3d84-b321-479b-8931-7584aaf21b1a" containerName="istio-proxy"
Oct 31 01:16:17.248606 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:16:17.248587    4284 memory_manager.go:365] "RemoveStaleState removing state" podUID="abe955f6-a7a7-4001-a592-c9c69b71b025" containerName="cleanup"
Oct 31 01:16:17.248723 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:16:17.248710    4284 memory_manager.go:365] "RemoveStaleState removing state" podUID="dfdcd040-7c98-4a93-83b3-13532fda03a8" containerName="cleanup"
Oct 31 01:16:17.248840 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:16:17.248820    4284 memory_manager.go:365] "RemoveStaleState removing state" podUID="dfdcd040-7c98-4a93-83b3-13532fda03a8" containerName="cleanup"
Oct 31 01:16:17.248956 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:16:17.248938    4284 memory_manager.go:365] "RemoveStaleState removing state" podUID="abe955f6-a7a7-4001-a592-c9c69b71b025" containerName="cleanup"
Oct 31 01:16:17.249053 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:16:17.249042    4284 memory_manager.go:365] "RemoveStaleState removing state" podUID="abe955f6-a7a7-4001-a592-c9c69b71b025" containerName="cleanup"
Oct 31 01:16:17.249165 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:16:17.249150    4284 memory_manager.go:365] "RemoveStaleState removing state" podUID="94cb0276-7866-47b5-aa67-1d50bc2250ae" containerName="istio-proxy"
Oct 31 01:16:17.261361 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:16:17.261338    4284 memory_manager.go:365] "RemoveStaleState removing state" podUID="205f3d84-b321-479b-8931-7584aaf21b1a" containerName="istio-proxy"
Oct 31 01:16:17.261515 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:16:17.261505    4284 memory_manager.go:365] "RemoveStaleState removing state" podUID="28346ae6-eb40-4e6f-a0b2-71fe57c02119" containerName="istio-proxy"
Oct 31 01:16:17.267231 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:16:17.265845    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-explicit-8ddd2c-69b95b5867-pfjc2"
Oct 31 01:16:17.311236 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:16:17.304601    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-8ddd2c-69b95b5867-pfjc2"]
Oct 31 01:16:17.346478 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:16:17.346447    4284 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-explicit-8ddd2c\""
Oct 31 01:16:17.355257 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:16:17.355227    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/10934caf-df36-4907-9138-27d4426c81f1-plugins-config-volume\") pod \"payload-pre-processing-e2e-explicit-8ddd2c-69b95b5867-pfjc2\" (UID: \"10934caf-df36-4907-9138-27d4426c81f1\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-8ddd2c-69b95b5867-pfjc2"
Oct 31 01:16:17.355445 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:16:17.355350    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pw4kf\" (UniqueName: \"kubernetes.io/projected/10934caf-df36-4907-9138-27d4426c81f1-kube-api-access-pw4kf\") pod \"payload-pre-processing-e2e-explicit-8ddd2c-69b95b5867-pfjc2\" (UID: \"10934caf-df36-4907-9138-27d4426c81f1\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-8ddd2c-69b95b5867-pfjc2"
Oct 31 01:16:17.456064 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:16:17.456026    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pw4kf\" (UniqueName: \"kubernetes.io/projected/10934caf-df36-4907-9138-27d4426c81f1-kube-api-access-pw4kf\") pod \"payload-pre-processing-e2e-explicit-8ddd2c-69b95b5867-pfjc2\" (UID: \"10934caf-df36-4907-9138-27d4426c81f1\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-8ddd2c-69b95b5867-pfjc2"
Oct 31 01:16:17.456275 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:16:17.456167    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/10934caf-df36-4907-9138-27d4426c81f1-plugins-config-volume\") pod \"payload-pre-processing-e2e-explicit-8ddd2c-69b95b5867-pfjc2\" (UID: \"10934caf-df36-4907-9138-27d4426c81f1\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-8ddd2c-69b95b5867-pfjc2"
Oct 31 01:16:17.456849 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:16:17.456830    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/10934caf-df36-4907-9138-27d4426c81f1-plugins-config-volume\") pod \"payload-pre-processing-e2e-explicit-8ddd2c-69b95b5867-pfjc2\" (UID: \"10934caf-df36-4907-9138-27d4426c81f1\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-8ddd2c-69b95b5867-pfjc2"
Oct 31 01:16:17.515720 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:16:17.515652    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pw4kf\" (UniqueName: \"kubernetes.io/projected/10934caf-df36-4907-9138-27d4426c81f1-kube-api-access-pw4kf\") pod \"payload-pre-processing-e2e-explicit-8ddd2c-69b95b5867-pfjc2\" (UID: \"10934caf-df36-4907-9138-27d4426c81f1\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-8ddd2c-69b95b5867-pfjc2"
Oct 31 01:16:17.601023 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:16:17.600982    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-explicit-8ddd2c-69b95b5867-pfjc2"
Oct 31 01:16:17.976406 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:16:17.976384    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-8ddd2c-69b95b5867-pfjc2"]
Oct 31 01:16:17.977059 ip-10-0-132-173 kubenswrapper[4284]: W1031 01:16:17.977036    4284 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod10934caf_df36_4907_9138_27d4426c81f1.slice/crio-7854698877d0a6638c8276177807893baca4b741308421d3e1a8c5a7da1951b4 WatchSource:0}: Error finding container 7854698877d0a6638c8276177807893baca4b741308421d3e1a8c5a7da1951b4: Status 404 returned error can't find the container with id 7854698877d0a6638c8276177807893baca4b741308421d3e1a8c5a7da1951b4
Oct 31 01:16:18.152492 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:16:18.152450    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-explicit-8ddd2c-69b95b5867-pfjc2" event={"ID":"10934caf-df36-4907-9138-27d4426c81f1","Type":"ContainerStarted","Data":"518045322ba14ac9e60b1d5268135bc1588aaf2510b26482e4eb36c61efbde4a"}
Oct 31 01:16:18.152669 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:16:18.152500    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-explicit-8ddd2c-69b95b5867-pfjc2" event={"ID":"10934caf-df36-4907-9138-27d4426c81f1","Type":"ContainerStarted","Data":"7854698877d0a6638c8276177807893baca4b741308421d3e1a8c5a7da1951b4"}
Oct 31 01:16:18.152725 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:16:18.152666    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-explicit-8ddd2c-69b95b5867-pfjc2"
Oct 31 01:16:18.266986 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:16:18.266889    4284 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-explicit-8ddd2c-69b95b5867-pfjc2" podStartSLOduration=1.266875755 podStartE2EDuration="1.266875755s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 01:16:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 01:16:18.264414933 +0000 UTC m=+2892.409421864" watchObservedRunningTime="2026-10-31 01:16:18.266875755 +0000 UTC m=+2892.411882697"
Oct 31 01:16:18.792593 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:16:18.792557    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:16:19.647097 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:16:19.647058    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:16:23.485341 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:16:23.485294    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:16:26.434461 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:16:26.434431    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:16:28.283075 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:16:28.283041    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:16:29.172726 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:16:29.172695    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-explicit-8ddd2c-69b95b5867-pfjc2"
Oct 31 01:16:33.093465 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:16:33.093426    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:16:37.809351 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:16:37.809315    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:16:42.512643 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:16:42.512598    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:16:45.501094 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:16:45.501052    4284 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-w1-3be6fa/auth-test-model-3be6fa-kserve-6f5d4f988f-zc2bc"]
Oct 31 01:16:45.501902 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:16:45.501878    4284 cpu_manager.go:414] "containerMap: removing container" podUID="abe955f6-a7a7-4001-a592-c9c69b71b025" containerName="cleanup"
Oct 31 01:16:45.502004 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:16:45.501902    4284 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="abe955f6-a7a7-4001-a592-c9c69b71b025" containerName="cleanup"
Oct 31 01:16:45.502091 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:16:45.502077    4284 memory_manager.go:365] "RemoveStaleState removing state" podUID="dfdcd040-7c98-4a93-83b3-13532fda03a8" containerName="cleanup"
Oct 31 01:16:45.505085 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:16:45.505066    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w1-3be6fa/auth-test-model-3be6fa-kserve-6f5d4f988f-zc2bc"
Oct 31 01:16:45.578365 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:16:45.578331    4284 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w1-3be6fa\"/\"kube-root-ca.crt\""
Oct 31 01:16:45.600891 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:16:45.600856    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:16:45.606140 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:16:45.606098    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xlmz4\" (UniqueName: \"kubernetes.io/projected/cac35176-4262-433c-ba9d-ded7a7c11502-kube-api-access-xlmz4\") pod \"auth-test-model-3be6fa-kserve-6f5d4f988f-zc2bc\" (UID: \"cac35176-4262-433c-ba9d-ded7a7c11502\") " pod="ai-tenant-e2e-shared-b-w1-3be6fa/auth-test-model-3be6fa-kserve-6f5d4f988f-zc2bc"
Oct 31 01:16:45.606288 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:16:45.606272    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cac35176-4262-433c-ba9d-ded7a7c11502-dshm\") pod \"auth-test-model-3be6fa-kserve-6f5d4f988f-zc2bc\" (UID: \"cac35176-4262-433c-ba9d-ded7a7c11502\") " pod="ai-tenant-e2e-shared-b-w1-3be6fa/auth-test-model-3be6fa-kserve-6f5d4f988f-zc2bc"
Oct 31 01:16:45.606334 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:16:45.606300    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cac35176-4262-433c-ba9d-ded7a7c11502-tmp-dir\") pod \"auth-test-model-3be6fa-kserve-6f5d4f988f-zc2bc\" (UID: \"cac35176-4262-433c-ba9d-ded7a7c11502\") " pod="ai-tenant-e2e-shared-b-w1-3be6fa/auth-test-model-3be6fa-kserve-6f5d4f988f-zc2bc"
Oct 31 01:16:45.606420 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:16:45.606399    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cac35176-4262-433c-ba9d-ded7a7c11502-tls-certs\") pod \"auth-test-model-3be6fa-kserve-6f5d4f988f-zc2bc\" (UID: \"cac35176-4262-433c-ba9d-ded7a7c11502\") " pod="ai-tenant-e2e-shared-b-w1-3be6fa/auth-test-model-3be6fa-kserve-6f5d4f988f-zc2bc"
Oct 31 01:16:45.606476 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:16:45.606462    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cac35176-4262-433c-ba9d-ded7a7c11502-home\") pod \"auth-test-model-3be6fa-kserve-6f5d4f988f-zc2bc\" (UID: \"cac35176-4262-433c-ba9d-ded7a7c11502\") " pod="ai-tenant-e2e-shared-b-w1-3be6fa/auth-test-model-3be6fa-kserve-6f5d4f988f-zc2bc"
Oct 31 01:16:45.606604 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:16:45.606585    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cac35176-4262-433c-ba9d-ded7a7c11502-model-cache\") pod \"auth-test-model-3be6fa-kserve-6f5d4f988f-zc2bc\" (UID: \"cac35176-4262-433c-ba9d-ded7a7c11502\") " pod="ai-tenant-e2e-shared-b-w1-3be6fa/auth-test-model-3be6fa-kserve-6f5d4f988f-zc2bc"
Oct 31 01:16:45.608099 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:16:45.608084    4284 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w1-3be6fa\"/\"openshift-service-ca.crt\""
Oct 31 01:16:45.608327 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:16:45.608300    4284 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w1-3be6fa\"/\"default-dockercfg-92brf\""
Oct 31 01:16:45.608327 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:16:45.608298    4284 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w1-3be6fa\"/\"auth-test-model-3be6fa-kserve-self-signed-certs\""
Oct 31 01:16:45.611325 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:16:45.611304    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w1-3be6fa/auth-test-model-3be6fa-kserve-6f5d4f988f-zc2bc"]
Oct 31 01:16:45.707805 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:16:45.707765    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cac35176-4262-433c-ba9d-ded7a7c11502-dshm\") pod \"auth-test-model-3be6fa-kserve-6f5d4f988f-zc2bc\" (UID: \"cac35176-4262-433c-ba9d-ded7a7c11502\") " pod="ai-tenant-e2e-shared-b-w1-3be6fa/auth-test-model-3be6fa-kserve-6f5d4f988f-zc2bc"
Oct 31 01:16:45.707805 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:16:45.707805    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cac35176-4262-433c-ba9d-ded7a7c11502-tmp-dir\") pod \"auth-test-model-3be6fa-kserve-6f5d4f988f-zc2bc\" (UID: \"cac35176-4262-433c-ba9d-ded7a7c11502\") " pod="ai-tenant-e2e-shared-b-w1-3be6fa/auth-test-model-3be6fa-kserve-6f5d4f988f-zc2bc"
Oct 31 01:16:45.708062 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:16:45.707855    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cac35176-4262-433c-ba9d-ded7a7c11502-tls-certs\") pod \"auth-test-model-3be6fa-kserve-6f5d4f988f-zc2bc\" (UID: \"cac35176-4262-433c-ba9d-ded7a7c11502\") " pod="ai-tenant-e2e-shared-b-w1-3be6fa/auth-test-model-3be6fa-kserve-6f5d4f988f-zc2bc"
Oct 31 01:16:45.708062 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:16:45.707901    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cac35176-4262-433c-ba9d-ded7a7c11502-home\") pod \"auth-test-model-3be6fa-kserve-6f5d4f988f-zc2bc\" (UID: \"cac35176-4262-433c-ba9d-ded7a7c11502\") " pod="ai-tenant-e2e-shared-b-w1-3be6fa/auth-test-model-3be6fa-kserve-6f5d4f988f-zc2bc"
Oct 31 01:16:45.708062 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:16:45.707985    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cac35176-4262-433c-ba9d-ded7a7c11502-model-cache\") pod \"auth-test-model-3be6fa-kserve-6f5d4f988f-zc2bc\" (UID: \"cac35176-4262-433c-ba9d-ded7a7c11502\") " pod="ai-tenant-e2e-shared-b-w1-3be6fa/auth-test-model-3be6fa-kserve-6f5d4f988f-zc2bc"
Oct 31 01:16:45.708062 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:16:45.708030    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xlmz4\" (UniqueName: \"kubernetes.io/projected/cac35176-4262-433c-ba9d-ded7a7c11502-kube-api-access-xlmz4\") pod \"auth-test-model-3be6fa-kserve-6f5d4f988f-zc2bc\" (UID: \"cac35176-4262-433c-ba9d-ded7a7c11502\") " pod="ai-tenant-e2e-shared-b-w1-3be6fa/auth-test-model-3be6fa-kserve-6f5d4f988f-zc2bc"
Oct 31 01:16:45.708380 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:16:45.708338    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cac35176-4262-433c-ba9d-ded7a7c11502-tmp-dir\") pod \"auth-test-model-3be6fa-kserve-6f5d4f988f-zc2bc\" (UID: \"cac35176-4262-433c-ba9d-ded7a7c11502\") " pod="ai-tenant-e2e-shared-b-w1-3be6fa/auth-test-model-3be6fa-kserve-6f5d4f988f-zc2bc"
Oct 31 01:16:45.708380 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:16:45.708343    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cac35176-4262-433c-ba9d-ded7a7c11502-model-cache\") pod \"auth-test-model-3be6fa-kserve-6f5d4f988f-zc2bc\" (UID: \"cac35176-4262-433c-ba9d-ded7a7c11502\") " pod="ai-tenant-e2e-shared-b-w1-3be6fa/auth-test-model-3be6fa-kserve-6f5d4f988f-zc2bc"
Oct 31 01:16:45.708572 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:16:45.708407    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cac35176-4262-433c-ba9d-ded7a7c11502-home\") pod \"auth-test-model-3be6fa-kserve-6f5d4f988f-zc2bc\" (UID: \"cac35176-4262-433c-ba9d-ded7a7c11502\") " pod="ai-tenant-e2e-shared-b-w1-3be6fa/auth-test-model-3be6fa-kserve-6f5d4f988f-zc2bc"
Oct 31 01:16:45.710320 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:16:45.710294    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cac35176-4262-433c-ba9d-ded7a7c11502-dshm\") pod \"auth-test-model-3be6fa-kserve-6f5d4f988f-zc2bc\" (UID: \"cac35176-4262-433c-ba9d-ded7a7c11502\") " pod="ai-tenant-e2e-shared-b-w1-3be6fa/auth-test-model-3be6fa-kserve-6f5d4f988f-zc2bc"
Oct 31 01:16:45.710712 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:16:45.710693    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cac35176-4262-433c-ba9d-ded7a7c11502-tls-certs\") pod \"auth-test-model-3be6fa-kserve-6f5d4f988f-zc2bc\" (UID: \"cac35176-4262-433c-ba9d-ded7a7c11502\") " pod="ai-tenant-e2e-shared-b-w1-3be6fa/auth-test-model-3be6fa-kserve-6f5d4f988f-zc2bc"
Oct 31 01:16:45.767387 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:16:45.767315    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xlmz4\" (UniqueName: \"kubernetes.io/projected/cac35176-4262-433c-ba9d-ded7a7c11502-kube-api-access-xlmz4\") pod \"auth-test-model-3be6fa-kserve-6f5d4f988f-zc2bc\" (UID: \"cac35176-4262-433c-ba9d-ded7a7c11502\") " pod="ai-tenant-e2e-shared-b-w1-3be6fa/auth-test-model-3be6fa-kserve-6f5d4f988f-zc2bc"
Oct 31 01:16:45.816970 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:16:45.816932    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w1-3be6fa/auth-test-model-3be6fa-kserve-6f5d4f988f-zc2bc"
Oct 31 01:16:46.195108 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:16:46.195082    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w1-3be6fa/auth-test-model-3be6fa-kserve-6f5d4f988f-zc2bc"]
Oct 31 01:16:46.195985 ip-10-0-132-173 kubenswrapper[4284]: W1031 01:16:46.195957    4284 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podcac35176_4262_433c_ba9d_ded7a7c11502.slice/crio-d57bb213456eb4f892c5850dc8bfb20600c3302582ed2eef1a185cae4e8a9027 WatchSource:0}: Error finding container d57bb213456eb4f892c5850dc8bfb20600c3302582ed2eef1a185cae4e8a9027: Status 404 returned error can't find the container with id d57bb213456eb4f892c5850dc8bfb20600c3302582ed2eef1a185cae4e8a9027
Oct 31 01:16:46.327780 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:16:46.327735    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w1-3be6fa/auth-test-model-3be6fa-kserve-6f5d4f988f-zc2bc" event={"ID":"cac35176-4262-433c-ba9d-ded7a7c11502","Type":"ContainerStarted","Data":"cc56aba19a1e4f7deb0abd5fdec6d09d085a27cf5e06637ea0f4ad037202257f"}
Oct 31 01:16:46.327780 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:16:46.327786    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w1-3be6fa/auth-test-model-3be6fa-kserve-6f5d4f988f-zc2bc" event={"ID":"cac35176-4262-433c-ba9d-ded7a7c11502","Type":"ContainerStarted","Data":"d57bb213456eb4f892c5850dc8bfb20600c3302582ed2eef1a185cae4e8a9027"}
Oct 31 01:16:46.446500 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:16:46.446352    4284 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-w1-3be6fa/auth-test-model-3be6fa-kserve-6f5d4f988f-zc2bc" podStartSLOduration=1.446339055 podStartE2EDuration="1.446339055s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 01:16:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 01:16:46.445717982 +0000 UTC m=+2920.590724898" watchObservedRunningTime="2026-10-31 01:16:46.446339055 +0000 UTC m=+2920.591345971"
Oct 31 01:16:46.817308 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:16:46.817205    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-w1-3be6fa/auth-test-model-3be6fa-kserve-6f5d4f988f-zc2bc"
Oct 31 01:16:47.015642 ip-10-0-132-173 kubenswrapper[4284]: E1031 01:16:47.015610    4284 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fd8sp" containerName="extractor"
Oct 31 01:16:47.292562 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:16:47.292527    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:16:52.075578 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:16:52.075540    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:16:55.817750 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:16:55.817709    4284 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-w1-3be6fa/auth-test-model-3be6fa-kserve-6f5d4f988f-zc2bc"
Oct 31 01:16:55.836342 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:16:55.836317    4284 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-w1-3be6fa/auth-test-model-3be6fa-kserve-6f5d4f988f-zc2bc"
Oct 31 01:16:56.415853 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:16:56.415823    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-w1-3be6fa/auth-test-model-3be6fa-kserve-6f5d4f988f-zc2bc"
Oct 31 01:16:56.824024 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:16:56.823942    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:17:01.588827 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:17:01.588790    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:17:05.002710 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:17:05.002668    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:17:05.301846 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:17:05.301749    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:17:06.494934 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:17:06.494899    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:17:08.037364 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:17:08.037324    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:17:11.335327 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:17:11.335276    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:17:16.045652 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:17:16.045579    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:17:17.016654 ip-10-0-132-173 kubenswrapper[4284]: E1031 01:17:17.016615    4284 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fd8sp" containerName="extractor"
Oct 31 01:17:20.528486 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:17:20.528450    4284 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-8ddd2c-7799d58f4-f7842"]
Oct 31 01:17:20.532792 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:17:20.532768    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-8ddd2c-7799d58f4-f7842"
Oct 31 01:17:20.601055 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:17:20.601021    4284 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-explicit-8ddd2c-serving-cert\""
Oct 31 01:17:20.604173 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:17:20.604137    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-8ddd2c-7799d58f4-f7842"]
Oct 31 01:17:20.679811 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:17:20.679774    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t622l\" (UniqueName: \"kubernetes.io/projected/4b2c2e2c-9b90-4bd9-acd6-0c9db4003197-kube-api-access-t622l\") pod \"maas-api-e2e-explicit-8ddd2c-7799d58f4-f7842\" (UID: \"4b2c2e2c-9b90-4bd9-acd6-0c9db4003197\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-8ddd2c-7799d58f4-f7842"
Oct 31 01:17:20.679987 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:17:20.679876    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4b2c2e2c-9b90-4bd9-acd6-0c9db4003197-metrics-certs\") pod \"maas-api-e2e-explicit-8ddd2c-7799d58f4-f7842\" (UID: \"4b2c2e2c-9b90-4bd9-acd6-0c9db4003197\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-8ddd2c-7799d58f4-f7842"
Oct 31 01:17:20.679987 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:17:20.679953    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4b2c2e2c-9b90-4bd9-acd6-0c9db4003197-openshift-service-ca\") pod \"maas-api-e2e-explicit-8ddd2c-7799d58f4-f7842\" (UID: \"4b2c2e2c-9b90-4bd9-acd6-0c9db4003197\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-8ddd2c-7799d58f4-f7842"
Oct 31 01:17:20.680099 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:17:20.680014    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4b2c2e2c-9b90-4bd9-acd6-0c9db4003197-maas-api-tls\") pod \"maas-api-e2e-explicit-8ddd2c-7799d58f4-f7842\" (UID: \"4b2c2e2c-9b90-4bd9-acd6-0c9db4003197\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-8ddd2c-7799d58f4-f7842"
Oct 31 01:17:20.714846 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:17:20.714802    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:17:20.781440 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:17:20.781340    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4b2c2e2c-9b90-4bd9-acd6-0c9db4003197-openshift-service-ca\") pod \"maas-api-e2e-explicit-8ddd2c-7799d58f4-f7842\" (UID: \"4b2c2e2c-9b90-4bd9-acd6-0c9db4003197\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-8ddd2c-7799d58f4-f7842"
Oct 31 01:17:20.788100 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:17:20.787860    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4b2c2e2c-9b90-4bd9-acd6-0c9db4003197-maas-api-tls\") pod \"maas-api-e2e-explicit-8ddd2c-7799d58f4-f7842\" (UID: \"4b2c2e2c-9b90-4bd9-acd6-0c9db4003197\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-8ddd2c-7799d58f4-f7842"
Oct 31 01:17:20.788100 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:17:20.788045    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t622l\" (UniqueName: \"kubernetes.io/projected/4b2c2e2c-9b90-4bd9-acd6-0c9db4003197-kube-api-access-t622l\") pod \"maas-api-e2e-explicit-8ddd2c-7799d58f4-f7842\" (UID: \"4b2c2e2c-9b90-4bd9-acd6-0c9db4003197\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-8ddd2c-7799d58f4-f7842"
Oct 31 01:17:20.788414 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:17:20.788114    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4b2c2e2c-9b90-4bd9-acd6-0c9db4003197-metrics-certs\") pod \"maas-api-e2e-explicit-8ddd2c-7799d58f4-f7842\" (UID: \"4b2c2e2c-9b90-4bd9-acd6-0c9db4003197\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-8ddd2c-7799d58f4-f7842"
Oct 31 01:17:20.788939 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:17:20.782482    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4b2c2e2c-9b90-4bd9-acd6-0c9db4003197-openshift-service-ca\") pod \"maas-api-e2e-explicit-8ddd2c-7799d58f4-f7842\" (UID: \"4b2c2e2c-9b90-4bd9-acd6-0c9db4003197\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-8ddd2c-7799d58f4-f7842"
Oct 31 01:17:20.789105 ip-10-0-132-173 kubenswrapper[4284]: E1031 01:17:20.789088    4284 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-explicit-8ddd2c-serving-cert: secret "maas-api-e2e-explicit-8ddd2c-serving-cert" not found
Oct 31 01:17:20.789532 ip-10-0-132-173 kubenswrapper[4284]: E1031 01:17:20.789507    4284 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4b2c2e2c-9b90-4bd9-acd6-0c9db4003197-maas-api-tls podName:4b2c2e2c-9b90-4bd9-acd6-0c9db4003197 nodeName:}" failed. No retries permitted until 2026-10-31 01:17:21.289482464 +0000 UTC m=+2955.434489372 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/4b2c2e2c-9b90-4bd9-acd6-0c9db4003197-maas-api-tls") pod "maas-api-e2e-explicit-8ddd2c-7799d58f4-f7842" (UID: "4b2c2e2c-9b90-4bd9-acd6-0c9db4003197") : secret "maas-api-e2e-explicit-8ddd2c-serving-cert" not found
Oct 31 01:17:20.791727 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:17:20.791701    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4b2c2e2c-9b90-4bd9-acd6-0c9db4003197-metrics-certs\") pod \"maas-api-e2e-explicit-8ddd2c-7799d58f4-f7842\" (UID: \"4b2c2e2c-9b90-4bd9-acd6-0c9db4003197\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-8ddd2c-7799d58f4-f7842"
Oct 31 01:17:20.848908 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:17:20.848871    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t622l\" (UniqueName: \"kubernetes.io/projected/4b2c2e2c-9b90-4bd9-acd6-0c9db4003197-kube-api-access-t622l\") pod \"maas-api-e2e-explicit-8ddd2c-7799d58f4-f7842\" (UID: \"4b2c2e2c-9b90-4bd9-acd6-0c9db4003197\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-8ddd2c-7799d58f4-f7842"
Oct 31 01:17:20.905667 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:17:20.905629    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:17:21.060758 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:17:21.060637    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:17:21.294040 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:17:21.294011    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4b2c2e2c-9b90-4bd9-acd6-0c9db4003197-maas-api-tls\") pod \"maas-api-e2e-explicit-8ddd2c-7799d58f4-f7842\" (UID: \"4b2c2e2c-9b90-4bd9-acd6-0c9db4003197\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-8ddd2c-7799d58f4-f7842"
Oct 31 01:17:21.309265 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:17:21.307933    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4b2c2e2c-9b90-4bd9-acd6-0c9db4003197-maas-api-tls\") pod \"maas-api-e2e-explicit-8ddd2c-7799d58f4-f7842\" (UID: \"4b2c2e2c-9b90-4bd9-acd6-0c9db4003197\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-8ddd2c-7799d58f4-f7842"
Oct 31 01:17:21.328176 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:17:21.328091    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:17:21.446061 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:17:21.446026    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-8ddd2c-7799d58f4-f7842"
Oct 31 01:17:21.831589 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:17:21.831562    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-8ddd2c-7799d58f4-f7842"]
Oct 31 01:17:21.832549 ip-10-0-132-173 kubenswrapper[4284]: W1031 01:17:21.832522    4284 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4b2c2e2c_9b90_4bd9_acd6_0c9db4003197.slice/crio-3be70a1422cfaa328a10397908f83bf12e6f45b1639b12c99933d6bbd57ac495 WatchSource:0}: Error finding container 3be70a1422cfaa328a10397908f83bf12e6f45b1639b12c99933d6bbd57ac495: Status 404 returned error can't find the container with id 3be70a1422cfaa328a10397908f83bf12e6f45b1639b12c99933d6bbd57ac495
Oct 31 01:17:22.568572 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:17:22.568535    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-8ddd2c-7799d58f4-f7842" event={"ID":"4b2c2e2c-9b90-4bd9-acd6-0c9db4003197","Type":"ContainerStarted","Data":"3be70a1422cfaa328a10397908f83bf12e6f45b1639b12c99933d6bbd57ac495"}
Oct 31 01:17:23.576903 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:17:23.576866    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-8ddd2c-7799d58f4-f7842" event={"ID":"4b2c2e2c-9b90-4bd9-acd6-0c9db4003197","Type":"ContainerStarted","Data":"42046bc5b0b9485f0d0ef2020cf15848823c8ef71d6e53c76fe9a9217e227172"}
Oct 31 01:17:23.577326 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:17:23.576995    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-8ddd2c-7799d58f4-f7842"
Oct 31 01:17:23.694829 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:17:23.694762    4284 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-8ddd2c-7799d58f4-f7842" podStartSLOduration=2.703350277 podStartE2EDuration="3.694746188s" totalImagesPullingTime="991.395911ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 01:17:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 01:17:23.69190647 +0000 UTC m=+2957.836913388" watchObservedRunningTime="2026-10-31 01:17:23.694746188 +0000 UTC m=+2957.839753103"
Oct 31 01:17:25.698383 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:17:25.698344    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:17:29.604791 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:17:29.604758    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-8ddd2c-7799d58f4-f7842"
Oct 31 01:17:30.396615 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:17:30.396581    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:17:35.182939 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:17:35.182897    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:17:39.882367 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:17:39.882327    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:17:44.608877 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:17:44.608834    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:17:47.014860 ip-10-0-132-173 kubenswrapper[4284]: E1031 01:17:47.014816    4284 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fd8sp" containerName="extractor"
Oct 31 01:17:49.333993 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:17:49.333952    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:17:54.040319 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:17:54.040276    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:17:58.817912 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:17:58.817861    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:18:03.586929 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:18:03.586884    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:18:05.654082 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:18:05.654040    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:18:08.363815 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:18:08.363770    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:18:13.193850 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:18:13.193811    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:18:17.015329 ip-10-0-132-173 kubenswrapper[4284]: E1031 01:18:17.015236    4284 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fd8sp" containerName="extractor"
Oct 31 01:18:18.084752 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:18:18.084717    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:18:20.206013 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:18:20.205978    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:18:22.861173 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:18:22.861138    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:18:27.649114 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:18:27.649059    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:18:32.387687 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:18:32.387654    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:18:37.198346 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:18:37.198305    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:18:42.186267 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:18:42.186229    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:18:42.417134 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:18:42.417101    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:18:45.037384 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:18:45.037352    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-b63c24-openshift-default-d8c64855f-snk9h"]
Oct 31 01:18:45.037809 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:18:45.037546    4284 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w1-b63c24-openshift-default-d8c64855f-snk9h" podUID="8b30791d-d7f2-425d-ad30-078d7ff079ee" containerName="istio-proxy" containerID="cri-o://ecc0a5bd9132f158bfc1c997618730ff87f033074322c55c0d6a84961317fbf6" gracePeriod=30
Oct 31 01:18:46.021014 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:18:46.020970    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:18:47.015954 ip-10-0-132-173 kubenswrapper[4284]: E1031 01:18:47.015919    4284 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fd8sp" containerName="extractor"
Oct 31 01:18:47.053020 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:18:47.052979    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:18:50.319939 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:18:50.319873    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-b63c24-openshift-default-d8c64855f-snk9h"
Oct 31 01:18:50.447242 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:18:50.447188    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8b30791d-d7f2-425d-ad30-078d7ff079ee-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8b30791d-d7f2-425d-ad30-078d7ff079ee-istio-envoy\") pod \"8b30791d-d7f2-425d-ad30-078d7ff079ee\" (UID: \"8b30791d-d7f2-425d-ad30-078d7ff079ee\") "
Oct 31 01:18:50.447432 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:18:50.447297    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/8b30791d-d7f2-425d-ad30-078d7ff079ee-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8b30791d-d7f2-425d-ad30-078d7ff079ee-istio-podinfo\") pod \"8b30791d-d7f2-425d-ad30-078d7ff079ee\" (UID: \"8b30791d-d7f2-425d-ad30-078d7ff079ee\") "
Oct 31 01:18:50.447432 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:18:50.447320    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8b30791d-d7f2-425d-ad30-078d7ff079ee-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8b30791d-d7f2-425d-ad30-078d7ff079ee-credential-socket\") pod \"8b30791d-d7f2-425d-ad30-078d7ff079ee\" (UID: \"8b30791d-d7f2-425d-ad30-078d7ff079ee\") "
Oct 31 01:18:50.447432 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:18:50.447373    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8b30791d-d7f2-425d-ad30-078d7ff079ee-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8b30791d-d7f2-425d-ad30-078d7ff079ee-workload-socket\") pod \"8b30791d-d7f2-425d-ad30-078d7ff079ee\" (UID: \"8b30791d-d7f2-425d-ad30-078d7ff079ee\") "
Oct 31 01:18:50.447582 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:18:50.447429    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/8b30791d-d7f2-425d-ad30-078d7ff079ee-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8b30791d-d7f2-425d-ad30-078d7ff079ee-istiod-ca-cert\") pod \"8b30791d-d7f2-425d-ad30-078d7ff079ee\" (UID: \"8b30791d-d7f2-425d-ad30-078d7ff079ee\") "
Oct 31 01:18:50.447644 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:18:50.447621    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8b30791d-d7f2-425d-ad30-078d7ff079ee-kube-api-access-9kh62\" (UniqueName: \"kubernetes.io/projected/8b30791d-d7f2-425d-ad30-078d7ff079ee-kube-api-access-9kh62\") pod \"8b30791d-d7f2-425d-ad30-078d7ff079ee\" (UID: \"8b30791d-d7f2-425d-ad30-078d7ff079ee\") "
Oct 31 01:18:50.447699 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:18:50.447659    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8b30791d-d7f2-425d-ad30-078d7ff079ee-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8b30791d-d7f2-425d-ad30-078d7ff079ee-istio-data\") pod \"8b30791d-d7f2-425d-ad30-078d7ff079ee\" (UID: \"8b30791d-d7f2-425d-ad30-078d7ff079ee\") "
Oct 31 01:18:50.447760 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:18:50.447666    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8b30791d-d7f2-425d-ad30-078d7ff079ee-credential-socket" pod "8b30791d-d7f2-425d-ad30-078d7ff079ee" (UID: "8b30791d-d7f2-425d-ad30-078d7ff079ee"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:18:50.447760 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:18:50.447739    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8b30791d-d7f2-425d-ad30-078d7ff079ee-istio-token\" (UniqueName: \"kubernetes.io/projected/8b30791d-d7f2-425d-ad30-078d7ff079ee-istio-token\") pod \"8b30791d-d7f2-425d-ad30-078d7ff079ee\" (UID: \"8b30791d-d7f2-425d-ad30-078d7ff079ee\") "
Oct 31 01:18:50.447872 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:18:50.447742    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8b30791d-d7f2-425d-ad30-078d7ff079ee-workload-socket" pod "8b30791d-d7f2-425d-ad30-078d7ff079ee" (UID: "8b30791d-d7f2-425d-ad30-078d7ff079ee"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:18:50.447872 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:18:50.447802    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8b30791d-d7f2-425d-ad30-078d7ff079ee-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8b30791d-d7f2-425d-ad30-078d7ff079ee-workload-certs\") pod \"8b30791d-d7f2-425d-ad30-078d7ff079ee\" (UID: \"8b30791d-d7f2-425d-ad30-078d7ff079ee\") "
Oct 31 01:18:50.447991 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:18:50.447902    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8b30791d-d7f2-425d-ad30-078d7ff079ee-istio-data" pod "8b30791d-d7f2-425d-ad30-078d7ff079ee" (UID: "8b30791d-d7f2-425d-ad30-078d7ff079ee"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:18:50.447991 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:18:50.447913    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8b30791d-d7f2-425d-ad30-078d7ff079ee-istiod-ca-cert" pod "8b30791d-d7f2-425d-ad30-078d7ff079ee" (UID: "8b30791d-d7f2-425d-ad30-078d7ff079ee"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 01:18:50.448458 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:18:50.448357    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8b30791d-d7f2-425d-ad30-078d7ff079ee-workload-certs" pod "8b30791d-d7f2-425d-ad30-078d7ff079ee" (UID: "8b30791d-d7f2-425d-ad30-078d7ff079ee"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:18:50.448585 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:18:50.448536    4284 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8b30791d-d7f2-425d-ad30-078d7ff079ee-istio-data\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:18:50.448585 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:18:50.448556    4284 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8b30791d-d7f2-425d-ad30-078d7ff079ee-workload-certs\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:18:50.448585 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:18:50.448571    4284 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8b30791d-d7f2-425d-ad30-078d7ff079ee-credential-socket\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:18:50.448768 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:18:50.448587    4284 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8b30791d-d7f2-425d-ad30-078d7ff079ee-workload-socket\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:18:50.448768 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:18:50.448602    4284 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8b30791d-d7f2-425d-ad30-078d7ff079ee-istiod-ca-cert\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:18:50.449809 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:18:50.449788    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/8b30791d-d7f2-425d-ad30-078d7ff079ee-istio-podinfo" pod "8b30791d-d7f2-425d-ad30-078d7ff079ee" (UID: "8b30791d-d7f2-425d-ad30-078d7ff079ee"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 01:18:50.449911 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:18:50.449797    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8b30791d-d7f2-425d-ad30-078d7ff079ee-istio-envoy" pod "8b30791d-d7f2-425d-ad30-078d7ff079ee" (UID: "8b30791d-d7f2-425d-ad30-078d7ff079ee"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:18:50.449983 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:18:50.449962    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8b30791d-d7f2-425d-ad30-078d7ff079ee-istio-token" pod "8b30791d-d7f2-425d-ad30-078d7ff079ee" (UID: "8b30791d-d7f2-425d-ad30-078d7ff079ee"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:18:50.450027 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:18:50.449991    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8b30791d-d7f2-425d-ad30-078d7ff079ee-kube-api-access-9kh62" pod "8b30791d-d7f2-425d-ad30-078d7ff079ee" (UID: "8b30791d-d7f2-425d-ad30-078d7ff079ee"). InnerVolumeSpecName "kube-api-access-9kh62". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:18:50.549686 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:18:50.549642    4284 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9kh62\" (UniqueName: \"kubernetes.io/projected/8b30791d-d7f2-425d-ad30-078d7ff079ee-kube-api-access-9kh62\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:18:50.549686 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:18:50.549676    4284 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8b30791d-d7f2-425d-ad30-078d7ff079ee-istio-token\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:18:50.549686 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:18:50.549686    4284 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8b30791d-d7f2-425d-ad30-078d7ff079ee-istio-envoy\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:18:50.549686 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:18:50.549695    4284 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8b30791d-d7f2-425d-ad30-078d7ff079ee-istio-podinfo\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:18:51.138857 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:18:51.138824    4284 generic.go:356] "Generic (PLEG): container finished" podID="8b30791d-d7f2-425d-ad30-078d7ff079ee" containerID="ecc0a5bd9132f158bfc1c997618730ff87f033074322c55c0d6a84961317fbf6" exitCode=0
Oct 31 01:18:51.139056 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:18:51.138867    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-b63c24-openshift-default-d8c64855f-snk9h"
Oct 31 01:18:51.139056 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:18:51.138880    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-b63c24-openshift-default-d8c64855f-snk9h" event={"ID":"8b30791d-d7f2-425d-ad30-078d7ff079ee","Type":"ContainerDied","Data":"ecc0a5bd9132f158bfc1c997618730ff87f033074322c55c0d6a84961317fbf6"}
Oct 31 01:18:51.139056 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:18:51.138926    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-b63c24-openshift-default-d8c64855f-snk9h" event={"ID":"8b30791d-d7f2-425d-ad30-078d7ff079ee","Type":"ContainerDied","Data":"e7f937b5618782f746b4dd478705572591df47cac6c43dfd274450082676bdae"}
Oct 31 01:18:51.139056 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:18:51.138946    4284 scope.go:122] "RemoveContainer" containerID="ecc0a5bd9132f158bfc1c997618730ff87f033074322c55c0d6a84961317fbf6"
Oct 31 01:18:51.149848 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:18:51.149830    4284 scope.go:122] "RemoveContainer" containerID="ecc0a5bd9132f158bfc1c997618730ff87f033074322c55c0d6a84961317fbf6"
Oct 31 01:18:51.150140 ip-10-0-132-173 kubenswrapper[4284]: E1031 01:18:51.150121    4284 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ecc0a5bd9132f158bfc1c997618730ff87f033074322c55c0d6a84961317fbf6\": container with ID starting with ecc0a5bd9132f158bfc1c997618730ff87f033074322c55c0d6a84961317fbf6 not found: ID does not exist" containerID="ecc0a5bd9132f158bfc1c997618730ff87f033074322c55c0d6a84961317fbf6"
Oct 31 01:18:51.150233 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:18:51.150147    4284 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ecc0a5bd9132f158bfc1c997618730ff87f033074322c55c0d6a84961317fbf6"} err="failed to get container status \"ecc0a5bd9132f158bfc1c997618730ff87f033074322c55c0d6a84961317fbf6\": rpc error: code = NotFound desc = could not find container \"ecc0a5bd9132f158bfc1c997618730ff87f033074322c55c0d6a84961317fbf6\": container with ID starting with ecc0a5bd9132f158bfc1c997618730ff87f033074322c55c0d6a84961317fbf6 not found: ID does not exist"
Oct 31 01:18:51.342518 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:18:51.342481    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-b63c24-openshift-default-d8c64855f-snk9h"]
Oct 31 01:18:51.346952 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:18:51.346925    4284 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w1-b63c24-openshift-default-d8c64855f-snk9h"]
Oct 31 01:18:51.904849 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:18:51.904815    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:18:51.971870 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:18:51.971835    4284 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="8b30791d-d7f2-425d-ad30-078d7ff079ee" path="/var/lib/kubelet/pods/8b30791d-d7f2-425d-ad30-078d7ff079ee/volumes"
Oct 31 01:18:55.967201 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:18:55.967161    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:18:56.704879 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:18:56.704837    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:19:01.694623 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:19:01.694579    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:19:05.976096 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:19:05.976060    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:19:06.488124 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:19:06.488084    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:19:11.387330 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:19:11.387291    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:19:16.220150 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:19:16.220112    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:19:17.014692 ip-10-0-132-173 kubenswrapper[4284]: E1031 01:19:17.014656    4284 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fd8sp" containerName="extractor"
Oct 31 01:19:21.043779 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:19:21.043742    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:19:25.877974 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:19:25.877936    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:19:29.302025 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:19:29.301984    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:19:30.891144 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:19:30.891102    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:19:35.636223 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:19:35.636182    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:19:36.962339 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:19:36.962307    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-8ddd2c-69b95b5867-pfjc2"]
Oct 31 01:19:37.222565 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:19:37.222479    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:19:37.422476 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:19:37.422433    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:19:40.375496 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:19:40.375453    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:19:41.962696 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:19:41.962653    4284 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-explicit-8ddd2c-69b95b5867-pfjc2" podUID="10934caf-df36-4907-9138-27d4426c81f1" containerName="payload-pre-processing" containerID="cri-o://518045322ba14ac9e60b1d5268135bc1588aaf2510b26482e4eb36c61efbde4a" gracePeriod=25
Oct 31 01:19:42.225538 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:19:42.225473    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-explicit-8ddd2c-69b95b5867-pfjc2"
Oct 31 01:19:42.281824 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:19:42.281785    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/10934caf-df36-4907-9138-27d4426c81f1-kube-api-access-pw4kf\" (UniqueName: \"kubernetes.io/projected/10934caf-df36-4907-9138-27d4426c81f1-kube-api-access-pw4kf\") pod \"10934caf-df36-4907-9138-27d4426c81f1\" (UID: \"10934caf-df36-4907-9138-27d4426c81f1\") "
Oct 31 01:19:42.282020 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:19:42.281845    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/10934caf-df36-4907-9138-27d4426c81f1-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/10934caf-df36-4907-9138-27d4426c81f1-plugins-config-volume\") pod \"10934caf-df36-4907-9138-27d4426c81f1\" (UID: \"10934caf-df36-4907-9138-27d4426c81f1\") "
Oct 31 01:19:42.282360 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:19:42.282329    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/10934caf-df36-4907-9138-27d4426c81f1-plugins-config-volume" pod "10934caf-df36-4907-9138-27d4426c81f1" (UID: "10934caf-df36-4907-9138-27d4426c81f1"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 01:19:42.284176 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:19:42.284141    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/10934caf-df36-4907-9138-27d4426c81f1-kube-api-access-pw4kf" pod "10934caf-df36-4907-9138-27d4426c81f1" (UID: "10934caf-df36-4907-9138-27d4426c81f1"). InnerVolumeSpecName "kube-api-access-pw4kf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:19:42.383656 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:19:42.383625    4284 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pw4kf\" (UniqueName: \"kubernetes.io/projected/10934caf-df36-4907-9138-27d4426c81f1-kube-api-access-pw4kf\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:19:42.383656 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:19:42.383655    4284 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/10934caf-df36-4907-9138-27d4426c81f1-plugins-config-volume\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:19:42.459991 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:19:42.459955    4284 generic.go:356] "Generic (PLEG): container finished" podID="10934caf-df36-4907-9138-27d4426c81f1" containerID="518045322ba14ac9e60b1d5268135bc1588aaf2510b26482e4eb36c61efbde4a" exitCode=0
Oct 31 01:19:42.460239 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:19:42.460002    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-explicit-8ddd2c-69b95b5867-pfjc2"
Oct 31 01:19:42.460239 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:19:42.460041    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-explicit-8ddd2c-69b95b5867-pfjc2" event={"ID":"10934caf-df36-4907-9138-27d4426c81f1","Type":"ContainerDied","Data":"518045322ba14ac9e60b1d5268135bc1588aaf2510b26482e4eb36c61efbde4a"}
Oct 31 01:19:42.460239 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:19:42.460097    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-explicit-8ddd2c-69b95b5867-pfjc2" event={"ID":"10934caf-df36-4907-9138-27d4426c81f1","Type":"ContainerDied","Data":"7854698877d0a6638c8276177807893baca4b741308421d3e1a8c5a7da1951b4"}
Oct 31 01:19:42.460239 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:19:42.460127    4284 scope.go:122] "RemoveContainer" containerID="518045322ba14ac9e60b1d5268135bc1588aaf2510b26482e4eb36c61efbde4a"
Oct 31 01:19:42.469773 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:19:42.469757    4284 scope.go:122] "RemoveContainer" containerID="518045322ba14ac9e60b1d5268135bc1588aaf2510b26482e4eb36c61efbde4a"
Oct 31 01:19:42.470071 ip-10-0-132-173 kubenswrapper[4284]: E1031 01:19:42.470053    4284 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"518045322ba14ac9e60b1d5268135bc1588aaf2510b26482e4eb36c61efbde4a\": container with ID starting with 518045322ba14ac9e60b1d5268135bc1588aaf2510b26482e4eb36c61efbde4a not found: ID does not exist" containerID="518045322ba14ac9e60b1d5268135bc1588aaf2510b26482e4eb36c61efbde4a"
Oct 31 01:19:42.470125 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:19:42.470078    4284 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"518045322ba14ac9e60b1d5268135bc1588aaf2510b26482e4eb36c61efbde4a"} err="failed to get container status \"518045322ba14ac9e60b1d5268135bc1588aaf2510b26482e4eb36c61efbde4a\": rpc error: code = NotFound desc = could not find container \"518045322ba14ac9e60b1d5268135bc1588aaf2510b26482e4eb36c61efbde4a\": container with ID starting with 518045322ba14ac9e60b1d5268135bc1588aaf2510b26482e4eb36c61efbde4a not found: ID does not exist"
Oct 31 01:19:42.620184 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:19:42.620102    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-8ddd2c-69b95b5867-pfjc2"]
Oct 31 01:19:42.627232 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:19:42.627184    4284 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-8ddd2c-69b95b5867-pfjc2"]
Oct 31 01:19:43.963344 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:19:43.963309    4284 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="10934caf-df36-4907-9138-27d4426c81f1" path="/var/lib/kubelet/pods/10934caf-df36-4907-9138-27d4426c81f1/volumes"
Oct 31 01:19:44.440963 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:19:44.440925    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-8ddd2c-7799d58f4-f7842"]
Oct 31 01:19:44.441170 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:19:44.441148    4284 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-8ddd2c-7799d58f4-f7842" podUID="4b2c2e2c-9b90-4bd9-acd6-0c9db4003197" containerName="maas-api" containerID="cri-o://42046bc5b0b9485f0d0ef2020cf15848823c8ef71d6e53c76fe9a9217e227172" gracePeriod=30
Oct 31 01:19:44.536846 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:19:44.536808    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:19:44.599271 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:19:44.599188    4284 prober.go:122] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-8ddd2c-7799d58f4-f7842" podUID="4b2c2e2c-9b90-4bd9-acd6-0c9db4003197" containerName="maas-api" probeResult="failure" output="Get \"https://10.134.0.174:8443/health\": dial tcp 10.134.0.174:8443: connect: connection refused"
Oct 31 01:19:44.738909 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:19:44.738883    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-8ddd2c-7799d58f4-f7842"
Oct 31 01:19:44.812332 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:19:44.812300    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/4b2c2e2c-9b90-4bd9-acd6-0c9db4003197-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4b2c2e2c-9b90-4bd9-acd6-0c9db4003197-maas-api-tls\") pod \"4b2c2e2c-9b90-4bd9-acd6-0c9db4003197\" (UID: \"4b2c2e2c-9b90-4bd9-acd6-0c9db4003197\") "
Oct 31 01:19:44.812549 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:19:44.812469    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/4b2c2e2c-9b90-4bd9-acd6-0c9db4003197-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4b2c2e2c-9b90-4bd9-acd6-0c9db4003197-openshift-service-ca\") pod \"4b2c2e2c-9b90-4bd9-acd6-0c9db4003197\" (UID: \"4b2c2e2c-9b90-4bd9-acd6-0c9db4003197\") "
Oct 31 01:19:44.812549 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:19:44.812529    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/4b2c2e2c-9b90-4bd9-acd6-0c9db4003197-metrics-certs\" (UniqueName: \"kubernetes.io/secret/4b2c2e2c-9b90-4bd9-acd6-0c9db4003197-metrics-certs\") pod \"4b2c2e2c-9b90-4bd9-acd6-0c9db4003197\" (UID: \"4b2c2e2c-9b90-4bd9-acd6-0c9db4003197\") "
Oct 31 01:19:44.812683 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:19:44.812572    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4b2c2e2c-9b90-4bd9-acd6-0c9db4003197-kube-api-access-t622l\" (UniqueName: \"kubernetes.io/projected/4b2c2e2c-9b90-4bd9-acd6-0c9db4003197-kube-api-access-t622l\") pod \"4b2c2e2c-9b90-4bd9-acd6-0c9db4003197\" (UID: \"4b2c2e2c-9b90-4bd9-acd6-0c9db4003197\") "
Oct 31 01:19:44.812868 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:19:44.812841    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4b2c2e2c-9b90-4bd9-acd6-0c9db4003197-openshift-service-ca" pod "4b2c2e2c-9b90-4bd9-acd6-0c9db4003197" (UID: "4b2c2e2c-9b90-4bd9-acd6-0c9db4003197"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 01:19:44.813026 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:19:44.813002    4284 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4b2c2e2c-9b90-4bd9-acd6-0c9db4003197-openshift-service-ca\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:19:44.814744 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:19:44.814721    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4b2c2e2c-9b90-4bd9-acd6-0c9db4003197-metrics-certs" pod "4b2c2e2c-9b90-4bd9-acd6-0c9db4003197" (UID: "4b2c2e2c-9b90-4bd9-acd6-0c9db4003197"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 01:19:44.814878 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:19:44.814860    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4b2c2e2c-9b90-4bd9-acd6-0c9db4003197-maas-api-tls" pod "4b2c2e2c-9b90-4bd9-acd6-0c9db4003197" (UID: "4b2c2e2c-9b90-4bd9-acd6-0c9db4003197"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 01:19:44.815222 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:19:44.815191    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4b2c2e2c-9b90-4bd9-acd6-0c9db4003197-kube-api-access-t622l" pod "4b2c2e2c-9b90-4bd9-acd6-0c9db4003197" (UID: "4b2c2e2c-9b90-4bd9-acd6-0c9db4003197"). InnerVolumeSpecName "kube-api-access-t622l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:19:44.913970 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:19:44.913937    4284 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4b2c2e2c-9b90-4bd9-acd6-0c9db4003197-metrics-certs\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:19:44.913970 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:19:44.913965    4284 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t622l\" (UniqueName: \"kubernetes.io/projected/4b2c2e2c-9b90-4bd9-acd6-0c9db4003197-kube-api-access-t622l\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:19:44.913970 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:19:44.913974    4284 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4b2c2e2c-9b90-4bd9-acd6-0c9db4003197-maas-api-tls\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:19:45.479601 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:19:45.479566    4284 generic.go:356] "Generic (PLEG): container finished" podID="4b2c2e2c-9b90-4bd9-acd6-0c9db4003197" containerID="42046bc5b0b9485f0d0ef2020cf15848823c8ef71d6e53c76fe9a9217e227172" exitCode=0
Oct 31 01:19:45.480039 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:19:45.479617    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-8ddd2c-7799d58f4-f7842"
Oct 31 01:19:45.480039 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:19:45.479657    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-8ddd2c-7799d58f4-f7842" event={"ID":"4b2c2e2c-9b90-4bd9-acd6-0c9db4003197","Type":"ContainerDied","Data":"42046bc5b0b9485f0d0ef2020cf15848823c8ef71d6e53c76fe9a9217e227172"}
Oct 31 01:19:45.480039 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:19:45.479715    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-8ddd2c-7799d58f4-f7842" event={"ID":"4b2c2e2c-9b90-4bd9-acd6-0c9db4003197","Type":"ContainerDied","Data":"3be70a1422cfaa328a10397908f83bf12e6f45b1639b12c99933d6bbd57ac495"}
Oct 31 01:19:45.480039 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:19:45.479742    4284 scope.go:122] "RemoveContainer" containerID="42046bc5b0b9485f0d0ef2020cf15848823c8ef71d6e53c76fe9a9217e227172"
Oct 31 01:19:45.491150 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:19:45.491126    4284 scope.go:122] "RemoveContainer" containerID="42046bc5b0b9485f0d0ef2020cf15848823c8ef71d6e53c76fe9a9217e227172"
Oct 31 01:19:45.491533 ip-10-0-132-173 kubenswrapper[4284]: E1031 01:19:45.491502    4284 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"42046bc5b0b9485f0d0ef2020cf15848823c8ef71d6e53c76fe9a9217e227172\": container with ID starting with 42046bc5b0b9485f0d0ef2020cf15848823c8ef71d6e53c76fe9a9217e227172 not found: ID does not exist" containerID="42046bc5b0b9485f0d0ef2020cf15848823c8ef71d6e53c76fe9a9217e227172"
Oct 31 01:19:45.491649 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:19:45.491541    4284 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"42046bc5b0b9485f0d0ef2020cf15848823c8ef71d6e53c76fe9a9217e227172"} err="failed to get container status \"42046bc5b0b9485f0d0ef2020cf15848823c8ef71d6e53c76fe9a9217e227172\": rpc error: code = NotFound desc = could not find container \"42046bc5b0b9485f0d0ef2020cf15848823c8ef71d6e53c76fe9a9217e227172\": container with ID starting with 42046bc5b0b9485f0d0ef2020cf15848823c8ef71d6e53c76fe9a9217e227172 not found: ID does not exist"
Oct 31 01:19:45.652291 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:19:45.652254    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-8ddd2c-7799d58f4-f7842"]
Oct 31 01:19:45.656717 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:19:45.656686    4284 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-8ddd2c-7799d58f4-f7842"]
Oct 31 01:19:45.985238 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:19:45.984252    4284 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="4b2c2e2c-9b90-4bd9-acd6-0c9db4003197" path="/var/lib/kubelet/pods/4b2c2e2c-9b90-4bd9-acd6-0c9db4003197/volumes"
Oct 31 01:19:46.015563 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:19:46.015515    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:19:46.326902 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:19:46.326816    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:19:46.480819 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:19:46.480721    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:19:47.014709 ip-10-0-132-173 kubenswrapper[4284]: E1031 01:19:47.014662    4284 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fd8sp" containerName="extractor"
Oct 31 01:19:47.166172 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:19:47.166133    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:19:47.363502 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:19:47.363420    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:19:51.769900 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:19:51.769855    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:19:52.190602 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:19:52.190572    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:19:56.553488 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:19:56.553449    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:19:58.798827 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:19:58.798784    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:20:00.051094 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:20:00.051060    4284 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7292-0d3744"]
Oct 31 01:20:00.051642 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:20:00.051628    4284 cpu_manager.go:414] "containerMap: removing container" podUID="8b30791d-d7f2-425d-ad30-078d7ff079ee" containerName="istio-proxy"
Oct 31 01:20:00.051691 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:20:00.051643    4284 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8b30791d-d7f2-425d-ad30-078d7ff079ee" containerName="istio-proxy"
Oct 31 01:20:00.051691 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:20:00.051651    4284 cpu_manager.go:414] "containerMap: removing container" podUID="4b2c2e2c-9b90-4bd9-acd6-0c9db4003197" containerName="maas-api"
Oct 31 01:20:00.051691 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:20:00.051656    4284 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4b2c2e2c-9b90-4bd9-acd6-0c9db4003197" containerName="maas-api"
Oct 31 01:20:00.051691 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:20:00.051677    4284 cpu_manager.go:414] "containerMap: removing container" podUID="10934caf-df36-4907-9138-27d4426c81f1" containerName="payload-pre-processing"
Oct 31 01:20:00.051691 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:20:00.051683    4284 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="10934caf-df36-4907-9138-27d4426c81f1" containerName="payload-pre-processing"
Oct 31 01:20:00.051859 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:20:00.051763    4284 memory_manager.go:365] "RemoveStaleState removing state" podUID="4b2c2e2c-9b90-4bd9-acd6-0c9db4003197" containerName="maas-api"
Oct 31 01:20:00.051859 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:20:00.051769    4284 memory_manager.go:365] "RemoveStaleState removing state" podUID="8b30791d-d7f2-425d-ad30-078d7ff079ee" containerName="istio-proxy"
Oct 31 01:20:00.051859 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:20:00.051779    4284 memory_manager.go:365] "RemoveStaleState removing state" podUID="10934caf-df36-4907-9138-27d4426c81f1" containerName="payload-pre-processing"
Oct 31 01:20:00.056639 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:20:00.056621    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7292-0d3744"
Oct 31 01:20:00.106951 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:20:00.106915    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7292-0d3744"]
Oct 31 01:20:00.151726 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:20:00.151690    4284 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-8gxx2\""
Oct 31 01:20:00.162686 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:20:00.162653    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w99mq\" (UniqueName: \"kubernetes.io/projected/20f25b20-8c14-44d8-bc6e-fd5452b23eb2-kube-api-access-w99mq\") pod \"test-curl-7292-0d3744\" (UID: \"20f25b20-8c14-44d8-bc6e-fd5452b23eb2\") " pod="opendatahub/test-curl-7292-0d3744"
Oct 31 01:20:00.263572 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:20:00.263542    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w99mq\" (UniqueName: \"kubernetes.io/projected/20f25b20-8c14-44d8-bc6e-fd5452b23eb2-kube-api-access-w99mq\") pod \"test-curl-7292-0d3744\" (UID: \"20f25b20-8c14-44d8-bc6e-fd5452b23eb2\") " pod="opendatahub/test-curl-7292-0d3744"
Oct 31 01:20:00.323150 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:20:00.323069    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w99mq\" (UniqueName: \"kubernetes.io/projected/20f25b20-8c14-44d8-bc6e-fd5452b23eb2-kube-api-access-w99mq\") pod \"test-curl-7292-0d3744\" (UID: \"20f25b20-8c14-44d8-bc6e-fd5452b23eb2\") " pod="opendatahub/test-curl-7292-0d3744"
Oct 31 01:20:00.369115 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:20:00.369085    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7292-0d3744"
Oct 31 01:20:00.735035 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:20:00.735007    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7292-0d3744"]
Oct 31 01:20:00.736200 ip-10-0-132-173 kubenswrapper[4284]: W1031 01:20:00.736176    4284 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod20f25b20_8c14_44d8_bc6e_fd5452b23eb2.slice/crio-50fc6345041dbca98079ff145bcef8d6903865f394b0631398d572e8306794ab WatchSource:0}: Error finding container 50fc6345041dbca98079ff145bcef8d6903865f394b0631398d572e8306794ab: Status 404 returned error can't find the container with id 50fc6345041dbca98079ff145bcef8d6903865f394b0631398d572e8306794ab
Oct 31 01:20:01.404726 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:20:01.404688    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:20:01.571278 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:20:01.571239    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7292-0d3744" event={"ID":"20f25b20-8c14-44d8-bc6e-fd5452b23eb2","Type":"ContainerStarted","Data":"ad5d59f8b61c34bd9332e90d6df0fd01e2429ccaa363fecd9123e7712597c86a"}
Oct 31 01:20:01.571278 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:20:01.571284    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7292-0d3744" event={"ID":"20f25b20-8c14-44d8-bc6e-fd5452b23eb2","Type":"ContainerStarted","Data":"50fc6345041dbca98079ff145bcef8d6903865f394b0631398d572e8306794ab"}
Oct 31 01:20:01.680911 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:20:01.680813    4284 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/test-curl-7292-0d3744" podStartSLOduration=1.680799615 podStartE2EDuration="1.680799615s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 01:20:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 01:20:01.680334614 +0000 UTC m=+3115.825341531" watchObservedRunningTime="2026-10-31 01:20:01.680799615 +0000 UTC m=+3115.825806532"
Oct 31 01:20:03.086712 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:20:03.086676    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7292-0d3744"]
Oct 31 01:20:03.090660 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:20:03.090629    4284 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7292-0d3744"]
Oct 31 01:20:03.252850 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:20:03.252818    4284 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7292-a9d7ca"]
Oct 31 01:20:03.253474 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:20:03.253459    4284 cpu_manager.go:414] "containerMap: removing container" podUID="20f25b20-8c14-44d8-bc6e-fd5452b23eb2" containerName="test-curl-7292-0d3744"
Oct 31 01:20:03.253536 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:20:03.253477    4284 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="20f25b20-8c14-44d8-bc6e-fd5452b23eb2" containerName="test-curl-7292-0d3744"
Oct 31 01:20:03.253599 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:20:03.253587    4284 memory_manager.go:365] "RemoveStaleState removing state" podUID="20f25b20-8c14-44d8-bc6e-fd5452b23eb2" containerName="test-curl-7292-0d3744"
Oct 31 01:20:03.257433 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:20:03.257410    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7292-a9d7ca"
Oct 31 01:20:03.308875 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:20:03.308845    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7292-a9d7ca"]
Oct 31 01:20:03.400396 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:20:03.400293    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8tchn\" (UniqueName: \"kubernetes.io/projected/348c8d06-1148-4251-84c9-bdf052ada488-kube-api-access-8tchn\") pod \"test-curl-7292-a9d7ca\" (UID: \"348c8d06-1148-4251-84c9-bdf052ada488\") " pod="opendatahub/test-curl-7292-a9d7ca"
Oct 31 01:20:03.501817 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:20:03.501776    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8tchn\" (UniqueName: \"kubernetes.io/projected/348c8d06-1148-4251-84c9-bdf052ada488-kube-api-access-8tchn\") pod \"test-curl-7292-a9d7ca\" (UID: \"348c8d06-1148-4251-84c9-bdf052ada488\") " pod="opendatahub/test-curl-7292-a9d7ca"
Oct 31 01:20:03.560718 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:20:03.560678    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8tchn\" (UniqueName: \"kubernetes.io/projected/348c8d06-1148-4251-84c9-bdf052ada488-kube-api-access-8tchn\") pod \"test-curl-7292-a9d7ca\" (UID: \"348c8d06-1148-4251-84c9-bdf052ada488\") " pod="opendatahub/test-curl-7292-a9d7ca"
Oct 31 01:20:03.570588 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:20:03.570560    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7292-a9d7ca"
Oct 31 01:20:03.583356 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:20:03.583319    4284 kuberuntime_container.go:898] "Killing container with a grace period" pod="opendatahub/test-curl-7292-0d3744" podUID="20f25b20-8c14-44d8-bc6e-fd5452b23eb2" containerName="test-curl-7292-0d3744" containerID="cri-o://ad5d59f8b61c34bd9332e90d6df0fd01e2429ccaa363fecd9123e7712597c86a" gracePeriod=2
Oct 31 01:20:03.632460 ip-10-0-132-173 kubenswrapper[4284]: E1031 01:20:03.632424    4284 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-7292-0d3744\" is forbidden: User \"system:node:ip-10-0-132-173.us-west-2.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-132-173.us-west-2.compute.internal' and this object" podUID="20f25b20-8c14-44d8-bc6e-fd5452b23eb2" pod="opendatahub/test-curl-7292-0d3744"
Oct 31 01:20:03.944001 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:20:03.943970    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7292-a9d7ca"]
Oct 31 01:20:03.944814 ip-10-0-132-173 kubenswrapper[4284]: W1031 01:20:03.944786    4284 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod348c8d06_1148_4251_84c9_bdf052ada488.slice/crio-ffdad524fbcf34df9d88fc83e438b9e051275e89af279bfce4f402446310ad77 WatchSource:0}: Error finding container ffdad524fbcf34df9d88fc83e438b9e051275e89af279bfce4f402446310ad77: Status 404 returned error can't find the container with id ffdad524fbcf34df9d88fc83e438b9e051275e89af279bfce4f402446310ad77
Oct 31 01:20:04.591014 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:20:04.590978    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7292-a9d7ca" event={"ID":"348c8d06-1148-4251-84c9-bdf052ada488","Type":"ContainerStarted","Data":"62f307f38da79ffea46546282967de68d94e0642904cd763a1197ee79e9f4eb9"}
Oct 31 01:20:04.591014 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:20:04.591027    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7292-a9d7ca" event={"ID":"348c8d06-1148-4251-84c9-bdf052ada488","Type":"ContainerStarted","Data":"ffdad524fbcf34df9d88fc83e438b9e051275e89af279bfce4f402446310ad77"}
Oct 31 01:20:04.702708 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:20:04.702654    4284 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/test-curl-7292-a9d7ca" podStartSLOduration=1.702639006 podStartE2EDuration="1.702639006s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 01:20:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 01:20:04.701973119 +0000 UTC m=+3118.846980018" watchObservedRunningTime="2026-10-31 01:20:04.702639006 +0000 UTC m=+3118.847645920"
Oct 31 01:20:05.833248 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:20:05.833203    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7292-0d3744"
Oct 31 01:20:05.928291 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:20:05.928258    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/20f25b20-8c14-44d8-bc6e-fd5452b23eb2-kube-api-access-w99mq\" (UniqueName: \"kubernetes.io/projected/20f25b20-8c14-44d8-bc6e-fd5452b23eb2-kube-api-access-w99mq\") pod \"20f25b20-8c14-44d8-bc6e-fd5452b23eb2\" (UID: \"20f25b20-8c14-44d8-bc6e-fd5452b23eb2\") "
Oct 31 01:20:05.930909 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:20:05.930874    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/20f25b20-8c14-44d8-bc6e-fd5452b23eb2-kube-api-access-w99mq" pod "20f25b20-8c14-44d8-bc6e-fd5452b23eb2" (UID: "20f25b20-8c14-44d8-bc6e-fd5452b23eb2"). InnerVolumeSpecName "kube-api-access-w99mq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:20:05.967056 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:20:05.967020    4284 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="20f25b20-8c14-44d8-bc6e-fd5452b23eb2" path="/var/lib/kubelet/pods/20f25b20-8c14-44d8-bc6e-fd5452b23eb2/volumes"
Oct 31 01:20:06.030244 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:20:06.030192    4284 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w99mq\" (UniqueName: \"kubernetes.io/projected/20f25b20-8c14-44d8-bc6e-fd5452b23eb2-kube-api-access-w99mq\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:20:06.199148 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:20:06.199065    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:20:06.258547 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:20:06.258512    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7292-a9d7ca"]
Oct 31 01:20:06.265687 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:20:06.263022    4284 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7292-a9d7ca"]
Oct 31 01:20:06.389882 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:20:06.389847    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:20:06.604068 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:20:06.603982    4284 generic.go:356] "Generic (PLEG): container finished" podID="20f25b20-8c14-44d8-bc6e-fd5452b23eb2" containerID="ad5d59f8b61c34bd9332e90d6df0fd01e2429ccaa363fecd9123e7712597c86a" exitCode=137
Oct 31 01:20:06.604068 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:20:06.604028    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7292-0d3744"
Oct 31 01:20:06.604351 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:20:06.604079    4284 scope.go:122] "RemoveContainer" containerID="ad5d59f8b61c34bd9332e90d6df0fd01e2429ccaa363fecd9123e7712597c86a"
Oct 31 01:20:06.604351 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:20:06.604158    4284 kuberuntime_container.go:898] "Killing container with a grace period" pod="opendatahub/test-curl-7292-a9d7ca" podUID="348c8d06-1148-4251-84c9-bdf052ada488" containerName="test-curl-7292-a9d7ca" containerID="cri-o://62f307f38da79ffea46546282967de68d94e0642904cd763a1197ee79e9f4eb9" gracePeriod=2
Oct 31 01:20:06.617791 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:20:06.617768    4284 scope.go:122] "RemoveContainer" containerID="ad5d59f8b61c34bd9332e90d6df0fd01e2429ccaa363fecd9123e7712597c86a"
Oct 31 01:20:06.618178 ip-10-0-132-173 kubenswrapper[4284]: E1031 01:20:06.618155    4284 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ad5d59f8b61c34bd9332e90d6df0fd01e2429ccaa363fecd9123e7712597c86a\": container with ID starting with ad5d59f8b61c34bd9332e90d6df0fd01e2429ccaa363fecd9123e7712597c86a not found: ID does not exist" containerID="ad5d59f8b61c34bd9332e90d6df0fd01e2429ccaa363fecd9123e7712597c86a"
Oct 31 01:20:06.618308 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:20:06.618184    4284 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ad5d59f8b61c34bd9332e90d6df0fd01e2429ccaa363fecd9123e7712597c86a"} err="failed to get container status \"ad5d59f8b61c34bd9332e90d6df0fd01e2429ccaa363fecd9123e7712597c86a\": rpc error: code = NotFound desc = could not find container \"ad5d59f8b61c34bd9332e90d6df0fd01e2429ccaa363fecd9123e7712597c86a\": container with ID starting with ad5d59f8b61c34bd9332e90d6df0fd01e2429ccaa363fecd9123e7712597c86a not found: ID does not exist"
Oct 31 01:20:06.653414 ip-10-0-132-173 kubenswrapper[4284]: E1031 01:20:06.653369    4284 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-7292-a9d7ca\" is forbidden: User \"system:node:ip-10-0-132-173.us-west-2.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-132-173.us-west-2.compute.internal' and this object" podUID="348c8d06-1148-4251-84c9-bdf052ada488" pod="opendatahub/test-curl-7292-a9d7ca"
Oct 31 01:20:06.703359 ip-10-0-132-173 kubenswrapper[4284]: E1031 01:20:06.703315    4284 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-7292-a9d7ca\" is forbidden: User \"system:node:ip-10-0-132-173.us-west-2.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-132-173.us-west-2.compute.internal' and this object" podUID="348c8d06-1148-4251-84c9-bdf052ada488" pod="opendatahub/test-curl-7292-a9d7ca"
Oct 31 01:20:06.736259 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:20:06.736207    4284 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7292-e8494e"]
Oct 31 01:20:06.736881 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:20:06.736864    4284 cpu_manager.go:414] "containerMap: removing container" podUID="348c8d06-1148-4251-84c9-bdf052ada488" containerName="test-curl-7292-a9d7ca"
Oct 31 01:20:06.736933 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:20:06.736884    4284 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="348c8d06-1148-4251-84c9-bdf052ada488" containerName="test-curl-7292-a9d7ca"
Oct 31 01:20:06.736980 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:20:06.736970    4284 memory_manager.go:365] "RemoveStaleState removing state" podUID="348c8d06-1148-4251-84c9-bdf052ada488" containerName="test-curl-7292-a9d7ca"
Oct 31 01:20:06.741694 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:20:06.741677    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7292-e8494e"
Oct 31 01:20:06.794134 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:20:06.794100    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7292-e8494e"]
Oct 31 01:20:06.847809 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:20:06.847771    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5f8cz\" (UniqueName: \"kubernetes.io/projected/9e209d2c-44db-4176-932f-fb584f2d507b-kube-api-access-5f8cz\") pod \"test-curl-7292-e8494e\" (UID: \"9e209d2c-44db-4176-932f-fb584f2d507b\") " pod="opendatahub/test-curl-7292-e8494e"
Oct 31 01:20:06.898994 ip-10-0-132-173 kubenswrapper[4284]: E1031 01:20:06.898906    4284 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-7292-a9d7ca\" is forbidden: User \"system:node:ip-10-0-132-173.us-west-2.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-132-173.us-west-2.compute.internal' and this object" podUID="348c8d06-1148-4251-84c9-bdf052ada488" pod="opendatahub/test-curl-7292-a9d7ca"
Oct 31 01:20:06.948855 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:20:06.948814    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5f8cz\" (UniqueName: \"kubernetes.io/projected/9e209d2c-44db-4176-932f-fb584f2d507b-kube-api-access-5f8cz\") pod \"test-curl-7292-e8494e\" (UID: \"9e209d2c-44db-4176-932f-fb584f2d507b\") " pod="opendatahub/test-curl-7292-e8494e"
Oct 31 01:20:07.007955 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:20:07.007921    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5f8cz\" (UniqueName: \"kubernetes.io/projected/9e209d2c-44db-4176-932f-fb584f2d507b-kube-api-access-5f8cz\") pod \"test-curl-7292-e8494e\" (UID: \"9e209d2c-44db-4176-932f-fb584f2d507b\") " pod="opendatahub/test-curl-7292-e8494e"
Oct 31 01:20:07.053360 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:20:07.053327    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7292-e8494e"
Oct 31 01:20:07.163770 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:20:07.163732    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:20:07.424916 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:20:07.424893    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7292-e8494e"]
Oct 31 01:20:07.425551 ip-10-0-132-173 kubenswrapper[4284]: W1031 01:20:07.425521    4284 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9e209d2c_44db_4176_932f_fb584f2d507b.slice/crio-684980fea817306a9b8de5a260748d1e777e8a6d3a3fc7a223c880474d843fd1 WatchSource:0}: Error finding container 684980fea817306a9b8de5a260748d1e777e8a6d3a3fc7a223c880474d843fd1: Status 404 returned error can't find the container with id 684980fea817306a9b8de5a260748d1e777e8a6d3a3fc7a223c880474d843fd1
Oct 31 01:20:07.611223 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:20:07.611174    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7292-e8494e" event={"ID":"9e209d2c-44db-4176-932f-fb584f2d507b","Type":"ContainerStarted","Data":"180f4cd58fb43b26a04f24928f0b270ce35124d604ac80e59cbff58dcda9a052"}
Oct 31 01:20:07.611420 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:20:07.611261    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7292-e8494e" event={"ID":"9e209d2c-44db-4176-932f-fb584f2d507b","Type":"ContainerStarted","Data":"684980fea817306a9b8de5a260748d1e777e8a6d3a3fc7a223c880474d843fd1"}
Oct 31 01:20:07.781102 ip-10-0-132-173 kubenswrapper[4284]: E1031 01:20:07.781004    4284 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-7292-a9d7ca\" is forbidden: User \"system:node:ip-10-0-132-173.us-west-2.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-132-173.us-west-2.compute.internal' and this object" podUID="348c8d06-1148-4251-84c9-bdf052ada488" pod="opendatahub/test-curl-7292-a9d7ca"
Oct 31 01:20:08.855667 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:20:08.855636    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7292-a9d7ca"
Oct 31 01:20:08.970896 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:20:08.970864    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/348c8d06-1148-4251-84c9-bdf052ada488-kube-api-access-8tchn\" (UniqueName: \"kubernetes.io/projected/348c8d06-1148-4251-84c9-bdf052ada488-kube-api-access-8tchn\") pod \"348c8d06-1148-4251-84c9-bdf052ada488\" (UID: \"348c8d06-1148-4251-84c9-bdf052ada488\") "
Oct 31 01:20:08.973098 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:20:08.973070    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/348c8d06-1148-4251-84c9-bdf052ada488-kube-api-access-8tchn" pod "348c8d06-1148-4251-84c9-bdf052ada488" (UID: "348c8d06-1148-4251-84c9-bdf052ada488"). InnerVolumeSpecName "kube-api-access-8tchn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:20:08.996406 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:20:08.996375    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:20:09.085662 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:20:09.085616    4284 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8tchn\" (UniqueName: \"kubernetes.io/projected/348c8d06-1148-4251-84c9-bdf052ada488-kube-api-access-8tchn\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:20:09.123324 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:20:09.123240    4284 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/test-curl-7292-e8494e" podStartSLOduration=3.123201278 podStartE2EDuration="3.123201278s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 01:20:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 01:20:07.731994223 +0000 UTC m=+3121.877001140" watchObservedRunningTime="2026-10-31 01:20:09.123201278 +0000 UTC m=+3123.268208198"
Oct 31 01:20:09.124674 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:20:09.124635    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7292-e8494e"]
Oct 31 01:20:09.129740 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:20:09.129709    4284 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7292-e8494e"]
Oct 31 01:20:09.274673 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:20:09.274595    4284 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7292-382c01"]
Oct 31 01:20:09.275159 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:20:09.275146    4284 cpu_manager.go:414] "containerMap: removing container" podUID="9e209d2c-44db-4176-932f-fb584f2d507b" containerName="test-curl-7292-e8494e"
Oct 31 01:20:09.275227 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:20:09.275161    4284 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9e209d2c-44db-4176-932f-fb584f2d507b" containerName="test-curl-7292-e8494e"
Oct 31 01:20:09.275283 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:20:09.275265    4284 memory_manager.go:365] "RemoveStaleState removing state" podUID="9e209d2c-44db-4176-932f-fb584f2d507b" containerName="test-curl-7292-e8494e"
Oct 31 01:20:09.278594 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:20:09.278575    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7292-382c01"
Oct 31 01:20:09.286809 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:20:09.286786    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9swhm\" (UniqueName: \"kubernetes.io/projected/9e130dfb-0c79-4143-9096-7842bb1ed2f2-kube-api-access-9swhm\") pod \"test-curl-7292-382c01\" (UID: \"9e130dfb-0c79-4143-9096-7842bb1ed2f2\") " pod="opendatahub/test-curl-7292-382c01"
Oct 31 01:20:09.331333 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:20:09.331303    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7292-382c01"]
Oct 31 01:20:09.388054 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:20:09.388023    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9swhm\" (UniqueName: \"kubernetes.io/projected/9e130dfb-0c79-4143-9096-7842bb1ed2f2-kube-api-access-9swhm\") pod \"test-curl-7292-382c01\" (UID: \"9e130dfb-0c79-4143-9096-7842bb1ed2f2\") " pod="opendatahub/test-curl-7292-382c01"
Oct 31 01:20:09.449289 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:20:09.449254    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9swhm\" (UniqueName: \"kubernetes.io/projected/9e130dfb-0c79-4143-9096-7842bb1ed2f2-kube-api-access-9swhm\") pod \"test-curl-7292-382c01\" (UID: \"9e130dfb-0c79-4143-9096-7842bb1ed2f2\") " pod="opendatahub/test-curl-7292-382c01"
Oct 31 01:20:09.590865 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:20:09.590758    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7292-382c01"
Oct 31 01:20:09.625367 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:20:09.625327    4284 generic.go:356] "Generic (PLEG): container finished" podID="348c8d06-1148-4251-84c9-bdf052ada488" containerID="62f307f38da79ffea46546282967de68d94e0642904cd763a1197ee79e9f4eb9" exitCode=137
Oct 31 01:20:09.625622 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:20:09.625428    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7292-a9d7ca"
Oct 31 01:20:09.625622 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:20:09.625481    4284 scope.go:122] "RemoveContainer" containerID="62f307f38da79ffea46546282967de68d94e0642904cd763a1197ee79e9f4eb9"
Oct 31 01:20:09.625901 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:20:09.625847    4284 kuberuntime_container.go:898] "Killing container with a grace period" pod="opendatahub/test-curl-7292-e8494e" podUID="9e209d2c-44db-4176-932f-fb584f2d507b" containerName="test-curl-7292-e8494e" containerID="cri-o://180f4cd58fb43b26a04f24928f0b270ce35124d604ac80e59cbff58dcda9a052" gracePeriod=2
Oct 31 01:20:09.640541 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:20:09.640469    4284 scope.go:122] "RemoveContainer" containerID="62f307f38da79ffea46546282967de68d94e0642904cd763a1197ee79e9f4eb9"
Oct 31 01:20:09.641068 ip-10-0-132-173 kubenswrapper[4284]: E1031 01:20:09.641045    4284 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"62f307f38da79ffea46546282967de68d94e0642904cd763a1197ee79e9f4eb9\": container with ID starting with 62f307f38da79ffea46546282967de68d94e0642904cd763a1197ee79e9f4eb9 not found: ID does not exist" containerID="62f307f38da79ffea46546282967de68d94e0642904cd763a1197ee79e9f4eb9"
Oct 31 01:20:09.641162 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:20:09.641079    4284 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"62f307f38da79ffea46546282967de68d94e0642904cd763a1197ee79e9f4eb9"} err="failed to get container status \"62f307f38da79ffea46546282967de68d94e0642904cd763a1197ee79e9f4eb9\": rpc error: code = NotFound desc = could not find container \"62f307f38da79ffea46546282967de68d94e0642904cd763a1197ee79e9f4eb9\": container with ID starting with 62f307f38da79ffea46546282967de68d94e0642904cd763a1197ee79e9f4eb9 not found: ID does not exist"
Oct 31 01:20:09.675547 ip-10-0-132-173 kubenswrapper[4284]: E1031 01:20:09.675489    4284 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-7292-e8494e\" is forbidden: User \"system:node:ip-10-0-132-173.us-west-2.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-132-173.us-west-2.compute.internal' and this object" podUID="9e209d2c-44db-4176-932f-fb584f2d507b" pod="opendatahub/test-curl-7292-e8494e"
Oct 31 01:20:09.724657 ip-10-0-132-173 kubenswrapper[4284]: E1031 01:20:09.724619    4284 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-7292-e8494e\" is forbidden: User \"system:node:ip-10-0-132-173.us-west-2.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-132-173.us-west-2.compute.internal' and this object" podUID="9e209d2c-44db-4176-932f-fb584f2d507b" pod="opendatahub/test-curl-7292-e8494e"
Oct 31 01:20:09.773737 ip-10-0-132-173 kubenswrapper[4284]: E1031 01:20:09.773690    4284 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-7292-a9d7ca\" is forbidden: User \"system:node:ip-10-0-132-173.us-west-2.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-132-173.us-west-2.compute.internal' and this object" podUID="348c8d06-1148-4251-84c9-bdf052ada488" pod="opendatahub/test-curl-7292-a9d7ca"
Oct 31 01:20:09.964139 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:20:09.964101    4284 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="348c8d06-1148-4251-84c9-bdf052ada488" path="/var/lib/kubelet/pods/348c8d06-1148-4251-84c9-bdf052ada488/volumes"
Oct 31 01:20:09.968983 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:20:09.968957    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7292-382c01"]
Oct 31 01:20:09.969955 ip-10-0-132-173 kubenswrapper[4284]: W1031 01:20:09.969930    4284 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9e130dfb_0c79_4143_9096_7842bb1ed2f2.slice/crio-2568b675877c8fa11c978a22eac284be442f8cff101358de788f9d37e6d9c947 WatchSource:0}: Error finding container 2568b675877c8fa11c978a22eac284be442f8cff101358de788f9d37e6d9c947: Status 404 returned error can't find the container with id 2568b675877c8fa11c978a22eac284be442f8cff101358de788f9d37e6d9c947
Oct 31 01:20:10.633555 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:20:10.633521    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7292-382c01" event={"ID":"9e130dfb-0c79-4143-9096-7842bb1ed2f2","Type":"ContainerStarted","Data":"a5d923dc285c4496b9ad935c1639338f2d56747edb4bdae9d4348fa2ff6f93f1"}
Oct 31 01:20:10.633555 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:20:10.633559    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7292-382c01" event={"ID":"9e130dfb-0c79-4143-9096-7842bb1ed2f2","Type":"ContainerStarted","Data":"2568b675877c8fa11c978a22eac284be442f8cff101358de788f9d37e6d9c947"}
Oct 31 01:20:10.746079 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:20:10.746025    4284 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/test-curl-7292-382c01" podStartSLOduration=1.746004658 podStartE2EDuration="1.746004658s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 01:20:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 01:20:10.743499778 +0000 UTC m=+3124.888506693" watchObservedRunningTime="2026-10-31 01:20:10.746004658 +0000 UTC m=+3124.891011576"
Oct 31 01:20:11.054301 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:20:11.054172    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:20:11.848990 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:20:11.848965    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7292-e8494e"
Oct 31 01:20:11.922715 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:20:11.922678    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9e209d2c-44db-4176-932f-fb584f2d507b-kube-api-access-5f8cz\" (UniqueName: \"kubernetes.io/projected/9e209d2c-44db-4176-932f-fb584f2d507b-kube-api-access-5f8cz\") pod \"9e209d2c-44db-4176-932f-fb584f2d507b\" (UID: \"9e209d2c-44db-4176-932f-fb584f2d507b\") "
Oct 31 01:20:11.924964 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:20:11.924934    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9e209d2c-44db-4176-932f-fb584f2d507b-kube-api-access-5f8cz" pod "9e209d2c-44db-4176-932f-fb584f2d507b" (UID: "9e209d2c-44db-4176-932f-fb584f2d507b"). InnerVolumeSpecName "kube-api-access-5f8cz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:20:11.964321 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:20:11.964288    4284 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="9e209d2c-44db-4176-932f-fb584f2d507b" path="/var/lib/kubelet/pods/9e209d2c-44db-4176-932f-fb584f2d507b/volumes"
Oct 31 01:20:12.023791 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:20:12.023757    4284 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5f8cz\" (UniqueName: \"kubernetes.io/projected/9e209d2c-44db-4176-932f-fb584f2d507b-kube-api-access-5f8cz\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:20:12.319323 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:20:12.319230    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7292-382c01"]
Oct 31 01:20:12.322438 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:20:12.322405    4284 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7292-382c01"]
Oct 31 01:20:12.477833 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:20:12.477799    4284 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7292-123d75"]
Oct 31 01:20:12.478393 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:20:12.478379    4284 cpu_manager.go:414] "containerMap: removing container" podUID="9e130dfb-0c79-4143-9096-7842bb1ed2f2" containerName="test-curl-7292-382c01"
Oct 31 01:20:12.478447 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:20:12.478397    4284 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9e130dfb-0c79-4143-9096-7842bb1ed2f2" containerName="test-curl-7292-382c01"
Oct 31 01:20:12.478505 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:20:12.478495    4284 memory_manager.go:365] "RemoveStaleState removing state" podUID="9e130dfb-0c79-4143-9096-7842bb1ed2f2" containerName="test-curl-7292-382c01"
Oct 31 01:20:12.482122 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:20:12.482104    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7292-123d75"
Oct 31 01:20:12.529788 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:20:12.529558    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mc55r\" (UniqueName: \"kubernetes.io/projected/bd590161-7df6-4088-a3f3-9d18cde8eca9-kube-api-access-mc55r\") pod \"test-curl-7292-123d75\" (UID: \"bd590161-7df6-4088-a3f3-9d18cde8eca9\") " pod="opendatahub/test-curl-7292-123d75"
Oct 31 01:20:12.532268 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:20:12.532244    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7292-123d75"]
Oct 31 01:20:12.630720 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:20:12.630614    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mc55r\" (UniqueName: \"kubernetes.io/projected/bd590161-7df6-4088-a3f3-9d18cde8eca9-kube-api-access-mc55r\") pod \"test-curl-7292-123d75\" (UID: \"bd590161-7df6-4088-a3f3-9d18cde8eca9\") " pod="opendatahub/test-curl-7292-123d75"
Oct 31 01:20:12.646477 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:20:12.646437    4284 generic.go:356] "Generic (PLEG): container finished" podID="9e209d2c-44db-4176-932f-fb584f2d507b" containerID="180f4cd58fb43b26a04f24928f0b270ce35124d604ac80e59cbff58dcda9a052" exitCode=137
Oct 31 01:20:12.646649 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:20:12.646485    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7292-e8494e"
Oct 31 01:20:12.646649 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:20:12.646511    4284 scope.go:122] "RemoveContainer" containerID="180f4cd58fb43b26a04f24928f0b270ce35124d604ac80e59cbff58dcda9a052"
Oct 31 01:20:12.646762 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:20:12.646667    4284 kuberuntime_container.go:898] "Killing container with a grace period" pod="opendatahub/test-curl-7292-382c01" podUID="9e130dfb-0c79-4143-9096-7842bb1ed2f2" containerName="test-curl-7292-382c01" containerID="cri-o://a5d923dc285c4496b9ad935c1639338f2d56747edb4bdae9d4348fa2ff6f93f1" gracePeriod=2
Oct 31 01:20:12.655372 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:20:12.655353    4284 scope.go:122] "RemoveContainer" containerID="180f4cd58fb43b26a04f24928f0b270ce35124d604ac80e59cbff58dcda9a052"
Oct 31 01:20:12.655661 ip-10-0-132-173 kubenswrapper[4284]: E1031 01:20:12.655642    4284 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"180f4cd58fb43b26a04f24928f0b270ce35124d604ac80e59cbff58dcda9a052\": container with ID starting with 180f4cd58fb43b26a04f24928f0b270ce35124d604ac80e59cbff58dcda9a052 not found: ID does not exist" containerID="180f4cd58fb43b26a04f24928f0b270ce35124d604ac80e59cbff58dcda9a052"
Oct 31 01:20:12.655705 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:20:12.655670    4284 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"180f4cd58fb43b26a04f24928f0b270ce35124d604ac80e59cbff58dcda9a052"} err="failed to get container status \"180f4cd58fb43b26a04f24928f0b270ce35124d604ac80e59cbff58dcda9a052\": rpc error: code = NotFound desc = could not find container \"180f4cd58fb43b26a04f24928f0b270ce35124d604ac80e59cbff58dcda9a052\": container with ID starting with 180f4cd58fb43b26a04f24928f0b270ce35124d604ac80e59cbff58dcda9a052 not found: ID does not exist"
Oct 31 01:20:12.696254 ip-10-0-132-173 kubenswrapper[4284]: E1031 01:20:12.696202    4284 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-7292-382c01\" is forbidden: User \"system:node:ip-10-0-132-173.us-west-2.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-132-173.us-west-2.compute.internal' and this object" podUID="9e130dfb-0c79-4143-9096-7842bb1ed2f2" pod="opendatahub/test-curl-7292-382c01"
Oct 31 01:20:12.709172 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:20:12.709142    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mc55r\" (UniqueName: \"kubernetes.io/projected/bd590161-7df6-4088-a3f3-9d18cde8eca9-kube-api-access-mc55r\") pod \"test-curl-7292-123d75\" (UID: \"bd590161-7df6-4088-a3f3-9d18cde8eca9\") " pod="opendatahub/test-curl-7292-123d75"
Oct 31 01:20:12.746808 ip-10-0-132-173 kubenswrapper[4284]: E1031 01:20:12.746770    4284 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-7292-382c01\" is forbidden: User \"system:node:ip-10-0-132-173.us-west-2.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-132-173.us-west-2.compute.internal' and this object" podUID="9e130dfb-0c79-4143-9096-7842bb1ed2f2" pod="opendatahub/test-curl-7292-382c01"
Oct 31 01:20:12.794129 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:20:12.794093    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7292-123d75"
Oct 31 01:20:13.166158 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:20:13.166133    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7292-123d75"]
Oct 31 01:20:13.166921 ip-10-0-132-173 kubenswrapper[4284]: W1031 01:20:13.166897    4284 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbd590161_7df6_4088_a3f3_9d18cde8eca9.slice/crio-bec6fb8228a93d559c9409d82789306823be1c43b7829c093255c70d35d7776b WatchSource:0}: Error finding container bec6fb8228a93d559c9409d82789306823be1c43b7829c093255c70d35d7776b: Status 404 returned error can't find the container with id bec6fb8228a93d559c9409d82789306823be1c43b7829c093255c70d35d7776b
Oct 31 01:20:13.654084 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:20:13.654045    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7292-123d75" event={"ID":"bd590161-7df6-4088-a3f3-9d18cde8eca9","Type":"ContainerStarted","Data":"780532a506a11aedc801a8e3eade3d25428496f5dff37d2d108f123ddd3be6b7"}
Oct 31 01:20:13.654583 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:20:13.654094    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7292-123d75" event={"ID":"bd590161-7df6-4088-a3f3-9d18cde8eca9","Type":"ContainerStarted","Data":"bec6fb8228a93d559c9409d82789306823be1c43b7829c093255c70d35d7776b"}
Oct 31 01:20:13.703238 ip-10-0-132-173 kubenswrapper[4284]: E1031 01:20:13.703170    4284 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-7292-382c01\" is forbidden: User \"system:node:ip-10-0-132-173.us-west-2.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-132-173.us-west-2.compute.internal' and this object" podUID="9e130dfb-0c79-4143-9096-7842bb1ed2f2" pod="opendatahub/test-curl-7292-382c01"
Oct 31 01:20:13.813492 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:20:13.813441    4284 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/test-curl-7292-123d75" podStartSLOduration=1.813426796 podStartE2EDuration="1.813426796s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 01:20:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 01:20:13.812736903 +0000 UTC m=+3127.957743820" watchObservedRunningTime="2026-10-31 01:20:13.813426796 +0000 UTC m=+3127.958433711"
Oct 31 01:20:14.880127 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:20:14.880100    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7292-382c01"
Oct 31 01:20:14.956898 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:20:14.956863    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9e130dfb-0c79-4143-9096-7842bb1ed2f2-kube-api-access-9swhm\" (UniqueName: \"kubernetes.io/projected/9e130dfb-0c79-4143-9096-7842bb1ed2f2-kube-api-access-9swhm\") pod \"9e130dfb-0c79-4143-9096-7842bb1ed2f2\" (UID: \"9e130dfb-0c79-4143-9096-7842bb1ed2f2\") "
Oct 31 01:20:14.959024 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:20:14.958998    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9e130dfb-0c79-4143-9096-7842bb1ed2f2-kube-api-access-9swhm" pod "9e130dfb-0c79-4143-9096-7842bb1ed2f2" (UID: "9e130dfb-0c79-4143-9096-7842bb1ed2f2"). InnerVolumeSpecName "kube-api-access-9swhm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:20:15.058879 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:20:15.058841    4284 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9swhm\" (UniqueName: \"kubernetes.io/projected/9e130dfb-0c79-4143-9096-7842bb1ed2f2-kube-api-access-9swhm\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:20:15.191947 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:20:15.191903    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7292-123d75"]
Oct 31 01:20:15.196432 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:20:15.196403    4284 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7292-123d75"]
Oct 31 01:20:15.668680 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:20:15.668646    4284 generic.go:356] "Generic (PLEG): container finished" podID="9e130dfb-0c79-4143-9096-7842bb1ed2f2" containerID="a5d923dc285c4496b9ad935c1639338f2d56747edb4bdae9d4348fa2ff6f93f1" exitCode=137
Oct 31 01:20:15.668862 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:20:15.668709    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7292-382c01"
Oct 31 01:20:15.668862 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:20:15.668750    4284 scope.go:122] "RemoveContainer" containerID="a5d923dc285c4496b9ad935c1639338f2d56747edb4bdae9d4348fa2ff6f93f1"
Oct 31 01:20:15.668974 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:20:15.668902    4284 kuberuntime_container.go:898] "Killing container with a grace period" pod="opendatahub/test-curl-7292-123d75" podUID="bd590161-7df6-4088-a3f3-9d18cde8eca9" containerName="test-curl-7292-123d75" containerID="cri-o://780532a506a11aedc801a8e3eade3d25428496f5dff37d2d108f123ddd3be6b7" gracePeriod=2
Oct 31 01:20:15.678308 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:20:15.678286    4284 scope.go:122] "RemoveContainer" containerID="a5d923dc285c4496b9ad935c1639338f2d56747edb4bdae9d4348fa2ff6f93f1"
Oct 31 01:20:15.678567 ip-10-0-132-173 kubenswrapper[4284]: E1031 01:20:15.678550    4284 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a5d923dc285c4496b9ad935c1639338f2d56747edb4bdae9d4348fa2ff6f93f1\": container with ID starting with a5d923dc285c4496b9ad935c1639338f2d56747edb4bdae9d4348fa2ff6f93f1 not found: ID does not exist" containerID="a5d923dc285c4496b9ad935c1639338f2d56747edb4bdae9d4348fa2ff6f93f1"
Oct 31 01:20:15.678627 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:20:15.678576    4284 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a5d923dc285c4496b9ad935c1639338f2d56747edb4bdae9d4348fa2ff6f93f1"} err="failed to get container status \"a5d923dc285c4496b9ad935c1639338f2d56747edb4bdae9d4348fa2ff6f93f1\": rpc error: code = NotFound desc = could not find container \"a5d923dc285c4496b9ad935c1639338f2d56747edb4bdae9d4348fa2ff6f93f1\": container with ID starting with a5d923dc285c4496b9ad935c1639338f2d56747edb4bdae9d4348fa2ff6f93f1 not found: ID does not exist"
Oct 31 01:20:15.718320 ip-10-0-132-173 kubenswrapper[4284]: E1031 01:20:15.718278    4284 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-7292-123d75\" is forbidden: User \"system:node:ip-10-0-132-173.us-west-2.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-132-173.us-west-2.compute.internal' and this object" podUID="bd590161-7df6-4088-a3f3-9d18cde8eca9" pod="opendatahub/test-curl-7292-123d75"
Oct 31 01:20:15.767811 ip-10-0-132-173 kubenswrapper[4284]: E1031 01:20:15.767772    4284 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-7292-382c01\" is forbidden: User \"system:node:ip-10-0-132-173.us-west-2.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-132-173.us-west-2.compute.internal' and this object" podUID="9e130dfb-0c79-4143-9096-7842bb1ed2f2" pod="opendatahub/test-curl-7292-382c01"
Oct 31 01:20:15.783783 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:20:15.783757    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:20:15.821539 ip-10-0-132-173 kubenswrapper[4284]: E1031 01:20:15.821490    4284 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-7292-123d75\" is forbidden: User \"system:node:ip-10-0-132-173.us-west-2.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-132-173.us-west-2.compute.internal' and this object" podUID="bd590161-7df6-4088-a3f3-9d18cde8eca9" pod="opendatahub/test-curl-7292-123d75"
Oct 31 01:20:15.963286 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:20:15.963168    4284 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="9e130dfb-0c79-4143-9096-7842bb1ed2f2" path="/var/lib/kubelet/pods/9e130dfb-0c79-4143-9096-7842bb1ed2f2/volumes"
Oct 31 01:20:16.007805 ip-10-0-132-173 kubenswrapper[4284]: E1031 01:20:16.007772    4284 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-7292-382c01\" is forbidden: User \"system:node:ip-10-0-132-173.us-west-2.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-132-173.us-west-2.compute.internal' and this object" podUID="9e130dfb-0c79-4143-9096-7842bb1ed2f2" pod="opendatahub/test-curl-7292-382c01"
Oct 31 01:20:16.056762 ip-10-0-132-173 kubenswrapper[4284]: E1031 01:20:16.056724    4284 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-7292-123d75\" is forbidden: User \"system:node:ip-10-0-132-173.us-west-2.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-132-173.us-west-2.compute.internal' and this object" podUID="bd590161-7df6-4088-a3f3-9d18cde8eca9" pod="opendatahub/test-curl-7292-123d75"
Oct 31 01:20:17.014787 ip-10-0-132-173 kubenswrapper[4284]: E1031 01:20:17.014749    4284 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fd8sp" containerName="extractor"
Oct 31 01:20:17.904438 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:20:17.904414    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7292-123d75"
Oct 31 01:20:17.993274 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:20:17.993189    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/bd590161-7df6-4088-a3f3-9d18cde8eca9-kube-api-access-mc55r\" (UniqueName: \"kubernetes.io/projected/bd590161-7df6-4088-a3f3-9d18cde8eca9-kube-api-access-mc55r\") pod \"bd590161-7df6-4088-a3f3-9d18cde8eca9\" (UID: \"bd590161-7df6-4088-a3f3-9d18cde8eca9\") "
Oct 31 01:20:17.995350 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:20:17.995327    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bd590161-7df6-4088-a3f3-9d18cde8eca9-kube-api-access-mc55r" pod "bd590161-7df6-4088-a3f3-9d18cde8eca9" (UID: "bd590161-7df6-4088-a3f3-9d18cde8eca9"). InnerVolumeSpecName "kube-api-access-mc55r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:20:18.094891 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:20:18.094853    4284 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mc55r\" (UniqueName: \"kubernetes.io/projected/bd590161-7df6-4088-a3f3-9d18cde8eca9-kube-api-access-mc55r\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:20:18.688549 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:20:18.688514    4284 generic.go:356] "Generic (PLEG): container finished" podID="bd590161-7df6-4088-a3f3-9d18cde8eca9" containerID="780532a506a11aedc801a8e3eade3d25428496f5dff37d2d108f123ddd3be6b7" exitCode=137
Oct 31 01:20:18.688748 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:20:18.688578    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7292-123d75"
Oct 31 01:20:18.688748 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:20:18.688628    4284 scope.go:122] "RemoveContainer" containerID="780532a506a11aedc801a8e3eade3d25428496f5dff37d2d108f123ddd3be6b7"
Oct 31 01:20:18.698517 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:20:18.698500    4284 scope.go:122] "RemoveContainer" containerID="780532a506a11aedc801a8e3eade3d25428496f5dff37d2d108f123ddd3be6b7"
Oct 31 01:20:18.698785 ip-10-0-132-173 kubenswrapper[4284]: E1031 01:20:18.698765    4284 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"780532a506a11aedc801a8e3eade3d25428496f5dff37d2d108f123ddd3be6b7\": container with ID starting with 780532a506a11aedc801a8e3eade3d25428496f5dff37d2d108f123ddd3be6b7 not found: ID does not exist" containerID="780532a506a11aedc801a8e3eade3d25428496f5dff37d2d108f123ddd3be6b7"
Oct 31 01:20:18.698866 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:20:18.698794    4284 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"780532a506a11aedc801a8e3eade3d25428496f5dff37d2d108f123ddd3be6b7"} err="failed to get container status \"780532a506a11aedc801a8e3eade3d25428496f5dff37d2d108f123ddd3be6b7\": rpc error: code = NotFound desc = could not find container \"780532a506a11aedc801a8e3eade3d25428496f5dff37d2d108f123ddd3be6b7\": container with ID starting with 780532a506a11aedc801a8e3eade3d25428496f5dff37d2d108f123ddd3be6b7 not found: ID does not exist"
Oct 31 01:20:19.971198 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:20:19.971166    4284 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="bd590161-7df6-4088-a3f3-9d18cde8eca9" path="/var/lib/kubelet/pods/bd590161-7df6-4088-a3f3-9d18cde8eca9/volumes"
Oct 31 01:20:20.587810 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:20:20.587772    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:20:21.543674 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:20:21.543641    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:20:22.269467 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:20:22.269433    4284 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-extmod-a-eb68dd-openshift-default-98b48c6cc-gfz48"]
Oct 31 01:20:22.269946 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:20:22.269934    4284 cpu_manager.go:414] "containerMap: removing container" podUID="bd590161-7df6-4088-a3f3-9d18cde8eca9" containerName="test-curl-7292-123d75"
Oct 31 01:20:22.270009 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:20:22.269949    4284 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="bd590161-7df6-4088-a3f3-9d18cde8eca9" containerName="test-curl-7292-123d75"
Oct 31 01:20:22.270070 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:20:22.270061    4284 memory_manager.go:365] "RemoveStaleState removing state" podUID="bd590161-7df6-4088-a3f3-9d18cde8eca9" containerName="test-curl-7292-123d75"
Oct 31 01:20:22.274755 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:20:22.274735    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-a-eb68dd-openshift-default-98b48c6cc-gfz48"
Oct 31 01:20:22.325093 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:20:22.325052    4284 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-extmod-a-eb68dd-openshift-default-dockercfg-9bqtn\""
Oct 31 01:20:22.353745 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:20:22.353711    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1dde1271-7b40-4dc0-97ca-c1be6b89c477-istio-podinfo\") pod \"e2e-extmod-a-eb68dd-openshift-default-98b48c6cc-gfz48\" (UID: \"1dde1271-7b40-4dc0-97ca-c1be6b89c477\") " pod="openshift-ingress/e2e-extmod-a-eb68dd-openshift-default-98b48c6cc-gfz48"
Oct 31 01:20:22.353913 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:20:22.353760    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1dde1271-7b40-4dc0-97ca-c1be6b89c477-istio-envoy\") pod \"e2e-extmod-a-eb68dd-openshift-default-98b48c6cc-gfz48\" (UID: \"1dde1271-7b40-4dc0-97ca-c1be6b89c477\") " pod="openshift-ingress/e2e-extmod-a-eb68dd-openshift-default-98b48c6cc-gfz48"
Oct 31 01:20:22.353913 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:20:22.353868    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1dde1271-7b40-4dc0-97ca-c1be6b89c477-istio-token\") pod \"e2e-extmod-a-eb68dd-openshift-default-98b48c6cc-gfz48\" (UID: \"1dde1271-7b40-4dc0-97ca-c1be6b89c477\") " pod="openshift-ingress/e2e-extmod-a-eb68dd-openshift-default-98b48c6cc-gfz48"
Oct 31 01:20:22.353913 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:20:22.353901    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1dde1271-7b40-4dc0-97ca-c1be6b89c477-credential-socket\") pod \"e2e-extmod-a-eb68dd-openshift-default-98b48c6cc-gfz48\" (UID: \"1dde1271-7b40-4dc0-97ca-c1be6b89c477\") " pod="openshift-ingress/e2e-extmod-a-eb68dd-openshift-default-98b48c6cc-gfz48"
Oct 31 01:20:22.354091 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:20:22.353924    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1dde1271-7b40-4dc0-97ca-c1be6b89c477-istio-data\") pod \"e2e-extmod-a-eb68dd-openshift-default-98b48c6cc-gfz48\" (UID: \"1dde1271-7b40-4dc0-97ca-c1be6b89c477\") " pod="openshift-ingress/e2e-extmod-a-eb68dd-openshift-default-98b48c6cc-gfz48"
Oct 31 01:20:22.354091 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:20:22.353976    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xb7gk\" (UniqueName: \"kubernetes.io/projected/1dde1271-7b40-4dc0-97ca-c1be6b89c477-kube-api-access-xb7gk\") pod \"e2e-extmod-a-eb68dd-openshift-default-98b48c6cc-gfz48\" (UID: \"1dde1271-7b40-4dc0-97ca-c1be6b89c477\") " pod="openshift-ingress/e2e-extmod-a-eb68dd-openshift-default-98b48c6cc-gfz48"
Oct 31 01:20:22.354091 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:20:22.354061    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1dde1271-7b40-4dc0-97ca-c1be6b89c477-istiod-ca-cert\") pod \"e2e-extmod-a-eb68dd-openshift-default-98b48c6cc-gfz48\" (UID: \"1dde1271-7b40-4dc0-97ca-c1be6b89c477\") " pod="openshift-ingress/e2e-extmod-a-eb68dd-openshift-default-98b48c6cc-gfz48"
Oct 31 01:20:22.354258 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:20:22.354154    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1dde1271-7b40-4dc0-97ca-c1be6b89c477-workload-socket\") pod \"e2e-extmod-a-eb68dd-openshift-default-98b48c6cc-gfz48\" (UID: \"1dde1271-7b40-4dc0-97ca-c1be6b89c477\") " pod="openshift-ingress/e2e-extmod-a-eb68dd-openshift-default-98b48c6cc-gfz48"
Oct 31 01:20:22.354258 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:20:22.354254    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1dde1271-7b40-4dc0-97ca-c1be6b89c477-workload-certs\") pod \"e2e-extmod-a-eb68dd-openshift-default-98b48c6cc-gfz48\" (UID: \"1dde1271-7b40-4dc0-97ca-c1be6b89c477\") " pod="openshift-ingress/e2e-extmod-a-eb68dd-openshift-default-98b48c6cc-gfz48"
Oct 31 01:20:22.359582 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:20:22.359555    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:20:22.375786 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:20:22.375759    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-extmod-a-eb68dd-openshift-default-98b48c6cc-gfz48"]
Oct 31 01:20:22.454999 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:20:22.454969    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1dde1271-7b40-4dc0-97ca-c1be6b89c477-istio-token\") pod \"e2e-extmod-a-eb68dd-openshift-default-98b48c6cc-gfz48\" (UID: \"1dde1271-7b40-4dc0-97ca-c1be6b89c477\") " pod="openshift-ingress/e2e-extmod-a-eb68dd-openshift-default-98b48c6cc-gfz48"
Oct 31 01:20:22.455248 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:20:22.455015    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1dde1271-7b40-4dc0-97ca-c1be6b89c477-credential-socket\") pod \"e2e-extmod-a-eb68dd-openshift-default-98b48c6cc-gfz48\" (UID: \"1dde1271-7b40-4dc0-97ca-c1be6b89c477\") " pod="openshift-ingress/e2e-extmod-a-eb68dd-openshift-default-98b48c6cc-gfz48"
Oct 31 01:20:22.455248 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:20:22.455044    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1dde1271-7b40-4dc0-97ca-c1be6b89c477-istio-data\") pod \"e2e-extmod-a-eb68dd-openshift-default-98b48c6cc-gfz48\" (UID: \"1dde1271-7b40-4dc0-97ca-c1be6b89c477\") " pod="openshift-ingress/e2e-extmod-a-eb68dd-openshift-default-98b48c6cc-gfz48"
Oct 31 01:20:22.455248 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:20:22.455070    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xb7gk\" (UniqueName: \"kubernetes.io/projected/1dde1271-7b40-4dc0-97ca-c1be6b89c477-kube-api-access-xb7gk\") pod \"e2e-extmod-a-eb68dd-openshift-default-98b48c6cc-gfz48\" (UID: \"1dde1271-7b40-4dc0-97ca-c1be6b89c477\") " pod="openshift-ingress/e2e-extmod-a-eb68dd-openshift-default-98b48c6cc-gfz48"
Oct 31 01:20:22.455471 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:20:22.455365    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1dde1271-7b40-4dc0-97ca-c1be6b89c477-istiod-ca-cert\") pod \"e2e-extmod-a-eb68dd-openshift-default-98b48c6cc-gfz48\" (UID: \"1dde1271-7b40-4dc0-97ca-c1be6b89c477\") " pod="openshift-ingress/e2e-extmod-a-eb68dd-openshift-default-98b48c6cc-gfz48"
Oct 31 01:20:22.455471 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:20:22.455424    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1dde1271-7b40-4dc0-97ca-c1be6b89c477-workload-socket\") pod \"e2e-extmod-a-eb68dd-openshift-default-98b48c6cc-gfz48\" (UID: \"1dde1271-7b40-4dc0-97ca-c1be6b89c477\") " pod="openshift-ingress/e2e-extmod-a-eb68dd-openshift-default-98b48c6cc-gfz48"
Oct 31 01:20:22.455598 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:20:22.455486    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1dde1271-7b40-4dc0-97ca-c1be6b89c477-workload-certs\") pod \"e2e-extmod-a-eb68dd-openshift-default-98b48c6cc-gfz48\" (UID: \"1dde1271-7b40-4dc0-97ca-c1be6b89c477\") " pod="openshift-ingress/e2e-extmod-a-eb68dd-openshift-default-98b48c6cc-gfz48"
Oct 31 01:20:22.455598 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:20:22.455563    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1dde1271-7b40-4dc0-97ca-c1be6b89c477-credential-socket\") pod \"e2e-extmod-a-eb68dd-openshift-default-98b48c6cc-gfz48\" (UID: \"1dde1271-7b40-4dc0-97ca-c1be6b89c477\") " pod="openshift-ingress/e2e-extmod-a-eb68dd-openshift-default-98b48c6cc-gfz48"
Oct 31 01:20:22.455707 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:20:22.455599    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1dde1271-7b40-4dc0-97ca-c1be6b89c477-istio-podinfo\") pod \"e2e-extmod-a-eb68dd-openshift-default-98b48c6cc-gfz48\" (UID: \"1dde1271-7b40-4dc0-97ca-c1be6b89c477\") " pod="openshift-ingress/e2e-extmod-a-eb68dd-openshift-default-98b48c6cc-gfz48"
Oct 31 01:20:22.455707 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:20:22.455644    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1dde1271-7b40-4dc0-97ca-c1be6b89c477-istio-envoy\") pod \"e2e-extmod-a-eb68dd-openshift-default-98b48c6cc-gfz48\" (UID: \"1dde1271-7b40-4dc0-97ca-c1be6b89c477\") " pod="openshift-ingress/e2e-extmod-a-eb68dd-openshift-default-98b48c6cc-gfz48"
Oct 31 01:20:22.455862 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:20:22.455840    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1dde1271-7b40-4dc0-97ca-c1be6b89c477-istio-data\") pod \"e2e-extmod-a-eb68dd-openshift-default-98b48c6cc-gfz48\" (UID: \"1dde1271-7b40-4dc0-97ca-c1be6b89c477\") " pod="openshift-ingress/e2e-extmod-a-eb68dd-openshift-default-98b48c6cc-gfz48"
Oct 31 01:20:22.455945 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:20:22.455924    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1dde1271-7b40-4dc0-97ca-c1be6b89c477-workload-socket\") pod \"e2e-extmod-a-eb68dd-openshift-default-98b48c6cc-gfz48\" (UID: \"1dde1271-7b40-4dc0-97ca-c1be6b89c477\") " pod="openshift-ingress/e2e-extmod-a-eb68dd-openshift-default-98b48c6cc-gfz48"
Oct 31 01:20:22.456278 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:20:22.456254    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1dde1271-7b40-4dc0-97ca-c1be6b89c477-workload-certs\") pod \"e2e-extmod-a-eb68dd-openshift-default-98b48c6cc-gfz48\" (UID: \"1dde1271-7b40-4dc0-97ca-c1be6b89c477\") " pod="openshift-ingress/e2e-extmod-a-eb68dd-openshift-default-98b48c6cc-gfz48"
Oct 31 01:20:22.456592 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:20:22.456571    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1dde1271-7b40-4dc0-97ca-c1be6b89c477-istiod-ca-cert\") pod \"e2e-extmod-a-eb68dd-openshift-default-98b48c6cc-gfz48\" (UID: \"1dde1271-7b40-4dc0-97ca-c1be6b89c477\") " pod="openshift-ingress/e2e-extmod-a-eb68dd-openshift-default-98b48c6cc-gfz48"
Oct 31 01:20:22.458783 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:20:22.458758    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1dde1271-7b40-4dc0-97ca-c1be6b89c477-istio-envoy\") pod \"e2e-extmod-a-eb68dd-openshift-default-98b48c6cc-gfz48\" (UID: \"1dde1271-7b40-4dc0-97ca-c1be6b89c477\") " pod="openshift-ingress/e2e-extmod-a-eb68dd-openshift-default-98b48c6cc-gfz48"
Oct 31 01:20:22.459021 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:20:22.458999    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1dde1271-7b40-4dc0-97ca-c1be6b89c477-istio-podinfo\") pod \"e2e-extmod-a-eb68dd-openshift-default-98b48c6cc-gfz48\" (UID: \"1dde1271-7b40-4dc0-97ca-c1be6b89c477\") " pod="openshift-ingress/e2e-extmod-a-eb68dd-openshift-default-98b48c6cc-gfz48"
Oct 31 01:20:22.514490 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:20:22.514460    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1dde1271-7b40-4dc0-97ca-c1be6b89c477-istio-token\") pod \"e2e-extmod-a-eb68dd-openshift-default-98b48c6cc-gfz48\" (UID: \"1dde1271-7b40-4dc0-97ca-c1be6b89c477\") " pod="openshift-ingress/e2e-extmod-a-eb68dd-openshift-default-98b48c6cc-gfz48"
Oct 31 01:20:22.514937 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:20:22.514919    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xb7gk\" (UniqueName: \"kubernetes.io/projected/1dde1271-7b40-4dc0-97ca-c1be6b89c477-kube-api-access-xb7gk\") pod \"e2e-extmod-a-eb68dd-openshift-default-98b48c6cc-gfz48\" (UID: \"1dde1271-7b40-4dc0-97ca-c1be6b89c477\") " pod="openshift-ingress/e2e-extmod-a-eb68dd-openshift-default-98b48c6cc-gfz48"
Oct 31 01:20:22.565294 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:20:22.563577    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:20:22.589132 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:20:22.589094    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-a-eb68dd-openshift-default-98b48c6cc-gfz48"
Oct 31 01:20:23.091633 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:20:23.091606    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-extmod-a-eb68dd-openshift-default-98b48c6cc-gfz48"]
Oct 31 01:20:23.092224 ip-10-0-132-173 kubenswrapper[4284]: W1031 01:20:23.092180    4284 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1dde1271_7b40_4dc0_97ca_c1be6b89c477.slice/crio-cbf89d3915a2b260d21d6e2a45430b9a01d6bd1bbb49c144b2e409de1929ce0a WatchSource:0}: Error finding container cbf89d3915a2b260d21d6e2a45430b9a01d6bd1bbb49c144b2e409de1929ce0a: Status 404 returned error can't find the container with id cbf89d3915a2b260d21d6e2a45430b9a01d6bd1bbb49c144b2e409de1929ce0a
Oct 31 01:20:23.094694 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:20:23.094661    4284 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 01:20:23.094778 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:20:23.094726    4284 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 01:20:23.094778 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:20:23.094756    4284 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 01:20:23.722187 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:20:23.722148    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-eb68dd-openshift-default-98b48c6cc-gfz48" event={"ID":"1dde1271-7b40-4dc0-97ca-c1be6b89c477","Type":"ContainerStarted","Data":"bbab3f04b06e5edb3eaae4cb48bd2bef652990c808bf7fae3bbbf37a9beabcb1"}
Oct 31 01:20:23.722661 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:20:23.722201    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-eb68dd-openshift-default-98b48c6cc-gfz48" event={"ID":"1dde1271-7b40-4dc0-97ca-c1be6b89c477","Type":"ContainerStarted","Data":"cbf89d3915a2b260d21d6e2a45430b9a01d6bd1bbb49c144b2e409de1929ce0a"}
Oct 31 01:20:23.847796 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:20:23.847739    4284 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-extmod-a-eb68dd-openshift-default-98b48c6cc-gfz48" podStartSLOduration=1.847724218 podStartE2EDuration="1.847724218s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 01:20:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 01:20:23.845269407 +0000 UTC m=+3137.990276324" watchObservedRunningTime="2026-10-31 01:20:23.847724218 +0000 UTC m=+3137.992731186"
Oct 31 01:20:24.590058 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:20:24.590023    4284 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-extmod-a-eb68dd-openshift-default-98b48c6cc-gfz48"
Oct 31 01:20:24.596110 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:20:24.596083    4284 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-extmod-a-eb68dd-openshift-default-98b48c6cc-gfz48"
Oct 31 01:20:24.729116 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:20:24.729078    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-extmod-a-eb68dd-openshift-default-98b48c6cc-gfz48"
Oct 31 01:20:24.730104 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:20:24.730088    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-extmod-a-eb68dd-openshift-default-98b48c6cc-gfz48"
Oct 31 01:20:25.421535 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:20:25.421489    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:20:26.738143 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:20:26.738105    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:20:26.954973 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:20:26.954933    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:20:27.114350 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:20:27.114250    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:20:29.410717 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:20:29.410684    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:20:29.780065 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:20:29.779979    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:20:29.926834 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:20:29.926795    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:20:30.077422 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:20:30.076962    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:20:30.460622 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:20:30.460587    4284 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-a-eb68dd-79b7d8c769-xr7cl"]
Oct 31 01:20:30.465653 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:20:30.465634    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-extmod-a-eb68dd-79b7d8c769-xr7cl"
Oct 31 01:20:30.474160 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:20:30.473974    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:20:30.486529 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:20:30.486506    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b96b200a-f8de-4419-8232-061be10ada45-plugins-config-volume\") pod \"payload-processing-e2e-extmod-a-eb68dd-79b7d8c769-xr7cl\" (UID: \"b96b200a-f8de-4419-8232-061be10ada45\") " pod="openshift-ingress/payload-processing-e2e-extmod-a-eb68dd-79b7d8c769-xr7cl"
Oct 31 01:20:30.486662 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:20:30.486535    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4q2km\" (UniqueName: \"kubernetes.io/projected/b96b200a-f8de-4419-8232-061be10ada45-kube-api-access-4q2km\") pod \"payload-processing-e2e-extmod-a-eb68dd-79b7d8c769-xr7cl\" (UID: \"b96b200a-f8de-4419-8232-061be10ada45\") " pod="openshift-ingress/payload-processing-e2e-extmod-a-eb68dd-79b7d8c769-xr7cl"
Oct 31 01:20:30.528046 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:20:30.528018    4284 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-extmod-a-eb68dd\""
Oct 31 01:20:30.531108 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:20:30.531085    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-a-eb68dd-79b7d8c769-xr7cl"]
Oct 31 01:20:30.554181 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:20:30.554154    4284 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-extmod-a-eb68dd-dockercfg-fg82r\""
Oct 31 01:20:30.607975 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:20:30.607771    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b96b200a-f8de-4419-8232-061be10ada45-plugins-config-volume\") pod \"payload-processing-e2e-extmod-a-eb68dd-79b7d8c769-xr7cl\" (UID: \"b96b200a-f8de-4419-8232-061be10ada45\") " pod="openshift-ingress/payload-processing-e2e-extmod-a-eb68dd-79b7d8c769-xr7cl"
Oct 31 01:20:30.607975 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:20:30.607830    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4q2km\" (UniqueName: \"kubernetes.io/projected/b96b200a-f8de-4419-8232-061be10ada45-kube-api-access-4q2km\") pod \"payload-processing-e2e-extmod-a-eb68dd-79b7d8c769-xr7cl\" (UID: \"b96b200a-f8de-4419-8232-061be10ada45\") " pod="openshift-ingress/payload-processing-e2e-extmod-a-eb68dd-79b7d8c769-xr7cl"
Oct 31 01:20:30.609656 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:20:30.609300    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b96b200a-f8de-4419-8232-061be10ada45-plugins-config-volume\") pod \"payload-processing-e2e-extmod-a-eb68dd-79b7d8c769-xr7cl\" (UID: \"b96b200a-f8de-4419-8232-061be10ada45\") " pod="openshift-ingress/payload-processing-e2e-extmod-a-eb68dd-79b7d8c769-xr7cl"
Oct 31 01:20:30.620026 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:20:30.619999    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:20:30.667184 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:20:30.667153    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4q2km\" (UniqueName: \"kubernetes.io/projected/b96b200a-f8de-4419-8232-061be10ada45-kube-api-access-4q2km\") pod \"payload-processing-e2e-extmod-a-eb68dd-79b7d8c769-xr7cl\" (UID: \"b96b200a-f8de-4419-8232-061be10ada45\") " pod="openshift-ingress/payload-processing-e2e-extmod-a-eb68dd-79b7d8c769-xr7cl"
Oct 31 01:20:30.780613 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:20:30.780525    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-extmod-a-eb68dd-79b7d8c769-xr7cl"
Oct 31 01:20:31.061630 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:20:31.061525    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:20:31.164141 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:20:31.164080    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-a-eb68dd-79b7d8c769-xr7cl"]
Oct 31 01:20:31.442296 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:20:31.442260    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:20:31.771664 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:20:31.771570    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-extmod-a-eb68dd-79b7d8c769-xr7cl" event={"ID":"b96b200a-f8de-4419-8232-061be10ada45","Type":"ContainerStarted","Data":"e38b8a3fa7a0cf8b0df5b76811063dd4a0b471177ce0dbae6e0ca0c5e8086227"}
Oct 31 01:20:31.771664 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:20:31.771648    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-extmod-a-eb68dd-79b7d8c769-xr7cl" event={"ID":"b96b200a-f8de-4419-8232-061be10ada45","Type":"ContainerStarted","Data":"e7c11e22d3deced77b411268b320a712fd8e94c5c335d89119a8824782eae369"}
Oct 31 01:20:31.772128 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:20:31.771670    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-extmod-a-eb68dd-79b7d8c769-xr7cl"
Oct 31 01:20:31.886134 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:20:31.886072    4284 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-extmod-a-eb68dd-79b7d8c769-xr7cl" podStartSLOduration=1.886058188 podStartE2EDuration="1.886058188s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 01:20:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 01:20:31.882891375 +0000 UTC m=+3146.027898291" watchObservedRunningTime="2026-10-31 01:20:31.886058188 +0000 UTC m=+3146.031065137"
Oct 31 01:20:33.009563 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:20:33.009526    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:20:33.155360 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:20:33.155321    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:20:33.360410 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:20:33.360321    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:20:35.271174 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:20:35.271136    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:20:40.126770 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:20:40.126733    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:20:42.780387 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:20:42.780351    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-extmod-a-eb68dd-79b7d8c769-xr7cl"
Oct 31 01:20:44.889074 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:20:44.889044    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:20:47.016517 ip-10-0-132-173 kubenswrapper[4284]: E1031 01:20:47.016481    4284 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fd8sp" containerName="extractor"
Oct 31 01:20:49.790002 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:20:49.789962    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:20:54.603899 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:20:54.603863    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:20:59.301835 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:20:59.301792    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:21:04.165870 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:21:04.165789    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:21:06.666845 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:21:06.666815    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:21:08.891795 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:21:08.891747    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:21:13.686129 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:21:13.686087    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:21:17.014940 ip-10-0-132-173 kubenswrapper[4284]: E1031 01:21:17.014851    4284 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fd8sp" containerName="extractor"
Oct 31 01:21:17.366294 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:21:17.353178    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:21:18.499981 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:21:18.499938    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:21:23.277778 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:21:23.277738    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:21:28.083805 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:21:28.083750    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:21:32.777903 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:21:32.777858    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:21:32.931345 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:21:32.931307    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:21:33.079956 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:21:33.079859    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:21:37.688469 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:21:37.688436    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:21:42.568863 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:21:42.568828    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:21:44.448356 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:21:44.448326    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w1-ba34eb/auth-test-model-ba34eb-kserve-55f46b9477-rd7dt"]
Oct 31 01:21:44.467729 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:21:44.467697    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:21:45.376364 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:21:45.376330    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:21:45.526604 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:21:45.526567    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:21:45.911165 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:21:45.911126    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:21:46.093516 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:21:46.093482    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:21:47.016921 ip-10-0-132-173 kubenswrapper[4284]: E1031 01:21:47.016888    4284 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fd8sp" containerName="extractor"
Oct 31 01:21:47.493170 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:21:47.493129    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:21:50.144683 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:21:50.144628    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w1-3be6fa/auth-test-model-3be6fa-kserve-6f5d4f988f-zc2bc"]
Oct 31 01:21:50.153087 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:21:50.153049    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:21:51.089964 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:21:51.089924    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:21:51.872495 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:21:51.872445    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:21:52.276961 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:21:52.276923    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:21:52.736961 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:21:52.736927    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:21:53.054305 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:21:53.054164    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:21:57.165819 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:21:57.165785    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:21:57.376782 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:21:57.376737    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:21:57.516708 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:21:57.516608    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:21:59.469443 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:21:59.469400    4284 kuberuntime_container.go:898] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-w1-ba34eb/auth-test-model-ba34eb-kserve-55f46b9477-rd7dt" podUID="4192aa2f-a5a8-44fe-9ff6-6bdc48478c39" containerName="main" containerID="cri-o://a33fcc9a8164acef4386d77c5b98358ccdd1f0ff1ec945b0b49be6b00389fe00" gracePeriod=45
Oct 31 01:21:59.730117 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:21:59.730050    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w1-ba34eb/auth-test-model-ba34eb-kserve-55f46b9477-rd7dt"
Oct 31 01:21:59.805898 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:21:59.805866    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4192aa2f-a5a8-44fe-9ff6-6bdc48478c39-dshm\" (UniqueName: \"kubernetes.io/empty-dir/4192aa2f-a5a8-44fe-9ff6-6bdc48478c39-dshm\") pod \"4192aa2f-a5a8-44fe-9ff6-6bdc48478c39\" (UID: \"4192aa2f-a5a8-44fe-9ff6-6bdc48478c39\") "
Oct 31 01:21:59.806103 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:21:59.805942    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4192aa2f-a5a8-44fe-9ff6-6bdc48478c39-home\" (UniqueName: \"kubernetes.io/empty-dir/4192aa2f-a5a8-44fe-9ff6-6bdc48478c39-home\") pod \"4192aa2f-a5a8-44fe-9ff6-6bdc48478c39\" (UID: \"4192aa2f-a5a8-44fe-9ff6-6bdc48478c39\") "
Oct 31 01:21:59.806103 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:21:59.805989    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/4192aa2f-a5a8-44fe-9ff6-6bdc48478c39-tls-certs\" (UniqueName: \"kubernetes.io/secret/4192aa2f-a5a8-44fe-9ff6-6bdc48478c39-tls-certs\") pod \"4192aa2f-a5a8-44fe-9ff6-6bdc48478c39\" (UID: \"4192aa2f-a5a8-44fe-9ff6-6bdc48478c39\") "
Oct 31 01:21:59.806103 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:21:59.806029    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4192aa2f-a5a8-44fe-9ff6-6bdc48478c39-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4192aa2f-a5a8-44fe-9ff6-6bdc48478c39-tmp-dir\") pod \"4192aa2f-a5a8-44fe-9ff6-6bdc48478c39\" (UID: \"4192aa2f-a5a8-44fe-9ff6-6bdc48478c39\") "
Oct 31 01:21:59.806310 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:21:59.806131    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4192aa2f-a5a8-44fe-9ff6-6bdc48478c39-kube-api-access-j7f66\" (UniqueName: \"kubernetes.io/projected/4192aa2f-a5a8-44fe-9ff6-6bdc48478c39-kube-api-access-j7f66\") pod \"4192aa2f-a5a8-44fe-9ff6-6bdc48478c39\" (UID: \"4192aa2f-a5a8-44fe-9ff6-6bdc48478c39\") "
Oct 31 01:21:59.806310 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:21:59.806196    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4192aa2f-a5a8-44fe-9ff6-6bdc48478c39-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4192aa2f-a5a8-44fe-9ff6-6bdc48478c39-model-cache\") pod \"4192aa2f-a5a8-44fe-9ff6-6bdc48478c39\" (UID: \"4192aa2f-a5a8-44fe-9ff6-6bdc48478c39\") "
Oct 31 01:21:59.806406 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:21:59.806305    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4192aa2f-a5a8-44fe-9ff6-6bdc48478c39-home" pod "4192aa2f-a5a8-44fe-9ff6-6bdc48478c39" (UID: "4192aa2f-a5a8-44fe-9ff6-6bdc48478c39"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:21:59.806406 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:21:59.806379    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4192aa2f-a5a8-44fe-9ff6-6bdc48478c39-tmp-dir" pod "4192aa2f-a5a8-44fe-9ff6-6bdc48478c39" (UID: "4192aa2f-a5a8-44fe-9ff6-6bdc48478c39"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:21:59.806551 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:21:59.806516    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4192aa2f-a5a8-44fe-9ff6-6bdc48478c39-model-cache" pod "4192aa2f-a5a8-44fe-9ff6-6bdc48478c39" (UID: "4192aa2f-a5a8-44fe-9ff6-6bdc48478c39"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:21:59.806747 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:21:59.806730    4284 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4192aa2f-a5a8-44fe-9ff6-6bdc48478c39-model-cache\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:21:59.806790 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:21:59.806753    4284 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4192aa2f-a5a8-44fe-9ff6-6bdc48478c39-home\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:21:59.806790 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:21:59.806765    4284 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4192aa2f-a5a8-44fe-9ff6-6bdc48478c39-tmp-dir\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:21:59.808574 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:21:59.808540    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4192aa2f-a5a8-44fe-9ff6-6bdc48478c39-dshm" pod "4192aa2f-a5a8-44fe-9ff6-6bdc48478c39" (UID: "4192aa2f-a5a8-44fe-9ff6-6bdc48478c39"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:21:59.808713 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:21:59.808617    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4192aa2f-a5a8-44fe-9ff6-6bdc48478c39-tls-certs" pod "4192aa2f-a5a8-44fe-9ff6-6bdc48478c39" (UID: "4192aa2f-a5a8-44fe-9ff6-6bdc48478c39"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 01:21:59.808780 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:21:59.808725    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4192aa2f-a5a8-44fe-9ff6-6bdc48478c39-kube-api-access-j7f66" pod "4192aa2f-a5a8-44fe-9ff6-6bdc48478c39" (UID: "4192aa2f-a5a8-44fe-9ff6-6bdc48478c39"). InnerVolumeSpecName "kube-api-access-j7f66". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:21:59.908201 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:21:59.908167    4284 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j7f66\" (UniqueName: \"kubernetes.io/projected/4192aa2f-a5a8-44fe-9ff6-6bdc48478c39-kube-api-access-j7f66\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:21:59.908201 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:21:59.908202    4284 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4192aa2f-a5a8-44fe-9ff6-6bdc48478c39-dshm\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:21:59.908201 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:21:59.908248    4284 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4192aa2f-a5a8-44fe-9ff6-6bdc48478c39-tls-certs\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:22:00.366512 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:22:00.366480    4284 generic.go:356] "Generic (PLEG): container finished" podID="4192aa2f-a5a8-44fe-9ff6-6bdc48478c39" containerID="a33fcc9a8164acef4386d77c5b98358ccdd1f0ff1ec945b0b49be6b00389fe00" exitCode=0
Oct 31 01:22:00.366716 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:22:00.366524    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w1-ba34eb/auth-test-model-ba34eb-kserve-55f46b9477-rd7dt"
Oct 31 01:22:00.366716 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:22:00.366545    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w1-ba34eb/auth-test-model-ba34eb-kserve-55f46b9477-rd7dt" event={"ID":"4192aa2f-a5a8-44fe-9ff6-6bdc48478c39","Type":"ContainerDied","Data":"a33fcc9a8164acef4386d77c5b98358ccdd1f0ff1ec945b0b49be6b00389fe00"}
Oct 31 01:22:00.366716 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:22:00.366578    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w1-ba34eb/auth-test-model-ba34eb-kserve-55f46b9477-rd7dt" event={"ID":"4192aa2f-a5a8-44fe-9ff6-6bdc48478c39","Type":"ContainerDied","Data":"a53df7541d6a3eb1470ab2bf410f43910f178f7edb2ecc30195c2587c31f542a"}
Oct 31 01:22:00.366716 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:22:00.366594    4284 scope.go:122] "RemoveContainer" containerID="a33fcc9a8164acef4386d77c5b98358ccdd1f0ff1ec945b0b49be6b00389fe00"
Oct 31 01:22:00.376453 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:22:00.376428    4284 scope.go:122] "RemoveContainer" containerID="a33fcc9a8164acef4386d77c5b98358ccdd1f0ff1ec945b0b49be6b00389fe00"
Oct 31 01:22:00.376730 ip-10-0-132-173 kubenswrapper[4284]: E1031 01:22:00.376711    4284 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a33fcc9a8164acef4386d77c5b98358ccdd1f0ff1ec945b0b49be6b00389fe00\": container with ID starting with a33fcc9a8164acef4386d77c5b98358ccdd1f0ff1ec945b0b49be6b00389fe00 not found: ID does not exist" containerID="a33fcc9a8164acef4386d77c5b98358ccdd1f0ff1ec945b0b49be6b00389fe00"
Oct 31 01:22:00.376780 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:22:00.376738    4284 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a33fcc9a8164acef4386d77c5b98358ccdd1f0ff1ec945b0b49be6b00389fe00"} err="failed to get container status \"a33fcc9a8164acef4386d77c5b98358ccdd1f0ff1ec945b0b49be6b00389fe00\": rpc error: code = NotFound desc = could not find container \"a33fcc9a8164acef4386d77c5b98358ccdd1f0ff1ec945b0b49be6b00389fe00\": container with ID starting with a33fcc9a8164acef4386d77c5b98358ccdd1f0ff1ec945b0b49be6b00389fe00 not found: ID does not exist"
Oct 31 01:22:00.466238 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:22:00.466163    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:22:00.526966 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:22:00.526921    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w1-ba34eb/auth-test-model-ba34eb-kserve-55f46b9477-rd7dt"]
Oct 31 01:22:00.531715 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:22:00.531687    4284 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-w1-ba34eb/auth-test-model-ba34eb-kserve-55f46b9477-rd7dt"]
Oct 31 01:22:01.966672 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:22:01.966637    4284 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="4192aa2f-a5a8-44fe-9ff6-6bdc48478c39" path="/var/lib/kubelet/pods/4192aa2f-a5a8-44fe-9ff6-6bdc48478c39/volumes"
Oct 31 01:22:02.361942 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:22:02.361848    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:22:03.450998 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:22:03.450965    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:22:03.643987 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:22:03.642923    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:22:03.880290 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:22:03.876834    4284 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-a-3aca3f-67994c4bc5-hwmd2"]
Oct 31 01:22:03.880290 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:22:03.877667    4284 cpu_manager.go:414] "containerMap: removing container" podUID="4192aa2f-a5a8-44fe-9ff6-6bdc48478c39" containerName="main"
Oct 31 01:22:03.880290 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:22:03.877686    4284 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4192aa2f-a5a8-44fe-9ff6-6bdc48478c39" containerName="main"
Oct 31 01:22:03.880290 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:22:03.877876    4284 memory_manager.go:365] "RemoveStaleState removing state" podUID="4192aa2f-a5a8-44fe-9ff6-6bdc48478c39" containerName="main"
Oct 31 01:22:03.895174 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:22:03.895138    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-3aca3f-67994c4bc5-hwmd2"
Oct 31 01:22:03.975838 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:22:03.975807    4284 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-extmod-a-3aca3f\""
Oct 31 01:22:04.028593 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:22:04.028523    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-a-3aca3f-67994c4bc5-hwmd2"]
Oct 31 01:22:04.028903 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:22:04.028874    4284 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-a-3aca3f-554dd84857-59rv6"]
Oct 31 01:22:04.055753 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:22:04.055706    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-extmod-a-3aca3f-554dd84857-59rv6"
Oct 31 01:22:04.070724 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:22:04.070688    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/16fc298a-4f7c-4efd-ac33-5ac97b7d3275-plugins-config-volume\") pod \"payload-pre-processing-e2e-extmod-a-3aca3f-67994c4bc5-hwmd2\" (UID: \"16fc298a-4f7c-4efd-ac33-5ac97b7d3275\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-3aca3f-67994c4bc5-hwmd2"
Oct 31 01:22:04.071126 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:22:04.071102    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2wwhj\" (UniqueName: \"kubernetes.io/projected/16fc298a-4f7c-4efd-ac33-5ac97b7d3275-kube-api-access-2wwhj\") pod \"payload-pre-processing-e2e-extmod-a-3aca3f-67994c4bc5-hwmd2\" (UID: \"16fc298a-4f7c-4efd-ac33-5ac97b7d3275\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-3aca3f-67994c4bc5-hwmd2"
Oct 31 01:22:04.073159 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:22:04.073134    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-a-3aca3f-554dd84857-59rv6"]
Oct 31 01:22:04.120299 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:22:04.109204    4284 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-extmod-a-3aca3f-dockercfg-kfvqd\""
Oct 31 01:22:04.172712 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:22:04.172669    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2wwhj\" (UniqueName: \"kubernetes.io/projected/16fc298a-4f7c-4efd-ac33-5ac97b7d3275-kube-api-access-2wwhj\") pod \"payload-pre-processing-e2e-extmod-a-3aca3f-67994c4bc5-hwmd2\" (UID: \"16fc298a-4f7c-4efd-ac33-5ac97b7d3275\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-3aca3f-67994c4bc5-hwmd2"
Oct 31 01:22:04.172911 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:22:04.172845    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f31cff38-07af-4d6c-83ba-fcc15b830cbc-plugins-config-volume\") pod \"payload-processing-e2e-extmod-a-3aca3f-554dd84857-59rv6\" (UID: \"f31cff38-07af-4d6c-83ba-fcc15b830cbc\") " pod="openshift-ingress/payload-processing-e2e-extmod-a-3aca3f-554dd84857-59rv6"
Oct 31 01:22:04.172911 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:22:04.172901    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-65jlb\" (UniqueName: \"kubernetes.io/projected/f31cff38-07af-4d6c-83ba-fcc15b830cbc-kube-api-access-65jlb\") pod \"payload-processing-e2e-extmod-a-3aca3f-554dd84857-59rv6\" (UID: \"f31cff38-07af-4d6c-83ba-fcc15b830cbc\") " pod="openshift-ingress/payload-processing-e2e-extmod-a-3aca3f-554dd84857-59rv6"
Oct 31 01:22:04.173065 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:22:04.173039    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/16fc298a-4f7c-4efd-ac33-5ac97b7d3275-plugins-config-volume\") pod \"payload-pre-processing-e2e-extmod-a-3aca3f-67994c4bc5-hwmd2\" (UID: \"16fc298a-4f7c-4efd-ac33-5ac97b7d3275\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-3aca3f-67994c4bc5-hwmd2"
Oct 31 01:22:04.173754 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:22:04.173736    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/16fc298a-4f7c-4efd-ac33-5ac97b7d3275-plugins-config-volume\") pod \"payload-pre-processing-e2e-extmod-a-3aca3f-67994c4bc5-hwmd2\" (UID: \"16fc298a-4f7c-4efd-ac33-5ac97b7d3275\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-3aca3f-67994c4bc5-hwmd2"
Oct 31 01:22:04.232557 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:22:04.232516    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2wwhj\" (UniqueName: \"kubernetes.io/projected/16fc298a-4f7c-4efd-ac33-5ac97b7d3275-kube-api-access-2wwhj\") pod \"payload-pre-processing-e2e-extmod-a-3aca3f-67994c4bc5-hwmd2\" (UID: \"16fc298a-4f7c-4efd-ac33-5ac97b7d3275\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-3aca3f-67994c4bc5-hwmd2"
Oct 31 01:22:04.274687 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:22:04.274645    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f31cff38-07af-4d6c-83ba-fcc15b830cbc-plugins-config-volume\") pod \"payload-processing-e2e-extmod-a-3aca3f-554dd84857-59rv6\" (UID: \"f31cff38-07af-4d6c-83ba-fcc15b830cbc\") " pod="openshift-ingress/payload-processing-e2e-extmod-a-3aca3f-554dd84857-59rv6"
Oct 31 01:22:04.274687 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:22:04.274696    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-65jlb\" (UniqueName: \"kubernetes.io/projected/f31cff38-07af-4d6c-83ba-fcc15b830cbc-kube-api-access-65jlb\") pod \"payload-processing-e2e-extmod-a-3aca3f-554dd84857-59rv6\" (UID: \"f31cff38-07af-4d6c-83ba-fcc15b830cbc\") " pod="openshift-ingress/payload-processing-e2e-extmod-a-3aca3f-554dd84857-59rv6"
Oct 31 01:22:04.275470 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:22:04.275448    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f31cff38-07af-4d6c-83ba-fcc15b830cbc-plugins-config-volume\") pod \"payload-processing-e2e-extmod-a-3aca3f-554dd84857-59rv6\" (UID: \"f31cff38-07af-4d6c-83ba-fcc15b830cbc\") " pod="openshift-ingress/payload-processing-e2e-extmod-a-3aca3f-554dd84857-59rv6"
Oct 31 01:22:04.314547 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:22:04.314506    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-3aca3f-67994c4bc5-hwmd2"
Oct 31 01:22:04.334483 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:22:04.334444    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-65jlb\" (UniqueName: \"kubernetes.io/projected/f31cff38-07af-4d6c-83ba-fcc15b830cbc-kube-api-access-65jlb\") pod \"payload-processing-e2e-extmod-a-3aca3f-554dd84857-59rv6\" (UID: \"f31cff38-07af-4d6c-83ba-fcc15b830cbc\") " pod="openshift-ingress/payload-processing-e2e-extmod-a-3aca3f-554dd84857-59rv6"
Oct 31 01:22:04.378686 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:22:04.378649    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-extmod-a-3aca3f-554dd84857-59rv6"
Oct 31 01:22:04.714757 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:22:04.714650    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-a-3aca3f-67994c4bc5-hwmd2"]
Oct 31 01:22:04.719266 ip-10-0-132-173 kubenswrapper[4284]: W1031 01:22:04.719238    4284 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod16fc298a_4f7c_4efd_ac33_5ac97b7d3275.slice/crio-5499da23d0bcae83e804d85f7ac4d0f75473f28b28d2280cecbffc68a7df6e57 WatchSource:0}: Error finding container 5499da23d0bcae83e804d85f7ac4d0f75473f28b28d2280cecbffc68a7df6e57: Status 404 returned error can't find the container with id 5499da23d0bcae83e804d85f7ac4d0f75473f28b28d2280cecbffc68a7df6e57
Oct 31 01:22:04.853756 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:22:04.853725    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-a-3aca3f-554dd84857-59rv6"]
Oct 31 01:22:04.854500 ip-10-0-132-173 kubenswrapper[4284]: W1031 01:22:04.854466    4284 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf31cff38_07af_4d6c_83ba_fcc15b830cbc.slice/crio-2d65d6c97cf60e16ed20ae4539ec1b062d94c28aa5b37550718667db1486488d WatchSource:0}: Error finding container 2d65d6c97cf60e16ed20ae4539ec1b062d94c28aa5b37550718667db1486488d: Status 404 returned error can't find the container with id 2d65d6c97cf60e16ed20ae4539ec1b062d94c28aa5b37550718667db1486488d
Oct 31 01:22:05.332019 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:22:05.331972    4284 kuberuntime_container.go:898] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-w1-3be6fa/auth-test-model-3be6fa-kserve-6f5d4f988f-zc2bc" podUID="cac35176-4262-433c-ba9d-ded7a7c11502" containerName="main" containerID="cri-o://cc56aba19a1e4f7deb0abd5fdec6d09d085a27cf5e06637ea0f4ad037202257f" gracePeriod=45
Oct 31 01:22:05.407238 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:22:05.405615    4284 prober.go:122] "Probe failed" probeType="Readiness" pod="ai-tenant-e2e-shared-b-w1-3be6fa/auth-test-model-3be6fa-kserve-6f5d4f988f-zc2bc" podUID="cac35176-4262-433c-ba9d-ded7a7c11502" containerName="main" probeResult="failure" output="Get \"https://10.134.0.173:8000/ready\": dial tcp 10.134.0.173:8000: connect: connection refused"
Oct 31 01:22:05.445627 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:22:05.445591    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-extmod-a-3aca3f-554dd84857-59rv6" event={"ID":"f31cff38-07af-4d6c-83ba-fcc15b830cbc","Type":"ContainerStarted","Data":"1e2a1519420a7ab584361faf5ec11a0db28214841e72854c92d4546e4304d730"}
Oct 31 01:22:05.445814 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:22:05.445688    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-extmod-a-3aca3f-554dd84857-59rv6" event={"ID":"f31cff38-07af-4d6c-83ba-fcc15b830cbc","Type":"ContainerStarted","Data":"2d65d6c97cf60e16ed20ae4539ec1b062d94c28aa5b37550718667db1486488d"}
Oct 31 01:22:05.445891 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:22:05.445836    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-extmod-a-3aca3f-554dd84857-59rv6"
Oct 31 01:22:05.447532 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:22:05.447499    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-3aca3f-67994c4bc5-hwmd2" event={"ID":"16fc298a-4f7c-4efd-ac33-5ac97b7d3275","Type":"ContainerStarted","Data":"7138bb9c84157a17fdcdd52579a072a4c17efaae3a7bebeea3b093e8a992f0f9"}
Oct 31 01:22:05.447664 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:22:05.447549    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-3aca3f-67994c4bc5-hwmd2" event={"ID":"16fc298a-4f7c-4efd-ac33-5ac97b7d3275","Type":"ContainerStarted","Data":"5499da23d0bcae83e804d85f7ac4d0f75473f28b28d2280cecbffc68a7df6e57"}
Oct 31 01:22:05.447664 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:22:05.447569    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-3aca3f-67994c4bc5-hwmd2"
Oct 31 01:22:05.560892 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:22:05.560835    4284 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-extmod-a-3aca3f-554dd84857-59rv6" podStartSLOduration=2.560821469 podStartE2EDuration="2.560821469s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 01:22:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 01:22:05.560368548 +0000 UTC m=+3239.705375466" watchObservedRunningTime="2026-10-31 01:22:05.560821469 +0000 UTC m=+3239.705828384"
Oct 31 01:22:05.642463 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:22:05.642435    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w1-3be6fa/auth-test-model-3be6fa-kserve-6f5d4f988f-zc2bc"
Oct 31 01:22:05.685317 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:22:05.684277    4284 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-3aca3f-67994c4bc5-hwmd2" podStartSLOduration=2.684258015 podStartE2EDuration="2.684258015s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 01:22:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 01:22:05.684190872 +0000 UTC m=+3239.829197789" watchObservedRunningTime="2026-10-31 01:22:05.684258015 +0000 UTC m=+3239.829264931"
Oct 31 01:22:05.838403 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:22:05.822867    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/cac35176-4262-433c-ba9d-ded7a7c11502-tls-certs\" (UniqueName: \"kubernetes.io/secret/cac35176-4262-433c-ba9d-ded7a7c11502-tls-certs\") pod \"cac35176-4262-433c-ba9d-ded7a7c11502\" (UID: \"cac35176-4262-433c-ba9d-ded7a7c11502\") "
Oct 31 01:22:05.838403 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:22:05.822956    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cac35176-4262-433c-ba9d-ded7a7c11502-home\" (UniqueName: \"kubernetes.io/empty-dir/cac35176-4262-433c-ba9d-ded7a7c11502-home\") pod \"cac35176-4262-433c-ba9d-ded7a7c11502\" (UID: \"cac35176-4262-433c-ba9d-ded7a7c11502\") "
Oct 31 01:22:05.838403 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:22:05.823117    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cac35176-4262-433c-ba9d-ded7a7c11502-dshm\" (UniqueName: \"kubernetes.io/empty-dir/cac35176-4262-433c-ba9d-ded7a7c11502-dshm\") pod \"cac35176-4262-433c-ba9d-ded7a7c11502\" (UID: \"cac35176-4262-433c-ba9d-ded7a7c11502\") "
Oct 31 01:22:05.838403 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:22:05.823356    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cac35176-4262-433c-ba9d-ded7a7c11502-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cac35176-4262-433c-ba9d-ded7a7c11502-model-cache\") pod \"cac35176-4262-433c-ba9d-ded7a7c11502\" (UID: \"cac35176-4262-433c-ba9d-ded7a7c11502\") "
Oct 31 01:22:05.838403 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:22:05.823429    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/cac35176-4262-433c-ba9d-ded7a7c11502-kube-api-access-xlmz4\" (UniqueName: \"kubernetes.io/projected/cac35176-4262-433c-ba9d-ded7a7c11502-kube-api-access-xlmz4\") pod \"cac35176-4262-433c-ba9d-ded7a7c11502\" (UID: \"cac35176-4262-433c-ba9d-ded7a7c11502\") "
Oct 31 01:22:05.838403 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:22:05.823459    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cac35176-4262-433c-ba9d-ded7a7c11502-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cac35176-4262-433c-ba9d-ded7a7c11502-tmp-dir\") pod \"cac35176-4262-433c-ba9d-ded7a7c11502\" (UID: \"cac35176-4262-433c-ba9d-ded7a7c11502\") "
Oct 31 01:22:05.838403 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:22:05.825496    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cac35176-4262-433c-ba9d-ded7a7c11502-home" pod "cac35176-4262-433c-ba9d-ded7a7c11502" (UID: "cac35176-4262-433c-ba9d-ded7a7c11502"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:22:05.838403 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:22:05.831784    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cac35176-4262-433c-ba9d-ded7a7c11502-tls-certs" pod "cac35176-4262-433c-ba9d-ded7a7c11502" (UID: "cac35176-4262-433c-ba9d-ded7a7c11502"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 01:22:05.838403 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:22:05.832031    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cac35176-4262-433c-ba9d-ded7a7c11502-model-cache" pod "cac35176-4262-433c-ba9d-ded7a7c11502" (UID: "cac35176-4262-433c-ba9d-ded7a7c11502"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:22:05.862065 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:22:05.845345    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cac35176-4262-433c-ba9d-ded7a7c11502-tmp-dir" pod "cac35176-4262-433c-ba9d-ded7a7c11502" (UID: "cac35176-4262-433c-ba9d-ded7a7c11502"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:22:05.862065 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:22:05.851426    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cac35176-4262-433c-ba9d-ded7a7c11502-dshm" pod "cac35176-4262-433c-ba9d-ded7a7c11502" (UID: "cac35176-4262-433c-ba9d-ded7a7c11502"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:22:05.862812 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:22:05.862766    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cac35176-4262-433c-ba9d-ded7a7c11502-kube-api-access-xlmz4" pod "cac35176-4262-433c-ba9d-ded7a7c11502" (UID: "cac35176-4262-433c-ba9d-ded7a7c11502"). InnerVolumeSpecName "kube-api-access-xlmz4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:22:05.925201 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:22:05.925168    4284 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cac35176-4262-433c-ba9d-ded7a7c11502-model-cache\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:22:05.925201 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:22:05.925196    4284 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xlmz4\" (UniqueName: \"kubernetes.io/projected/cac35176-4262-433c-ba9d-ded7a7c11502-kube-api-access-xlmz4\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:22:05.925201 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:22:05.925206    4284 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cac35176-4262-433c-ba9d-ded7a7c11502-tmp-dir\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:22:05.925201 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:22:05.925231    4284 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cac35176-4262-433c-ba9d-ded7a7c11502-tls-certs\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:22:05.925501 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:22:05.925239    4284 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cac35176-4262-433c-ba9d-ded7a7c11502-home\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:22:05.925501 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:22:05.925248    4284 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cac35176-4262-433c-ba9d-ded7a7c11502-dshm\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:22:06.454585 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:22:06.454554    4284 generic.go:356] "Generic (PLEG): container finished" podID="cac35176-4262-433c-ba9d-ded7a7c11502" containerID="cc56aba19a1e4f7deb0abd5fdec6d09d085a27cf5e06637ea0f4ad037202257f" exitCode=0
Oct 31 01:22:06.454778 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:22:06.454597    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w1-3be6fa/auth-test-model-3be6fa-kserve-6f5d4f988f-zc2bc"
Oct 31 01:22:06.454778 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:22:06.454639    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w1-3be6fa/auth-test-model-3be6fa-kserve-6f5d4f988f-zc2bc" event={"ID":"cac35176-4262-433c-ba9d-ded7a7c11502","Type":"ContainerDied","Data":"cc56aba19a1e4f7deb0abd5fdec6d09d085a27cf5e06637ea0f4ad037202257f"}
Oct 31 01:22:06.454778 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:22:06.454687    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w1-3be6fa/auth-test-model-3be6fa-kserve-6f5d4f988f-zc2bc" event={"ID":"cac35176-4262-433c-ba9d-ded7a7c11502","Type":"ContainerDied","Data":"d57bb213456eb4f892c5850dc8bfb20600c3302582ed2eef1a185cae4e8a9027"}
Oct 31 01:22:06.454778 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:22:06.454709    4284 scope.go:122] "RemoveContainer" containerID="cc56aba19a1e4f7deb0abd5fdec6d09d085a27cf5e06637ea0f4ad037202257f"
Oct 31 01:22:06.464737 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:22:06.464713    4284 scope.go:122] "RemoveContainer" containerID="cc56aba19a1e4f7deb0abd5fdec6d09d085a27cf5e06637ea0f4ad037202257f"
Oct 31 01:22:06.465008 ip-10-0-132-173 kubenswrapper[4284]: E1031 01:22:06.464991    4284 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cc56aba19a1e4f7deb0abd5fdec6d09d085a27cf5e06637ea0f4ad037202257f\": container with ID starting with cc56aba19a1e4f7deb0abd5fdec6d09d085a27cf5e06637ea0f4ad037202257f not found: ID does not exist" containerID="cc56aba19a1e4f7deb0abd5fdec6d09d085a27cf5e06637ea0f4ad037202257f"
Oct 31 01:22:06.465078 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:22:06.465017    4284 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cc56aba19a1e4f7deb0abd5fdec6d09d085a27cf5e06637ea0f4ad037202257f"} err="failed to get container status \"cc56aba19a1e4f7deb0abd5fdec6d09d085a27cf5e06637ea0f4ad037202257f\": rpc error: code = NotFound desc = could not find container \"cc56aba19a1e4f7deb0abd5fdec6d09d085a27cf5e06637ea0f4ad037202257f\": container with ID starting with cc56aba19a1e4f7deb0abd5fdec6d09d085a27cf5e06637ea0f4ad037202257f not found: ID does not exist"
Oct 31 01:22:06.614942 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:22:06.614861    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w1-3be6fa/auth-test-model-3be6fa-kserve-6f5d4f988f-zc2bc"]
Oct 31 01:22:06.662381 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:22:06.662348    4284 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-w1-3be6fa/auth-test-model-3be6fa-kserve-6f5d4f988f-zc2bc"]
Oct 31 01:22:07.010091 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:22:07.010022    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:22:07.134838 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:22:07.134810    4284 scope.go:122] "RemoveContainer" containerID="580ec896304e3f1366a874654ee80a193ae88148c7d7418ea5509b5801370547"
Oct 31 01:22:07.707020 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:22:07.706985    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:22:07.963549 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:22:07.963469    4284 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="cac35176-4262-433c-ba9d-ded7a7c11502" path="/var/lib/kubelet/pods/cac35176-4262-433c-ba9d-ded7a7c11502/volumes"
Oct 31 01:22:10.771392 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:22:10.771343    4284 patch_prober.go:29] interesting pod/e2e-extmod-a-eb68dd-openshift-default-98b48c6cc-gfz48 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.180:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 31 01:22:10.771986 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:22:10.771437    4284 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-extmod-a-eb68dd-openshift-default-98b48c6cc-gfz48" podUID="1dde1271-7b40-4dc0-97ca-c1be6b89c477" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.180:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 31 01:22:12.416503 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:22:12.416467    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:22:12.493921 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:22:12.493888    4284 generic.go:356] "Generic (PLEG): container finished" podID="1dde1271-7b40-4dc0-97ca-c1be6b89c477" containerID="bbab3f04b06e5edb3eaae4cb48bd2bef652990c808bf7fae3bbbf37a9beabcb1" exitCode=0
Oct 31 01:22:12.494102 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:22:12.493962    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-eb68dd-openshift-default-98b48c6cc-gfz48" event={"ID":"1dde1271-7b40-4dc0-97ca-c1be6b89c477","Type":"ContainerDied","Data":"bbab3f04b06e5edb3eaae4cb48bd2bef652990c808bf7fae3bbbf37a9beabcb1"}
Oct 31 01:22:12.494397 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:22:12.494382    4284 scope.go:122] "RemoveContainer" containerID="bbab3f04b06e5edb3eaae4cb48bd2bef652990c808bf7fae3bbbf37a9beabcb1"
Oct 31 01:22:12.496007 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:22:12.495980    4284 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 01:22:12.496072 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:22:12.496049    4284 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 01:22:12.496109 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:22:12.496079    4284 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 01:22:13.502010 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:22:13.501973    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-eb68dd-openshift-default-98b48c6cc-gfz48" event={"ID":"1dde1271-7b40-4dc0-97ca-c1be6b89c477","Type":"ContainerStarted","Data":"7010ec84037e565c88b766656f2b922c7077c92442970dde7646b0425199d25d"}
Oct 31 01:22:13.589940 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:22:13.589896    4284 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-extmod-a-eb68dd-openshift-default-98b48c6cc-gfz48"
Oct 31 01:22:13.590145 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:22:13.590123    4284 patch_prober.go:29] interesting pod/e2e-extmod-a-eb68dd-openshift-default-98b48c6cc-gfz48 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.180:15021/healthz/ready\": dial tcp 10.134.0.180:15021: connect: connection refused" start-of-body=
Oct 31 01:22:13.590200 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:22:13.590174    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-eb68dd-openshift-default-98b48c6cc-gfz48" podUID="1dde1271-7b40-4dc0-97ca-c1be6b89c477" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.180:15021/healthz/ready\": dial tcp 10.134.0.180:15021: connect: connection refused"
Oct 31 01:22:14.593271 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:22:14.589854    4284 patch_prober.go:29] interesting pod/e2e-extmod-a-eb68dd-openshift-default-98b48c6cc-gfz48 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.180:15021/healthz/ready\": dial tcp 10.134.0.180:15021: connect: connection refused" start-of-body=
Oct 31 01:22:14.593271 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:22:14.589955    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-eb68dd-openshift-default-98b48c6cc-gfz48" podUID="1dde1271-7b40-4dc0-97ca-c1be6b89c477" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.180:15021/healthz/ready\": dial tcp 10.134.0.180:15021: connect: connection refused"
Oct 31 01:22:15.604239 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:22:15.589580    4284 patch_prober.go:29] interesting pod/e2e-extmod-a-eb68dd-openshift-default-98b48c6cc-gfz48 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.180:15021/healthz/ready\": dial tcp 10.134.0.180:15021: connect: connection refused" start-of-body=
Oct 31 01:22:15.604239 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:22:15.589638    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-eb68dd-openshift-default-98b48c6cc-gfz48" podUID="1dde1271-7b40-4dc0-97ca-c1be6b89c477" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.180:15021/healthz/ready\": dial tcp 10.134.0.180:15021: connect: connection refused"
Oct 31 01:22:16.458253 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:22:16.458199    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-extmod-a-3aca3f-554dd84857-59rv6"
Oct 31 01:22:16.458524 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:22:16.458301    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-3aca3f-67994c4bc5-hwmd2"
Oct 31 01:22:16.589629 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:22:16.589594    4284 patch_prober.go:29] interesting pod/e2e-extmod-a-eb68dd-openshift-default-98b48c6cc-gfz48 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.180:15021/healthz/ready\": dial tcp 10.134.0.180:15021: connect: connection refused" start-of-body=
Oct 31 01:22:16.589898 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:22:16.589869    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-eb68dd-openshift-default-98b48c6cc-gfz48" podUID="1dde1271-7b40-4dc0-97ca-c1be6b89c477" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.180:15021/healthz/ready\": dial tcp 10.134.0.180:15021: connect: connection refused"
Oct 31 01:22:17.036864 ip-10-0-132-173 kubenswrapper[4284]: E1031 01:22:17.036832    4284 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fd8sp" containerName="extractor"
Oct 31 01:22:17.601238 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:22:17.598656    4284 patch_prober.go:29] interesting pod/e2e-extmod-a-eb68dd-openshift-default-98b48c6cc-gfz48 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.180:15021/healthz/ready\": dial tcp 10.134.0.180:15021: connect: connection refused" start-of-body=
Oct 31 01:22:17.601238 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:22:17.598731    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-eb68dd-openshift-default-98b48c6cc-gfz48" podUID="1dde1271-7b40-4dc0-97ca-c1be6b89c477" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.180:15021/healthz/ready\": dial tcp 10.134.0.180:15021: connect: connection refused"
Oct 31 01:22:18.110190 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:22:18.110143    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:22:18.334868 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:22:18.334817    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:22:18.479433 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:22:18.479399    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:22:18.593238 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:22:18.590543    4284 patch_prober.go:29] interesting pod/e2e-extmod-a-eb68dd-openshift-default-98b48c6cc-gfz48 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.180:15021/healthz/ready\": dial tcp 10.134.0.180:15021: connect: connection refused" start-of-body=
Oct 31 01:22:18.593238 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:22:18.590611    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-eb68dd-openshift-default-98b48c6cc-gfz48" podUID="1dde1271-7b40-4dc0-97ca-c1be6b89c477" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.180:15021/healthz/ready\": dial tcp 10.134.0.180:15021: connect: connection refused"
Oct 31 01:22:19.593541 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:22:19.593512    4284 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-extmod-a-eb68dd-openshift-default-98b48c6cc-gfz48"
Oct 31 01:22:19.593950 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:22:19.593815    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-extmod-a-eb68dd-openshift-default-98b48c6cc-gfz48"
Oct 31 01:22:19.594653 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:22:19.594637    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-extmod-a-eb68dd-openshift-default-98b48c6cc-gfz48"
Oct 31 01:22:22.088573 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:22:22.088541    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:22:26.828750 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:22:26.828708    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:22:28.519774 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:22:28.519734    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:22:30.201729 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:22:30.201691    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:22:30.768878 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:22:30.768841    4284 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-extmod-a-d067bd-openshift-default-766dbd44fc-7zjcd"]
Oct 31 01:22:30.769785 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:22:30.769759    4284 cpu_manager.go:414] "containerMap: removing container" podUID="cac35176-4262-433c-ba9d-ded7a7c11502" containerName="main"
Oct 31 01:22:30.769989 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:22:30.769972    4284 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="cac35176-4262-433c-ba9d-ded7a7c11502" containerName="main"
Oct 31 01:22:30.770336 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:22:30.770320    4284 memory_manager.go:365] "RemoveStaleState removing state" podUID="cac35176-4262-433c-ba9d-ded7a7c11502" containerName="main"
Oct 31 01:22:30.774635 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:22:30.774614    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-a-d067bd-openshift-default-766dbd44fc-7zjcd"
Oct 31 01:22:30.833149 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:22:30.828292    4284 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-extmod-a-d067bd-openshift-default-dockercfg-ht25d\""
Oct 31 01:22:30.841293 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:22:30.835828    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:22:30.910920 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:22:30.910839    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-extmod-a-d067bd-openshift-default-766dbd44fc-7zjcd"]
Oct 31 01:22:30.943736 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:22:30.943698    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6ac9b2b9-a157-4f05-8c7e-e7a2987397f6-workload-certs\") pod \"e2e-extmod-a-d067bd-openshift-default-766dbd44fc-7zjcd\" (UID: \"6ac9b2b9-a157-4f05-8c7e-e7a2987397f6\") " pod="openshift-ingress/e2e-extmod-a-d067bd-openshift-default-766dbd44fc-7zjcd"
Oct 31 01:22:30.943736 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:22:30.943737    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6ac9b2b9-a157-4f05-8c7e-e7a2987397f6-credential-socket\") pod \"e2e-extmod-a-d067bd-openshift-default-766dbd44fc-7zjcd\" (UID: \"6ac9b2b9-a157-4f05-8c7e-e7a2987397f6\") " pod="openshift-ingress/e2e-extmod-a-d067bd-openshift-default-766dbd44fc-7zjcd"
Oct 31 01:22:30.943974 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:22:30.943954    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6ac9b2b9-a157-4f05-8c7e-e7a2987397f6-istiod-ca-cert\") pod \"e2e-extmod-a-d067bd-openshift-default-766dbd44fc-7zjcd\" (UID: \"6ac9b2b9-a157-4f05-8c7e-e7a2987397f6\") " pod="openshift-ingress/e2e-extmod-a-d067bd-openshift-default-766dbd44fc-7zjcd"
Oct 31 01:22:30.944041 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:22:30.944025    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6ac9b2b9-a157-4f05-8c7e-e7a2987397f6-istio-data\") pod \"e2e-extmod-a-d067bd-openshift-default-766dbd44fc-7zjcd\" (UID: \"6ac9b2b9-a157-4f05-8c7e-e7a2987397f6\") " pod="openshift-ingress/e2e-extmod-a-d067bd-openshift-default-766dbd44fc-7zjcd"
Oct 31 01:22:30.944087 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:22:30.944072    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2pgzz\" (UniqueName: \"kubernetes.io/projected/6ac9b2b9-a157-4f05-8c7e-e7a2987397f6-kube-api-access-2pgzz\") pod \"e2e-extmod-a-d067bd-openshift-default-766dbd44fc-7zjcd\" (UID: \"6ac9b2b9-a157-4f05-8c7e-e7a2987397f6\") " pod="openshift-ingress/e2e-extmod-a-d067bd-openshift-default-766dbd44fc-7zjcd"
Oct 31 01:22:30.944138 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:22:30.944126    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6ac9b2b9-a157-4f05-8c7e-e7a2987397f6-istio-token\") pod \"e2e-extmod-a-d067bd-openshift-default-766dbd44fc-7zjcd\" (UID: \"6ac9b2b9-a157-4f05-8c7e-e7a2987397f6\") " pod="openshift-ingress/e2e-extmod-a-d067bd-openshift-default-766dbd44fc-7zjcd"
Oct 31 01:22:30.944203 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:22:30.944191    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6ac9b2b9-a157-4f05-8c7e-e7a2987397f6-workload-socket\") pod \"e2e-extmod-a-d067bd-openshift-default-766dbd44fc-7zjcd\" (UID: \"6ac9b2b9-a157-4f05-8c7e-e7a2987397f6\") " pod="openshift-ingress/e2e-extmod-a-d067bd-openshift-default-766dbd44fc-7zjcd"
Oct 31 01:22:30.944265 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:22:30.944245    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6ac9b2b9-a157-4f05-8c7e-e7a2987397f6-istio-podinfo\") pod \"e2e-extmod-a-d067bd-openshift-default-766dbd44fc-7zjcd\" (UID: \"6ac9b2b9-a157-4f05-8c7e-e7a2987397f6\") " pod="openshift-ingress/e2e-extmod-a-d067bd-openshift-default-766dbd44fc-7zjcd"
Oct 31 01:22:30.944308 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:22:30.944272    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6ac9b2b9-a157-4f05-8c7e-e7a2987397f6-istio-envoy\") pod \"e2e-extmod-a-d067bd-openshift-default-766dbd44fc-7zjcd\" (UID: \"6ac9b2b9-a157-4f05-8c7e-e7a2987397f6\") " pod="openshift-ingress/e2e-extmod-a-d067bd-openshift-default-766dbd44fc-7zjcd"
Oct 31 01:22:30.999808 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:22:30.999771    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:22:31.045534 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:22:31.045434    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6ac9b2b9-a157-4f05-8c7e-e7a2987397f6-workload-socket\") pod \"e2e-extmod-a-d067bd-openshift-default-766dbd44fc-7zjcd\" (UID: \"6ac9b2b9-a157-4f05-8c7e-e7a2987397f6\") " pod="openshift-ingress/e2e-extmod-a-d067bd-openshift-default-766dbd44fc-7zjcd"
Oct 31 01:22:31.045534 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:22:31.045482    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6ac9b2b9-a157-4f05-8c7e-e7a2987397f6-istio-podinfo\") pod \"e2e-extmod-a-d067bd-openshift-default-766dbd44fc-7zjcd\" (UID: \"6ac9b2b9-a157-4f05-8c7e-e7a2987397f6\") " pod="openshift-ingress/e2e-extmod-a-d067bd-openshift-default-766dbd44fc-7zjcd"
Oct 31 01:22:31.045802 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:22:31.045610    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6ac9b2b9-a157-4f05-8c7e-e7a2987397f6-istio-envoy\") pod \"e2e-extmod-a-d067bd-openshift-default-766dbd44fc-7zjcd\" (UID: \"6ac9b2b9-a157-4f05-8c7e-e7a2987397f6\") " pod="openshift-ingress/e2e-extmod-a-d067bd-openshift-default-766dbd44fc-7zjcd"
Oct 31 01:22:31.045802 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:22:31.045680    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6ac9b2b9-a157-4f05-8c7e-e7a2987397f6-workload-certs\") pod \"e2e-extmod-a-d067bd-openshift-default-766dbd44fc-7zjcd\" (UID: \"6ac9b2b9-a157-4f05-8c7e-e7a2987397f6\") " pod="openshift-ingress/e2e-extmod-a-d067bd-openshift-default-766dbd44fc-7zjcd"
Oct 31 01:22:31.045802 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:22:31.045702    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6ac9b2b9-a157-4f05-8c7e-e7a2987397f6-credential-socket\") pod \"e2e-extmod-a-d067bd-openshift-default-766dbd44fc-7zjcd\" (UID: \"6ac9b2b9-a157-4f05-8c7e-e7a2987397f6\") " pod="openshift-ingress/e2e-extmod-a-d067bd-openshift-default-766dbd44fc-7zjcd"
Oct 31 01:22:31.045978 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:22:31.045819    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6ac9b2b9-a157-4f05-8c7e-e7a2987397f6-istiod-ca-cert\") pod \"e2e-extmod-a-d067bd-openshift-default-766dbd44fc-7zjcd\" (UID: \"6ac9b2b9-a157-4f05-8c7e-e7a2987397f6\") " pod="openshift-ingress/e2e-extmod-a-d067bd-openshift-default-766dbd44fc-7zjcd"
Oct 31 01:22:31.045978 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:22:31.045870    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6ac9b2b9-a157-4f05-8c7e-e7a2987397f6-istio-data\") pod \"e2e-extmod-a-d067bd-openshift-default-766dbd44fc-7zjcd\" (UID: \"6ac9b2b9-a157-4f05-8c7e-e7a2987397f6\") " pod="openshift-ingress/e2e-extmod-a-d067bd-openshift-default-766dbd44fc-7zjcd"
Oct 31 01:22:31.045978 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:22:31.045889    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6ac9b2b9-a157-4f05-8c7e-e7a2987397f6-workload-socket\") pod \"e2e-extmod-a-d067bd-openshift-default-766dbd44fc-7zjcd\" (UID: \"6ac9b2b9-a157-4f05-8c7e-e7a2987397f6\") " pod="openshift-ingress/e2e-extmod-a-d067bd-openshift-default-766dbd44fc-7zjcd"
Oct 31 01:22:31.045978 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:22:31.045902    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2pgzz\" (UniqueName: \"kubernetes.io/projected/6ac9b2b9-a157-4f05-8c7e-e7a2987397f6-kube-api-access-2pgzz\") pod \"e2e-extmod-a-d067bd-openshift-default-766dbd44fc-7zjcd\" (UID: \"6ac9b2b9-a157-4f05-8c7e-e7a2987397f6\") " pod="openshift-ingress/e2e-extmod-a-d067bd-openshift-default-766dbd44fc-7zjcd"
Oct 31 01:22:31.046190 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:22:31.045999    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6ac9b2b9-a157-4f05-8c7e-e7a2987397f6-istio-token\") pod \"e2e-extmod-a-d067bd-openshift-default-766dbd44fc-7zjcd\" (UID: \"6ac9b2b9-a157-4f05-8c7e-e7a2987397f6\") " pod="openshift-ingress/e2e-extmod-a-d067bd-openshift-default-766dbd44fc-7zjcd"
Oct 31 01:22:31.046190 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:22:31.046010    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6ac9b2b9-a157-4f05-8c7e-e7a2987397f6-workload-certs\") pod \"e2e-extmod-a-d067bd-openshift-default-766dbd44fc-7zjcd\" (UID: \"6ac9b2b9-a157-4f05-8c7e-e7a2987397f6\") " pod="openshift-ingress/e2e-extmod-a-d067bd-openshift-default-766dbd44fc-7zjcd"
Oct 31 01:22:31.046190 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:22:31.046089    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6ac9b2b9-a157-4f05-8c7e-e7a2987397f6-credential-socket\") pod \"e2e-extmod-a-d067bd-openshift-default-766dbd44fc-7zjcd\" (UID: \"6ac9b2b9-a157-4f05-8c7e-e7a2987397f6\") " pod="openshift-ingress/e2e-extmod-a-d067bd-openshift-default-766dbd44fc-7zjcd"
Oct 31 01:22:31.046411 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:22:31.046363    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6ac9b2b9-a157-4f05-8c7e-e7a2987397f6-istio-data\") pod \"e2e-extmod-a-d067bd-openshift-default-766dbd44fc-7zjcd\" (UID: \"6ac9b2b9-a157-4f05-8c7e-e7a2987397f6\") " pod="openshift-ingress/e2e-extmod-a-d067bd-openshift-default-766dbd44fc-7zjcd"
Oct 31 01:22:31.046641 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:22:31.046621    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6ac9b2b9-a157-4f05-8c7e-e7a2987397f6-istiod-ca-cert\") pod \"e2e-extmod-a-d067bd-openshift-default-766dbd44fc-7zjcd\" (UID: \"6ac9b2b9-a157-4f05-8c7e-e7a2987397f6\") " pod="openshift-ingress/e2e-extmod-a-d067bd-openshift-default-766dbd44fc-7zjcd"
Oct 31 01:22:31.048046 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:22:31.048025    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6ac9b2b9-a157-4f05-8c7e-e7a2987397f6-istio-envoy\") pod \"e2e-extmod-a-d067bd-openshift-default-766dbd44fc-7zjcd\" (UID: \"6ac9b2b9-a157-4f05-8c7e-e7a2987397f6\") " pod="openshift-ingress/e2e-extmod-a-d067bd-openshift-default-766dbd44fc-7zjcd"
Oct 31 01:22:31.048265 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:22:31.048249    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6ac9b2b9-a157-4f05-8c7e-e7a2987397f6-istio-podinfo\") pod \"e2e-extmod-a-d067bd-openshift-default-766dbd44fc-7zjcd\" (UID: \"6ac9b2b9-a157-4f05-8c7e-e7a2987397f6\") " pod="openshift-ingress/e2e-extmod-a-d067bd-openshift-default-766dbd44fc-7zjcd"
Oct 31 01:22:31.105173 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:22:31.105137    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6ac9b2b9-a157-4f05-8c7e-e7a2987397f6-istio-token\") pod \"e2e-extmod-a-d067bd-openshift-default-766dbd44fc-7zjcd\" (UID: \"6ac9b2b9-a157-4f05-8c7e-e7a2987397f6\") " pod="openshift-ingress/e2e-extmod-a-d067bd-openshift-default-766dbd44fc-7zjcd"
Oct 31 01:22:31.105388 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:22:31.105364    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2pgzz\" (UniqueName: \"kubernetes.io/projected/6ac9b2b9-a157-4f05-8c7e-e7a2987397f6-kube-api-access-2pgzz\") pod \"e2e-extmod-a-d067bd-openshift-default-766dbd44fc-7zjcd\" (UID: \"6ac9b2b9-a157-4f05-8c7e-e7a2987397f6\") " pod="openshift-ingress/e2e-extmod-a-d067bd-openshift-default-766dbd44fc-7zjcd"
Oct 31 01:22:31.134724 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:22:31.134688    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-a-d067bd-openshift-default-766dbd44fc-7zjcd"
Oct 31 01:22:31.553768 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:22:31.553738    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-extmod-a-d067bd-openshift-default-766dbd44fc-7zjcd"]
Oct 31 01:22:31.554393 ip-10-0-132-173 kubenswrapper[4284]: W1031 01:22:31.554367    4284 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6ac9b2b9_a157_4f05_8c7e_e7a2987397f6.slice/crio-ccb855dbe24faa26a511e2f7328734903e4ba5a06a61dc1a0ed1117e4073045f WatchSource:0}: Error finding container ccb855dbe24faa26a511e2f7328734903e4ba5a06a61dc1a0ed1117e4073045f: Status 404 returned error can't find the container with id ccb855dbe24faa26a511e2f7328734903e4ba5a06a61dc1a0ed1117e4073045f
Oct 31 01:22:31.557266 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:22:31.557203    4284 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 01:22:31.557371 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:22:31.557304    4284 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 01:22:31.557371 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:22:31.557333    4284 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 01:22:31.565445 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:22:31.565420    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:22:31.630841 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:22:31.630799    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-d067bd-openshift-default-766dbd44fc-7zjcd" event={"ID":"6ac9b2b9-a157-4f05-8c7e-e7a2987397f6","Type":"ContainerStarted","Data":"ccb855dbe24faa26a511e2f7328734903e4ba5a06a61dc1a0ed1117e4073045f"}
Oct 31 01:22:31.806377 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:22:31.806290    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:22:32.638968 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:22:32.638935    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-d067bd-openshift-default-766dbd44fc-7zjcd" event={"ID":"6ac9b2b9-a157-4f05-8c7e-e7a2987397f6","Type":"ContainerStarted","Data":"58899a926296d61cc7d889798b19a5ec9104930e416b0c0ab3c601e7b9acd728"}
Oct 31 01:22:32.852521 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:22:32.852462    4284 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-extmod-a-d067bd-openshift-default-766dbd44fc-7zjcd" podStartSLOduration=2.852445264 podStartE2EDuration="2.852445264s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 01:22:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 01:22:32.850401595 +0000 UTC m=+3266.995408511" watchObservedRunningTime="2026-10-31 01:22:32.852445264 +0000 UTC m=+3266.997452247"
Oct 31 01:22:33.135411 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:22:33.135375    4284 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-extmod-a-d067bd-openshift-default-766dbd44fc-7zjcd"
Oct 31 01:22:33.141361 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:22:33.141335    4284 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-extmod-a-d067bd-openshift-default-766dbd44fc-7zjcd"
Oct 31 01:22:33.163272 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:22:33.163239    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:22:33.330421 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:22:33.330385    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:22:33.644710 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:22:33.644675    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-extmod-a-d067bd-openshift-default-766dbd44fc-7zjcd"
Oct 31 01:22:33.645627 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:22:33.645603    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-extmod-a-d067bd-openshift-default-766dbd44fc-7zjcd"
Oct 31 01:22:35.769837 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:22:35.769794    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:22:35.919834 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:22:35.919802    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:22:36.069641 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:22:36.069551    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:22:36.655495 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:22:36.655461    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:22:38.395947 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:22:38.395907    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:22:38.603336 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:22:38.603301    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:22:38.756648 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:22:38.756609    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:22:38.993953 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:22:38.993912    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:22:39.168367 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:22:39.168332    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:22:39.410437 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:22:39.410407    4284 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-a-d067bd-7469974c5f-22k9f"]
Oct 31 01:22:39.414528 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:22:39.414508    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-d067bd-7469974c5f-22k9f"
Oct 31 01:22:39.455020 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:22:39.454983    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t8lsq\" (UniqueName: \"kubernetes.io/projected/a71dc36e-312b-4dff-b412-056d51f65731-kube-api-access-t8lsq\") pod \"payload-pre-processing-e2e-extmod-a-d067bd-7469974c5f-22k9f\" (UID: \"a71dc36e-312b-4dff-b412-056d51f65731\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-d067bd-7469974c5f-22k9f"
Oct 31 01:22:39.455179 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:22:39.455033    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a71dc36e-312b-4dff-b412-056d51f65731-plugins-config-volume\") pod \"payload-pre-processing-e2e-extmod-a-d067bd-7469974c5f-22k9f\" (UID: \"a71dc36e-312b-4dff-b412-056d51f65731\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-d067bd-7469974c5f-22k9f"
Oct 31 01:22:39.468544 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:22:39.468516    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-a-d067bd-7469974c5f-22k9f"]
Oct 31 01:22:39.528824 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:22:39.528797    4284 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-extmod-a-d067bd-dockercfg-hhg9k\""
Oct 31 01:22:39.529168 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:22:39.529153    4284 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-extmod-a-d067bd\""
Oct 31 01:22:39.556527 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:22:39.556491    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t8lsq\" (UniqueName: \"kubernetes.io/projected/a71dc36e-312b-4dff-b412-056d51f65731-kube-api-access-t8lsq\") pod \"payload-pre-processing-e2e-extmod-a-d067bd-7469974c5f-22k9f\" (UID: \"a71dc36e-312b-4dff-b412-056d51f65731\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-d067bd-7469974c5f-22k9f"
Oct 31 01:22:39.556805 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:22:39.556787    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a71dc36e-312b-4dff-b412-056d51f65731-plugins-config-volume\") pod \"payload-pre-processing-e2e-extmod-a-d067bd-7469974c5f-22k9f\" (UID: \"a71dc36e-312b-4dff-b412-056d51f65731\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-d067bd-7469974c5f-22k9f"
Oct 31 01:22:39.558194 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:22:39.558161    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a71dc36e-312b-4dff-b412-056d51f65731-plugins-config-volume\") pod \"payload-pre-processing-e2e-extmod-a-d067bd-7469974c5f-22k9f\" (UID: \"a71dc36e-312b-4dff-b412-056d51f65731\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-d067bd-7469974c5f-22k9f"
Oct 31 01:22:39.633659 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:22:39.633626    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t8lsq\" (UniqueName: \"kubernetes.io/projected/a71dc36e-312b-4dff-b412-056d51f65731-kube-api-access-t8lsq\") pod \"payload-pre-processing-e2e-extmod-a-d067bd-7469974c5f-22k9f\" (UID: \"a71dc36e-312b-4dff-b412-056d51f65731\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-d067bd-7469974c5f-22k9f"
Oct 31 01:22:39.728988 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:22:39.728909    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-d067bd-7469974c5f-22k9f"
Oct 31 01:22:40.098434 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:22:40.098410    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-a-d067bd-7469974c5f-22k9f"]
Oct 31 01:22:40.100580 ip-10-0-132-173 kubenswrapper[4284]: W1031 01:22:40.100535    4284 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda71dc36e_312b_4dff_b412_056d51f65731.slice/crio-6a9a8e3e7eff65fc8efa8679bf4a45f2446f692649af2dcb7b2fce5b9dd2eb3b WatchSource:0}: Error finding container 6a9a8e3e7eff65fc8efa8679bf4a45f2446f692649af2dcb7b2fce5b9dd2eb3b: Status 404 returned error can't find the container with id 6a9a8e3e7eff65fc8efa8679bf4a45f2446f692649af2dcb7b2fce5b9dd2eb3b
Oct 31 01:22:40.686905 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:22:40.686864    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-d067bd-7469974c5f-22k9f" event={"ID":"a71dc36e-312b-4dff-b412-056d51f65731","Type":"ContainerStarted","Data":"607c407d4b9760aab40202699c66e68bad42f08ba96973142df16ede4a406cdb"}
Oct 31 01:22:40.687388 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:22:40.687068    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-d067bd-7469974c5f-22k9f"
Oct 31 01:22:40.687388 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:22:40.687127    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-d067bd-7469974c5f-22k9f" event={"ID":"a71dc36e-312b-4dff-b412-056d51f65731","Type":"ContainerStarted","Data":"6a9a8e3e7eff65fc8efa8679bf4a45f2446f692649af2dcb7b2fce5b9dd2eb3b"}
Oct 31 01:22:40.798577 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:22:40.798530    4284 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-d067bd-7469974c5f-22k9f" podStartSLOduration=1.798516239 podStartE2EDuration="1.798516239s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 01:22:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 01:22:40.797921216 +0000 UTC m=+3274.942928133" watchObservedRunningTime="2026-10-31 01:22:40.798516239 +0000 UTC m=+3274.943523155"
Oct 31 01:22:41.461978 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:22:41.461947    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:22:41.909625 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:22:41.909592    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:22:45.862183 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:22:45.862148    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:22:46.044342 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:22:46.044301    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:22:46.201021 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:22:46.200983    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:22:47.019019 ip-10-0-132-173 kubenswrapper[4284]: E1031 01:22:47.018982    4284 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fd8sp" containerName="extractor"
Oct 31 01:22:47.045059 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:22:47.045022    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:22:47.231236 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:22:47.229294    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:22:47.727793 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:22:47.727757    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:22:49.337297 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:22:49.337264    4284 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-extmod-a-08fcc4-openshift-default-7985d77b8f-t22ds"]
Oct 31 01:22:49.341470 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:22:49.341451    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-a-08fcc4-openshift-default-7985d77b8f-t22ds"
Oct 31 01:22:49.392176 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:22:49.392145    4284 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-extmod-a-08fcc4-openshift-default-dockercfg-2pmsw\""
Oct 31 01:22:49.427472 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:22:49.427437    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/04e2ad9d-c5b9-47b0-ba97-86114775393c-istio-podinfo\") pod \"e2e-extmod-a-08fcc4-openshift-default-7985d77b8f-t22ds\" (UID: \"04e2ad9d-c5b9-47b0-ba97-86114775393c\") " pod="openshift-ingress/e2e-extmod-a-08fcc4-openshift-default-7985d77b8f-t22ds"
Oct 31 01:22:49.427642 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:22:49.427491    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/04e2ad9d-c5b9-47b0-ba97-86114775393c-istio-data\") pod \"e2e-extmod-a-08fcc4-openshift-default-7985d77b8f-t22ds\" (UID: \"04e2ad9d-c5b9-47b0-ba97-86114775393c\") " pod="openshift-ingress/e2e-extmod-a-08fcc4-openshift-default-7985d77b8f-t22ds"
Oct 31 01:22:49.427642 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:22:49.427525    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/04e2ad9d-c5b9-47b0-ba97-86114775393c-workload-certs\") pod \"e2e-extmod-a-08fcc4-openshift-default-7985d77b8f-t22ds\" (UID: \"04e2ad9d-c5b9-47b0-ba97-86114775393c\") " pod="openshift-ingress/e2e-extmod-a-08fcc4-openshift-default-7985d77b8f-t22ds"
Oct 31 01:22:49.427719 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:22:49.427651    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/04e2ad9d-c5b9-47b0-ba97-86114775393c-istio-envoy\") pod \"e2e-extmod-a-08fcc4-openshift-default-7985d77b8f-t22ds\" (UID: \"04e2ad9d-c5b9-47b0-ba97-86114775393c\") " pod="openshift-ingress/e2e-extmod-a-08fcc4-openshift-default-7985d77b8f-t22ds"
Oct 31 01:22:49.427719 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:22:49.427683    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/04e2ad9d-c5b9-47b0-ba97-86114775393c-workload-socket\") pod \"e2e-extmod-a-08fcc4-openshift-default-7985d77b8f-t22ds\" (UID: \"04e2ad9d-c5b9-47b0-ba97-86114775393c\") " pod="openshift-ingress/e2e-extmod-a-08fcc4-openshift-default-7985d77b8f-t22ds"
Oct 31 01:22:49.427794 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:22:49.427717    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/04e2ad9d-c5b9-47b0-ba97-86114775393c-istio-token\") pod \"e2e-extmod-a-08fcc4-openshift-default-7985d77b8f-t22ds\" (UID: \"04e2ad9d-c5b9-47b0-ba97-86114775393c\") " pod="openshift-ingress/e2e-extmod-a-08fcc4-openshift-default-7985d77b8f-t22ds"
Oct 31 01:22:49.427833 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:22:49.427798    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/04e2ad9d-c5b9-47b0-ba97-86114775393c-credential-socket\") pod \"e2e-extmod-a-08fcc4-openshift-default-7985d77b8f-t22ds\" (UID: \"04e2ad9d-c5b9-47b0-ba97-86114775393c\") " pod="openshift-ingress/e2e-extmod-a-08fcc4-openshift-default-7985d77b8f-t22ds"
Oct 31 01:22:49.427894 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:22:49.427881    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/04e2ad9d-c5b9-47b0-ba97-86114775393c-istiod-ca-cert\") pod \"e2e-extmod-a-08fcc4-openshift-default-7985d77b8f-t22ds\" (UID: \"04e2ad9d-c5b9-47b0-ba97-86114775393c\") " pod="openshift-ingress/e2e-extmod-a-08fcc4-openshift-default-7985d77b8f-t22ds"
Oct 31 01:22:49.427983 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:22:49.427971    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x8nqn\" (UniqueName: \"kubernetes.io/projected/04e2ad9d-c5b9-47b0-ba97-86114775393c-kube-api-access-x8nqn\") pod \"e2e-extmod-a-08fcc4-openshift-default-7985d77b8f-t22ds\" (UID: \"04e2ad9d-c5b9-47b0-ba97-86114775393c\") " pod="openshift-ingress/e2e-extmod-a-08fcc4-openshift-default-7985d77b8f-t22ds"
Oct 31 01:22:49.434561 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:22:49.434531    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:22:49.444389 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:22:49.444357    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-extmod-a-08fcc4-openshift-default-7985d77b8f-t22ds"]
Oct 31 01:22:49.529321 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:22:49.529280    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x8nqn\" (UniqueName: \"kubernetes.io/projected/04e2ad9d-c5b9-47b0-ba97-86114775393c-kube-api-access-x8nqn\") pod \"e2e-extmod-a-08fcc4-openshift-default-7985d77b8f-t22ds\" (UID: \"04e2ad9d-c5b9-47b0-ba97-86114775393c\") " pod="openshift-ingress/e2e-extmod-a-08fcc4-openshift-default-7985d77b8f-t22ds"
Oct 31 01:22:49.529552 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:22:49.529347    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/04e2ad9d-c5b9-47b0-ba97-86114775393c-istio-podinfo\") pod \"e2e-extmod-a-08fcc4-openshift-default-7985d77b8f-t22ds\" (UID: \"04e2ad9d-c5b9-47b0-ba97-86114775393c\") " pod="openshift-ingress/e2e-extmod-a-08fcc4-openshift-default-7985d77b8f-t22ds"
Oct 31 01:22:49.529552 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:22:49.529391    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/04e2ad9d-c5b9-47b0-ba97-86114775393c-istio-data\") pod \"e2e-extmod-a-08fcc4-openshift-default-7985d77b8f-t22ds\" (UID: \"04e2ad9d-c5b9-47b0-ba97-86114775393c\") " pod="openshift-ingress/e2e-extmod-a-08fcc4-openshift-default-7985d77b8f-t22ds"
Oct 31 01:22:49.529552 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:22:49.529418    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/04e2ad9d-c5b9-47b0-ba97-86114775393c-workload-certs\") pod \"e2e-extmod-a-08fcc4-openshift-default-7985d77b8f-t22ds\" (UID: \"04e2ad9d-c5b9-47b0-ba97-86114775393c\") " pod="openshift-ingress/e2e-extmod-a-08fcc4-openshift-default-7985d77b8f-t22ds"
Oct 31 01:22:49.529552 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:22:49.529477    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/04e2ad9d-c5b9-47b0-ba97-86114775393c-istio-envoy\") pod \"e2e-extmod-a-08fcc4-openshift-default-7985d77b8f-t22ds\" (UID: \"04e2ad9d-c5b9-47b0-ba97-86114775393c\") " pod="openshift-ingress/e2e-extmod-a-08fcc4-openshift-default-7985d77b8f-t22ds"
Oct 31 01:22:49.529552 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:22:49.529516    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/04e2ad9d-c5b9-47b0-ba97-86114775393c-workload-socket\") pod \"e2e-extmod-a-08fcc4-openshift-default-7985d77b8f-t22ds\" (UID: \"04e2ad9d-c5b9-47b0-ba97-86114775393c\") " pod="openshift-ingress/e2e-extmod-a-08fcc4-openshift-default-7985d77b8f-t22ds"
Oct 31 01:22:49.529830 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:22:49.529566    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/04e2ad9d-c5b9-47b0-ba97-86114775393c-istio-token\") pod \"e2e-extmod-a-08fcc4-openshift-default-7985d77b8f-t22ds\" (UID: \"04e2ad9d-c5b9-47b0-ba97-86114775393c\") " pod="openshift-ingress/e2e-extmod-a-08fcc4-openshift-default-7985d77b8f-t22ds"
Oct 31 01:22:49.529830 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:22:49.529608    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/04e2ad9d-c5b9-47b0-ba97-86114775393c-credential-socket\") pod \"e2e-extmod-a-08fcc4-openshift-default-7985d77b8f-t22ds\" (UID: \"04e2ad9d-c5b9-47b0-ba97-86114775393c\") " pod="openshift-ingress/e2e-extmod-a-08fcc4-openshift-default-7985d77b8f-t22ds"
Oct 31 01:22:49.529830 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:22:49.529685    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/04e2ad9d-c5b9-47b0-ba97-86114775393c-istiod-ca-cert\") pod \"e2e-extmod-a-08fcc4-openshift-default-7985d77b8f-t22ds\" (UID: \"04e2ad9d-c5b9-47b0-ba97-86114775393c\") " pod="openshift-ingress/e2e-extmod-a-08fcc4-openshift-default-7985d77b8f-t22ds"
Oct 31 01:22:49.530600 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:22:49.530553    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/04e2ad9d-c5b9-47b0-ba97-86114775393c-istiod-ca-cert\") pod \"e2e-extmod-a-08fcc4-openshift-default-7985d77b8f-t22ds\" (UID: \"04e2ad9d-c5b9-47b0-ba97-86114775393c\") " pod="openshift-ingress/e2e-extmod-a-08fcc4-openshift-default-7985d77b8f-t22ds"
Oct 31 01:22:49.532561 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:22:49.531070    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/04e2ad9d-c5b9-47b0-ba97-86114775393c-workload-certs\") pod \"e2e-extmod-a-08fcc4-openshift-default-7985d77b8f-t22ds\" (UID: \"04e2ad9d-c5b9-47b0-ba97-86114775393c\") " pod="openshift-ingress/e2e-extmod-a-08fcc4-openshift-default-7985d77b8f-t22ds"
Oct 31 01:22:49.532561 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:22:49.532012    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/04e2ad9d-c5b9-47b0-ba97-86114775393c-workload-socket\") pod \"e2e-extmod-a-08fcc4-openshift-default-7985d77b8f-t22ds\" (UID: \"04e2ad9d-c5b9-47b0-ba97-86114775393c\") " pod="openshift-ingress/e2e-extmod-a-08fcc4-openshift-default-7985d77b8f-t22ds"
Oct 31 01:22:49.532930 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:22:49.532461    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/04e2ad9d-c5b9-47b0-ba97-86114775393c-credential-socket\") pod \"e2e-extmod-a-08fcc4-openshift-default-7985d77b8f-t22ds\" (UID: \"04e2ad9d-c5b9-47b0-ba97-86114775393c\") " pod="openshift-ingress/e2e-extmod-a-08fcc4-openshift-default-7985d77b8f-t22ds"
Oct 31 01:22:49.532930 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:22:49.532501    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/04e2ad9d-c5b9-47b0-ba97-86114775393c-istio-data\") pod \"e2e-extmod-a-08fcc4-openshift-default-7985d77b8f-t22ds\" (UID: \"04e2ad9d-c5b9-47b0-ba97-86114775393c\") " pod="openshift-ingress/e2e-extmod-a-08fcc4-openshift-default-7985d77b8f-t22ds"
Oct 31 01:22:49.534623 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:22:49.534599    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/04e2ad9d-c5b9-47b0-ba97-86114775393c-istio-podinfo\") pod \"e2e-extmod-a-08fcc4-openshift-default-7985d77b8f-t22ds\" (UID: \"04e2ad9d-c5b9-47b0-ba97-86114775393c\") " pod="openshift-ingress/e2e-extmod-a-08fcc4-openshift-default-7985d77b8f-t22ds"
Oct 31 01:22:49.536473 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:22:49.536447    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/04e2ad9d-c5b9-47b0-ba97-86114775393c-istio-envoy\") pod \"e2e-extmod-a-08fcc4-openshift-default-7985d77b8f-t22ds\" (UID: \"04e2ad9d-c5b9-47b0-ba97-86114775393c\") " pod="openshift-ingress/e2e-extmod-a-08fcc4-openshift-default-7985d77b8f-t22ds"
Oct 31 01:22:49.590358 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:22:49.590280    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/04e2ad9d-c5b9-47b0-ba97-86114775393c-istio-token\") pod \"e2e-extmod-a-08fcc4-openshift-default-7985d77b8f-t22ds\" (UID: \"04e2ad9d-c5b9-47b0-ba97-86114775393c\") " pod="openshift-ingress/e2e-extmod-a-08fcc4-openshift-default-7985d77b8f-t22ds"
Oct 31 01:22:49.590499 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:22:49.590445    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x8nqn\" (UniqueName: \"kubernetes.io/projected/04e2ad9d-c5b9-47b0-ba97-86114775393c-kube-api-access-x8nqn\") pod \"e2e-extmod-a-08fcc4-openshift-default-7985d77b8f-t22ds\" (UID: \"04e2ad9d-c5b9-47b0-ba97-86114775393c\") " pod="openshift-ingress/e2e-extmod-a-08fcc4-openshift-default-7985d77b8f-t22ds"
Oct 31 01:22:49.664233 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:22:49.655494    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-a-08fcc4-openshift-default-7985d77b8f-t22ds"
Oct 31 01:22:49.667957 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:22:49.667904    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:22:50.083614 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:22:50.083590    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-extmod-a-08fcc4-openshift-default-7985d77b8f-t22ds"]
Oct 31 01:22:50.084291 ip-10-0-132-173 kubenswrapper[4284]: W1031 01:22:50.084269    4284 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod04e2ad9d_c5b9_47b0_ba97_86114775393c.slice/crio-42a7f69d84606ef4dc540f248bc38cce6697aad2067783da937c44d1ee556bc3 WatchSource:0}: Error finding container 42a7f69d84606ef4dc540f248bc38cce6697aad2067783da937c44d1ee556bc3: Status 404 returned error can't find the container with id 42a7f69d84606ef4dc540f248bc38cce6697aad2067783da937c44d1ee556bc3
Oct 31 01:22:50.087133 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:22:50.087100    4284 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 01:22:50.087247 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:22:50.087167    4284 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 01:22:50.087247 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:22:50.087194    4284 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 01:22:50.783731 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:22:50.783693    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-08fcc4-openshift-default-7985d77b8f-t22ds" event={"ID":"04e2ad9d-c5b9-47b0-ba97-86114775393c","Type":"ContainerStarted","Data":"549889e4e2ee33f009fd5e45398cc88baf53ae9706354a3075d755a1b45735e4"}
Oct 31 01:22:50.784140 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:22:50.783746    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-08fcc4-openshift-default-7985d77b8f-t22ds" event={"ID":"04e2ad9d-c5b9-47b0-ba97-86114775393c","Type":"ContainerStarted","Data":"42a7f69d84606ef4dc540f248bc38cce6697aad2067783da937c44d1ee556bc3"}
Oct 31 01:22:50.950016 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:22:50.949966    4284 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-extmod-a-08fcc4-openshift-default-7985d77b8f-t22ds" podStartSLOduration=1.9499520750000001 podStartE2EDuration="1.949952075s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 01:22:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 01:22:50.946976241 +0000 UTC m=+3285.091983157" watchObservedRunningTime="2026-10-31 01:22:50.949952075 +0000 UTC m=+3285.094959009"
Oct 31 01:22:51.661585 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:22:51.661540    4284 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-extmod-a-08fcc4-openshift-default-7985d77b8f-t22ds"
Oct 31 01:22:51.668116 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:22:51.668088    4284 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-extmod-a-08fcc4-openshift-default-7985d77b8f-t22ds"
Oct 31 01:22:51.696267 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:22:51.696232    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-d067bd-7469974c5f-22k9f"
Oct 31 01:22:51.790630 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:22:51.790597    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-extmod-a-08fcc4-openshift-default-7985d77b8f-t22ds"
Oct 31 01:22:51.791640 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:22:51.791619    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-extmod-a-08fcc4-openshift-default-7985d77b8f-t22ds"
Oct 31 01:22:52.209679 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:22:52.209642    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:22:53.815917 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:22:53.815880    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:22:53.979297 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:22:53.979264    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:22:54.152909 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:22:54.152822    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:22:55.236589 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:22:55.236552    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:22:56.639083 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:22:56.639043    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:22:57.294157 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:22:57.294125    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:22:57.445243 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:22:57.439695    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:22:57.601298 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:22:57.601185    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:23:00.610660 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:23:00.610626    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:23:00.794658 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:23:00.794620    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:23:01.730759 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:23:01.730720    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:23:01.873963 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:23:01.873928    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:23:02.038197 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:23:02.038102    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:23:02.209590 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:23:02.209535    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:23:02.353345 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:23:02.353203    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:23:02.525743 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:23:02.525709    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:23:05.774475 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:23:05.774443    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:23:05.953300 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:23:05.953258    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:23:07.359544 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:23:07.359495    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:23:07.673481 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:23:07.673449    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:23:12.335728 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:23:12.335689    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:23:12.521994 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:23:12.521961    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:23:12.670047 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:23:12.670012    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:23:17.016060 ip-10-0-132-173 kubenswrapper[4284]: E1031 01:23:17.016024    4284 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fd8sp" containerName="extractor"
Oct 31 01:23:17.415585 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:23:17.415548    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:23:22.284777 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:23:22.284743    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:23:27.097417 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:23:27.097378    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:23:31.907101 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:23:31.907062    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:23:36.623392 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:23:36.623355    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:23:41.417678 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:23:41.417640    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:23:46.215271 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:23:46.215206    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:23:47.016137 ip-10-0-132-173 kubenswrapper[4284]: E1031 01:23:47.016100    4284 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fd8sp" containerName="extractor"
Oct 31 01:23:51.072104 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:23:51.072063    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:23:55.576328 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:23:55.576292    4284 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-a-d067bd-7ff8568785-rqbm9"]
Oct 31 01:23:55.580771 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:23:55.580745    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-d067bd-7ff8568785-rqbm9"
Oct 31 01:23:55.632495 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:23:55.632453    4284 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-extmod-a-d067bd-serving-cert\""
Oct 31 01:23:55.673993 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:23:55.673950    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/178450d7-7cf3-46f9-8e25-e09b73075316-maas-api-tls\") pod \"maas-api-e2e-extmod-a-d067bd-7ff8568785-rqbm9\" (UID: \"178450d7-7cf3-46f9-8e25-e09b73075316\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-d067bd-7ff8568785-rqbm9"
Oct 31 01:23:55.674245 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:23:55.674150    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/178450d7-7cf3-46f9-8e25-e09b73075316-openshift-service-ca\") pod \"maas-api-e2e-extmod-a-d067bd-7ff8568785-rqbm9\" (UID: \"178450d7-7cf3-46f9-8e25-e09b73075316\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-d067bd-7ff8568785-rqbm9"
Oct 31 01:23:55.674245 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:23:55.674196    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mxpg7\" (UniqueName: \"kubernetes.io/projected/178450d7-7cf3-46f9-8e25-e09b73075316-kube-api-access-mxpg7\") pod \"maas-api-e2e-extmod-a-d067bd-7ff8568785-rqbm9\" (UID: \"178450d7-7cf3-46f9-8e25-e09b73075316\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-d067bd-7ff8568785-rqbm9"
Oct 31 01:23:55.674355 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:23:55.674295    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/178450d7-7cf3-46f9-8e25-e09b73075316-metrics-certs\") pod \"maas-api-e2e-extmod-a-d067bd-7ff8568785-rqbm9\" (UID: \"178450d7-7cf3-46f9-8e25-e09b73075316\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-d067bd-7ff8568785-rqbm9"
Oct 31 01:23:55.683739 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:23:55.683713    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-a-d067bd-7ff8568785-rqbm9"]
Oct 31 01:23:55.765542 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:23:55.765511    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:23:55.775795 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:23:55.775765    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/178450d7-7cf3-46f9-8e25-e09b73075316-openshift-service-ca\") pod \"maas-api-e2e-extmod-a-d067bd-7ff8568785-rqbm9\" (UID: \"178450d7-7cf3-46f9-8e25-e09b73075316\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-d067bd-7ff8568785-rqbm9"
Oct 31 01:23:55.775935 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:23:55.775801    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mxpg7\" (UniqueName: \"kubernetes.io/projected/178450d7-7cf3-46f9-8e25-e09b73075316-kube-api-access-mxpg7\") pod \"maas-api-e2e-extmod-a-d067bd-7ff8568785-rqbm9\" (UID: \"178450d7-7cf3-46f9-8e25-e09b73075316\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-d067bd-7ff8568785-rqbm9"
Oct 31 01:23:55.775935 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:23:55.775829    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/178450d7-7cf3-46f9-8e25-e09b73075316-metrics-certs\") pod \"maas-api-e2e-extmod-a-d067bd-7ff8568785-rqbm9\" (UID: \"178450d7-7cf3-46f9-8e25-e09b73075316\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-d067bd-7ff8568785-rqbm9"
Oct 31 01:23:55.776019 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:23:55.776002    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/178450d7-7cf3-46f9-8e25-e09b73075316-maas-api-tls\") pod \"maas-api-e2e-extmod-a-d067bd-7ff8568785-rqbm9\" (UID: \"178450d7-7cf3-46f9-8e25-e09b73075316\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-d067bd-7ff8568785-rqbm9"
Oct 31 01:23:55.776407 ip-10-0-132-173 kubenswrapper[4284]: E1031 01:23:55.776153    4284 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-extmod-a-d067bd-serving-cert: secret "maas-api-e2e-extmod-a-d067bd-serving-cert" not found
Oct 31 01:23:55.776407 ip-10-0-132-173 kubenswrapper[4284]: E1031 01:23:55.776286    4284 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/178450d7-7cf3-46f9-8e25-e09b73075316-maas-api-tls podName:178450d7-7cf3-46f9-8e25-e09b73075316 nodeName:}" failed. No retries permitted until 2026-10-31 01:23:56.276259444 +0000 UTC m=+3350.421266357 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/178450d7-7cf3-46f9-8e25-e09b73075316-maas-api-tls") pod "maas-api-e2e-extmod-a-d067bd-7ff8568785-rqbm9" (UID: "178450d7-7cf3-46f9-8e25-e09b73075316") : secret "maas-api-e2e-extmod-a-d067bd-serving-cert" not found
Oct 31 01:23:55.776614 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:23:55.776596    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/178450d7-7cf3-46f9-8e25-e09b73075316-openshift-service-ca\") pod \"maas-api-e2e-extmod-a-d067bd-7ff8568785-rqbm9\" (UID: \"178450d7-7cf3-46f9-8e25-e09b73075316\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-d067bd-7ff8568785-rqbm9"
Oct 31 01:23:55.778412 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:23:55.778393    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/178450d7-7cf3-46f9-8e25-e09b73075316-metrics-certs\") pod \"maas-api-e2e-extmod-a-d067bd-7ff8568785-rqbm9\" (UID: \"178450d7-7cf3-46f9-8e25-e09b73075316\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-d067bd-7ff8568785-rqbm9"
Oct 31 01:23:55.854669 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:23:55.854596    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mxpg7\" (UniqueName: \"kubernetes.io/projected/178450d7-7cf3-46f9-8e25-e09b73075316-kube-api-access-mxpg7\") pod \"maas-api-e2e-extmod-a-d067bd-7ff8568785-rqbm9\" (UID: \"178450d7-7cf3-46f9-8e25-e09b73075316\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-d067bd-7ff8568785-rqbm9"
Oct 31 01:23:55.986741 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:23:55.986705    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:23:56.133112 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:23:56.133015    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:23:56.283998 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:23:56.283961    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/178450d7-7cf3-46f9-8e25-e09b73075316-maas-api-tls\") pod \"maas-api-e2e-extmod-a-d067bd-7ff8568785-rqbm9\" (UID: \"178450d7-7cf3-46f9-8e25-e09b73075316\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-d067bd-7ff8568785-rqbm9"
Oct 31 01:23:56.286700 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:23:56.286671    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/178450d7-7cf3-46f9-8e25-e09b73075316-maas-api-tls\") pod \"maas-api-e2e-extmod-a-d067bd-7ff8568785-rqbm9\" (UID: \"178450d7-7cf3-46f9-8e25-e09b73075316\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-d067bd-7ff8568785-rqbm9"
Oct 31 01:23:56.494322 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:23:56.494292    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-d067bd-7ff8568785-rqbm9"
Oct 31 01:23:56.870285 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:23:56.870257    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-a-d067bd-7ff8568785-rqbm9"]
Oct 31 01:23:56.871531 ip-10-0-132-173 kubenswrapper[4284]: W1031 01:23:56.871494    4284 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod178450d7_7cf3_46f9_8e25_e09b73075316.slice/crio-66ec309223334d8461962a293482f24506c5ad9a7ed9a03a036fb4a734fd702a WatchSource:0}: Error finding container 66ec309223334d8461962a293482f24506c5ad9a7ed9a03a036fb4a734fd702a: Status 404 returned error can't find the container with id 66ec309223334d8461962a293482f24506c5ad9a7ed9a03a036fb4a734fd702a
Oct 31 01:23:56.873068 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:23:56.873043    4284 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 31 01:23:57.253417 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:23:57.253382    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-d067bd-7ff8568785-rqbm9" event={"ID":"178450d7-7cf3-46f9-8e25-e09b73075316","Type":"ContainerStarted","Data":"66ec309223334d8461962a293482f24506c5ad9a7ed9a03a036fb4a734fd702a"}
Oct 31 01:23:58.261867 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:23:58.261833    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-d067bd-7ff8568785-rqbm9" event={"ID":"178450d7-7cf3-46f9-8e25-e09b73075316","Type":"ContainerStarted","Data":"1b87f2b4406fba76ae112312e6e05d181992f56022288b937495e8d04fb92ba1"}
Oct 31 01:23:58.262373 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:23:58.261904    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-d067bd-7ff8568785-rqbm9"
Oct 31 01:23:58.379463 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:23:58.379413    4284 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-d067bd-7ff8568785-rqbm9" podStartSLOduration=2.476529036 podStartE2EDuration="3.379397452s" totalImagesPullingTime="902.868416ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 01:23:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 01:23:58.375819982 +0000 UTC m=+3352.520826900" watchObservedRunningTime="2026-10-31 01:23:58.379397452 +0000 UTC m=+3352.524404412"
Oct 31 01:24:00.589167 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:24:00.589125    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:24:00.749092 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:24:00.749055    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:24:00.896280 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:24:00.896170    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:24:04.274516 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:24:04.274490    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-d067bd-7ff8568785-rqbm9"
Oct 31 01:24:05.505748 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:24:05.505713    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:24:07.672580 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:24:07.672544    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:24:10.287771 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:24:10.287717    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:24:15.053365 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:24:15.053326    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:24:17.039350 ip-10-0-132-173 kubenswrapper[4284]: E1031 01:24:17.039269    4284 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fd8sp" containerName="extractor"
Oct 31 01:24:19.941142 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:24:19.941106    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:24:20.132929 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:24:20.132892    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:24:20.278715 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:24:20.278635    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:24:24.999124 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:24:24.999083    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:24:25.474983 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:24:25.474951    4284 generic.go:356] "Generic (PLEG): container finished" podID="6ac9b2b9-a157-4f05-8c7e-e7a2987397f6" containerID="58899a926296d61cc7d889798b19a5ec9104930e416b0c0ab3c601e7b9acd728" exitCode=0
Oct 31 01:24:25.475200 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:24:25.475022    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-d067bd-openshift-default-766dbd44fc-7zjcd" event={"ID":"6ac9b2b9-a157-4f05-8c7e-e7a2987397f6","Type":"ContainerDied","Data":"58899a926296d61cc7d889798b19a5ec9104930e416b0c0ab3c601e7b9acd728"}
Oct 31 01:24:25.475516 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:24:25.475493    4284 scope.go:122] "RemoveContainer" containerID="58899a926296d61cc7d889798b19a5ec9104930e416b0c0ab3c601e7b9acd728"
Oct 31 01:24:25.477265 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:24:25.477233    4284 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 01:24:25.477394 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:24:25.477324    4284 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 01:24:25.477394 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:24:25.477374    4284 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 01:24:26.483583 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:24:26.483552    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-d067bd-openshift-default-766dbd44fc-7zjcd" event={"ID":"6ac9b2b9-a157-4f05-8c7e-e7a2987397f6","Type":"ContainerStarted","Data":"dbaf6050821a1875f066f86ae5c41b61638802ac781c0f784cc15f1df1d56580"}
Oct 31 01:24:27.135409 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:24:27.135366    4284 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-extmod-a-d067bd-openshift-default-766dbd44fc-7zjcd"
Oct 31 01:24:27.135665 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:24:27.135642    4284 patch_prober.go:29] interesting pod/e2e-extmod-a-d067bd-openshift-default-766dbd44fc-7zjcd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.184:15021/healthz/ready\": dial tcp 10.134.0.184:15021: connect: connection refused" start-of-body=
Oct 31 01:24:27.135757 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:24:27.135709    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-d067bd-openshift-default-766dbd44fc-7zjcd" podUID="6ac9b2b9-a157-4f05-8c7e-e7a2987397f6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.184:15021/healthz/ready\": dial tcp 10.134.0.184:15021: connect: connection refused"
Oct 31 01:24:28.135891 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:24:28.135849    4284 patch_prober.go:29] interesting pod/e2e-extmod-a-d067bd-openshift-default-766dbd44fc-7zjcd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.184:15021/healthz/ready\": dial tcp 10.134.0.184:15021: connect: connection refused" start-of-body=
Oct 31 01:24:28.136432 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:24:28.135908    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-d067bd-openshift-default-766dbd44fc-7zjcd" podUID="6ac9b2b9-a157-4f05-8c7e-e7a2987397f6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.184:15021/healthz/ready\": dial tcp 10.134.0.184:15021: connect: connection refused"
Oct 31 01:24:29.135237 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:24:29.135181    4284 patch_prober.go:29] interesting pod/e2e-extmod-a-d067bd-openshift-default-766dbd44fc-7zjcd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.184:15021/healthz/ready\": dial tcp 10.134.0.184:15021: connect: connection refused" start-of-body=
Oct 31 01:24:29.135424 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:24:29.135260    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-d067bd-openshift-default-766dbd44fc-7zjcd" podUID="6ac9b2b9-a157-4f05-8c7e-e7a2987397f6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.184:15021/healthz/ready\": dial tcp 10.134.0.184:15021: connect: connection refused"
Oct 31 01:24:29.947254 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:24:29.947197    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:24:30.135423 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:24:30.135388    4284 patch_prober.go:29] interesting pod/e2e-extmod-a-d067bd-openshift-default-766dbd44fc-7zjcd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.184:15021/healthz/ready\": dial tcp 10.134.0.184:15021: connect: connection refused" start-of-body=
Oct 31 01:24:30.135641 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:24:30.135446    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-d067bd-openshift-default-766dbd44fc-7zjcd" podUID="6ac9b2b9-a157-4f05-8c7e-e7a2987397f6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.184:15021/healthz/ready\": dial tcp 10.134.0.184:15021: connect: connection refused"
Oct 31 01:24:31.135714 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:24:31.135679    4284 patch_prober.go:29] interesting pod/e2e-extmod-a-d067bd-openshift-default-766dbd44fc-7zjcd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.184:15021/healthz/ready\": dial tcp 10.134.0.184:15021: connect: connection refused" start-of-body=
Oct 31 01:24:31.136131 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:24:31.135741    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-d067bd-openshift-default-766dbd44fc-7zjcd" podUID="6ac9b2b9-a157-4f05-8c7e-e7a2987397f6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.184:15021/healthz/ready\": dial tcp 10.134.0.184:15021: connect: connection refused"
Oct 31 01:24:32.139831 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:24:32.139799    4284 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-extmod-a-d067bd-openshift-default-766dbd44fc-7zjcd"
Oct 31 01:24:32.140251 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:24:32.140204    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-extmod-a-d067bd-openshift-default-766dbd44fc-7zjcd"
Oct 31 01:24:32.141012 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:24:32.140993    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-extmod-a-d067bd-openshift-default-766dbd44fc-7zjcd"
Oct 31 01:24:34.630947 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:24:34.630913    4284 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-b-9332a5-5bd6d599b-t2454"]
Oct 31 01:24:34.634344 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:24:34.634323    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-9332a5-5bd6d599b-t2454"
Oct 31 01:24:34.687522 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:24:34.687483    4284 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-extmod-b-9332a5-serving-cert\""
Oct 31 01:24:34.696777 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:24:34.696746    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-b-9332a5-5bd6d599b-t2454"]
Oct 31 01:24:34.719827 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:24:34.719791    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/40b79195-6d51-4041-9cbb-5acc6f1e7345-openshift-service-ca\") pod \"maas-api-e2e-extmod-b-9332a5-5bd6d599b-t2454\" (UID: \"40b79195-6d51-4041-9cbb-5acc6f1e7345\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-9332a5-5bd6d599b-t2454"
Oct 31 01:24:34.720024 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:24:34.719955    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8jmpc\" (UniqueName: \"kubernetes.io/projected/40b79195-6d51-4041-9cbb-5acc6f1e7345-kube-api-access-8jmpc\") pod \"maas-api-e2e-extmod-b-9332a5-5bd6d599b-t2454\" (UID: \"40b79195-6d51-4041-9cbb-5acc6f1e7345\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-9332a5-5bd6d599b-t2454"
Oct 31 01:24:34.720024 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:24:34.720004    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/40b79195-6d51-4041-9cbb-5acc6f1e7345-maas-api-tls\") pod \"maas-api-e2e-extmod-b-9332a5-5bd6d599b-t2454\" (UID: \"40b79195-6d51-4041-9cbb-5acc6f1e7345\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-9332a5-5bd6d599b-t2454"
Oct 31 01:24:34.720165 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:24:34.720027    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/40b79195-6d51-4041-9cbb-5acc6f1e7345-metrics-certs\") pod \"maas-api-e2e-extmod-b-9332a5-5bd6d599b-t2454\" (UID: \"40b79195-6d51-4041-9cbb-5acc6f1e7345\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-9332a5-5bd6d599b-t2454"
Oct 31 01:24:34.809615 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:24:34.809581    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:24:34.821803 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:24:34.821774    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8jmpc\" (UniqueName: \"kubernetes.io/projected/40b79195-6d51-4041-9cbb-5acc6f1e7345-kube-api-access-8jmpc\") pod \"maas-api-e2e-extmod-b-9332a5-5bd6d599b-t2454\" (UID: \"40b79195-6d51-4041-9cbb-5acc6f1e7345\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-9332a5-5bd6d599b-t2454"
Oct 31 01:24:34.821957 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:24:34.821819    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/40b79195-6d51-4041-9cbb-5acc6f1e7345-maas-api-tls\") pod \"maas-api-e2e-extmod-b-9332a5-5bd6d599b-t2454\" (UID: \"40b79195-6d51-4041-9cbb-5acc6f1e7345\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-9332a5-5bd6d599b-t2454"
Oct 31 01:24:34.821957 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:24:34.821843    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/40b79195-6d51-4041-9cbb-5acc6f1e7345-metrics-certs\") pod \"maas-api-e2e-extmod-b-9332a5-5bd6d599b-t2454\" (UID: \"40b79195-6d51-4041-9cbb-5acc6f1e7345\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-9332a5-5bd6d599b-t2454"
Oct 31 01:24:34.821957 ip-10-0-132-173 kubenswrapper[4284]: E1031 01:24:34.821937    4284 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-extmod-b-9332a5-serving-cert: secret "maas-api-e2e-extmod-b-9332a5-serving-cert" not found
Oct 31 01:24:34.822110 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:24:34.821979    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/40b79195-6d51-4041-9cbb-5acc6f1e7345-openshift-service-ca\") pod \"maas-api-e2e-extmod-b-9332a5-5bd6d599b-t2454\" (UID: \"40b79195-6d51-4041-9cbb-5acc6f1e7345\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-9332a5-5bd6d599b-t2454"
Oct 31 01:24:34.822110 ip-10-0-132-173 kubenswrapper[4284]: E1031 01:24:34.822012    4284 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/40b79195-6d51-4041-9cbb-5acc6f1e7345-maas-api-tls podName:40b79195-6d51-4041-9cbb-5acc6f1e7345 nodeName:}" failed. No retries permitted until 2026-10-31 01:24:35.321992673 +0000 UTC m=+3389.466999587 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/40b79195-6d51-4041-9cbb-5acc6f1e7345-maas-api-tls") pod "maas-api-e2e-extmod-b-9332a5-5bd6d599b-t2454" (UID: "40b79195-6d51-4041-9cbb-5acc6f1e7345") : secret "maas-api-e2e-extmod-b-9332a5-serving-cert" not found
Oct 31 01:24:34.822685 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:24:34.822611    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/40b79195-6d51-4041-9cbb-5acc6f1e7345-openshift-service-ca\") pod \"maas-api-e2e-extmod-b-9332a5-5bd6d599b-t2454\" (UID: \"40b79195-6d51-4041-9cbb-5acc6f1e7345\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-9332a5-5bd6d599b-t2454"
Oct 31 01:24:34.824564 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:24:34.824540    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/40b79195-6d51-4041-9cbb-5acc6f1e7345-metrics-certs\") pod \"maas-api-e2e-extmod-b-9332a5-5bd6d599b-t2454\" (UID: \"40b79195-6d51-4041-9cbb-5acc6f1e7345\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-9332a5-5bd6d599b-t2454"
Oct 31 01:24:34.894383 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:24:34.894316    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8jmpc\" (UniqueName: \"kubernetes.io/projected/40b79195-6d51-4041-9cbb-5acc6f1e7345-kube-api-access-8jmpc\") pod \"maas-api-e2e-extmod-b-9332a5-5bd6d599b-t2454\" (UID: \"40b79195-6d51-4041-9cbb-5acc6f1e7345\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-9332a5-5bd6d599b-t2454"
Oct 31 01:24:35.055141 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:24:35.054762    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:24:35.204707 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:24:35.204670    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:24:35.351192 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:24:35.351147    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/40b79195-6d51-4041-9cbb-5acc6f1e7345-maas-api-tls\") pod \"maas-api-e2e-extmod-b-9332a5-5bd6d599b-t2454\" (UID: \"40b79195-6d51-4041-9cbb-5acc6f1e7345\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-9332a5-5bd6d599b-t2454"
Oct 31 01:24:35.353724 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:24:35.353704    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/40b79195-6d51-4041-9cbb-5acc6f1e7345-maas-api-tls\") pod \"maas-api-e2e-extmod-b-9332a5-5bd6d599b-t2454\" (UID: \"40b79195-6d51-4041-9cbb-5acc6f1e7345\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-9332a5-5bd6d599b-t2454"
Oct 31 01:24:35.546594 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:24:35.546509    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-9332a5-5bd6d599b-t2454"
Oct 31 01:24:35.940875 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:24:35.940852    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-b-9332a5-5bd6d599b-t2454"]
Oct 31 01:24:35.941656 ip-10-0-132-173 kubenswrapper[4284]: W1031 01:24:35.941630    4284 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod40b79195_6d51_4041_9cbb_5acc6f1e7345.slice/crio-d5e163e8926b8066516bed3928c90c8353ba1264a4d5851d54015b70937855d5 WatchSource:0}: Error finding container d5e163e8926b8066516bed3928c90c8353ba1264a4d5851d54015b70937855d5: Status 404 returned error can't find the container with id d5e163e8926b8066516bed3928c90c8353ba1264a4d5851d54015b70937855d5
Oct 31 01:24:36.573999 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:24:36.573960    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-9332a5-5bd6d599b-t2454" event={"ID":"40b79195-6d51-4041-9cbb-5acc6f1e7345","Type":"ContainerStarted","Data":"d5e163e8926b8066516bed3928c90c8353ba1264a4d5851d54015b70937855d5"}
Oct 31 01:24:37.581891 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:24:37.581856    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-9332a5-5bd6d599b-t2454" event={"ID":"40b79195-6d51-4041-9cbb-5acc6f1e7345","Type":"ContainerStarted","Data":"9e768c9b62eb3cef7402630ab5bea547457dc52b05a44e4fa76dfefb0cd4b2f2"}
Oct 31 01:24:37.582433 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:24:37.581913    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-9332a5-5bd6d599b-t2454"
Oct 31 01:24:37.707540 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:24:37.704685    4284 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-9332a5-5bd6d599b-t2454" podStartSLOduration=2.853152748 podStartE2EDuration="3.704665111s" totalImagesPullingTime="851.512363ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 01:24:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 01:24:37.701944716 +0000 UTC m=+3391.846951634" watchObservedRunningTime="2026-10-31 01:24:37.704665111 +0000 UTC m=+3391.849672028"
Oct 31 01:24:39.964724 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:24:39.964688    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:24:43.593893 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:24:43.593863    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-9332a5-5bd6d599b-t2454"
Oct 31 01:24:44.793474 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:24:44.793438    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:24:47.016430 ip-10-0-132-173 kubenswrapper[4284]: E1031 01:24:47.016394    4284 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fd8sp" containerName="extractor"
Oct 31 01:24:49.716873 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:24:49.716835    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:24:54.731575 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:24:54.731540    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:24:59.939988 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:24:59.939938    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:25:03.779149 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:25:03.779115    4284 generic.go:356] "Generic (PLEG): container finished" podID="04e2ad9d-c5b9-47b0-ba97-86114775393c" containerID="549889e4e2ee33f009fd5e45398cc88baf53ae9706354a3075d755a1b45735e4" exitCode=0
Oct 31 01:25:03.779638 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:25:03.779188    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-08fcc4-openshift-default-7985d77b8f-t22ds" event={"ID":"04e2ad9d-c5b9-47b0-ba97-86114775393c","Type":"ContainerDied","Data":"549889e4e2ee33f009fd5e45398cc88baf53ae9706354a3075d755a1b45735e4"}
Oct 31 01:25:03.779686 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:25:03.779648    4284 scope.go:122] "RemoveContainer" containerID="549889e4e2ee33f009fd5e45398cc88baf53ae9706354a3075d755a1b45735e4"
Oct 31 01:25:03.781378 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:25:03.781350    4284 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 01:25:03.781462 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:25:03.781423    4284 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 01:25:03.781462 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:25:03.781452    4284 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 01:25:04.809812 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:25:04.809771    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-08fcc4-openshift-default-7985d77b8f-t22ds" event={"ID":"04e2ad9d-c5b9-47b0-ba97-86114775393c","Type":"ContainerStarted","Data":"308a3655c034978cc676a0a87c53e6476371fba9cb33992a37d64198792752c9"}
Oct 31 01:25:04.982900 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:25:04.982864    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:25:05.661672 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:25:05.661628    4284 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-extmod-a-08fcc4-openshift-default-7985d77b8f-t22ds"
Oct 31 01:25:05.662797 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:25:05.662774    4284 patch_prober.go:29] interesting pod/e2e-extmod-a-08fcc4-openshift-default-7985d77b8f-t22ds container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.186:15021/healthz/ready\": dial tcp 10.134.0.186:15021: connect: connection refused" start-of-body=
Oct 31 01:25:05.662868 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:25:05.662825    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-08fcc4-openshift-default-7985d77b8f-t22ds" podUID="04e2ad9d-c5b9-47b0-ba97-86114775393c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.186:15021/healthz/ready\": dial tcp 10.134.0.186:15021: connect: connection refused"
Oct 31 01:25:06.661301 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:25:06.661261    4284 patch_prober.go:29] interesting pod/e2e-extmod-a-08fcc4-openshift-default-7985d77b8f-t22ds container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.186:15021/healthz/ready\": dial tcp 10.134.0.186:15021: connect: connection refused" start-of-body=
Oct 31 01:25:06.661718 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:25:06.661317    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-08fcc4-openshift-default-7985d77b8f-t22ds" podUID="04e2ad9d-c5b9-47b0-ba97-86114775393c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.186:15021/healthz/ready\": dial tcp 10.134.0.186:15021: connect: connection refused"
Oct 31 01:25:06.791347 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:25:06.791311    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-extmod-a-08fcc4-openshift-default-7985d77b8f-t22ds"
Oct 31 01:25:07.661612 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:25:07.661572    4284 patch_prober.go:29] interesting pod/e2e-extmod-a-08fcc4-openshift-default-7985d77b8f-t22ds container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.186:15021/healthz/ready\": dial tcp 10.134.0.186:15021: connect: connection refused" start-of-body=
Oct 31 01:25:07.662021 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:25:07.661660    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-08fcc4-openshift-default-7985d77b8f-t22ds" podUID="04e2ad9d-c5b9-47b0-ba97-86114775393c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.186:15021/healthz/ready\": dial tcp 10.134.0.186:15021: connect: connection refused"
Oct 31 01:25:08.011364 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:25:08.011277    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:25:08.338071 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:25:08.337991    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-a-3aca3f-554dd84857-59rv6"]
Oct 31 01:25:08.460094 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:25:08.460058    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-a-3aca3f-67994c4bc5-hwmd2"]
Oct 31 01:25:08.661603 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:25:08.661573    4284 patch_prober.go:29] interesting pod/e2e-extmod-a-08fcc4-openshift-default-7985d77b8f-t22ds container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.186:15021/healthz/ready\": dial tcp 10.134.0.186:15021: connect: connection refused" start-of-body=
Oct 31 01:25:08.661774 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:25:08.661627    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-08fcc4-openshift-default-7985d77b8f-t22ds" podUID="04e2ad9d-c5b9-47b0-ba97-86114775393c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.186:15021/healthz/ready\": dial tcp 10.134.0.186:15021: connect: connection refused"
Oct 31 01:25:08.710790 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:25:08.710742    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:25:08.861691 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:25:08.861641    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:25:09.661586 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:25:09.661553    4284 patch_prober.go:29] interesting pod/e2e-extmod-a-08fcc4-openshift-default-7985d77b8f-t22ds container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.186:15021/healthz/ready\": dial tcp 10.134.0.186:15021: connect: connection refused" start-of-body=
Oct 31 01:25:09.661787 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:25:09.661611    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-08fcc4-openshift-default-7985d77b8f-t22ds" podUID="04e2ad9d-c5b9-47b0-ba97-86114775393c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.186:15021/healthz/ready\": dial tcp 10.134.0.186:15021: connect: connection refused"
Oct 31 01:25:09.808606 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:25:09.808562    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:25:10.667105 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:25:10.667074    4284 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-extmod-a-08fcc4-openshift-default-7985d77b8f-t22ds"
Oct 31 01:25:10.668074 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:25:10.668053    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-extmod-a-08fcc4-openshift-default-7985d77b8f-t22ds"
Oct 31 01:25:13.338570 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:25:13.338512    4284 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-extmod-a-3aca3f-554dd84857-59rv6" podUID="f31cff38-07af-4d6c-83ba-fcc15b830cbc" containerName="payload-processing" containerID="cri-o://1e2a1519420a7ab584361faf5ec11a0db28214841e72854c92d4546e4304d730" gracePeriod=25
Oct 31 01:25:13.461364 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:25:13.461324    4284 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-3aca3f-67994c4bc5-hwmd2" podUID="16fc298a-4f7c-4efd-ac33-5ac97b7d3275" containerName="payload-pre-processing" containerID="cri-o://7138bb9c84157a17fdcdd52579a072a4c17efaae3a7bebeea3b093e8a992f0f9" gracePeriod=25
Oct 31 01:25:13.611011 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:25:13.610977    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-extmod-a-3aca3f-554dd84857-59rv6"
Oct 31 01:25:13.697795 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:25:13.697756    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f31cff38-07af-4d6c-83ba-fcc15b830cbc-kube-api-access-65jlb\" (UniqueName: \"kubernetes.io/projected/f31cff38-07af-4d6c-83ba-fcc15b830cbc-kube-api-access-65jlb\") pod \"f31cff38-07af-4d6c-83ba-fcc15b830cbc\" (UID: \"f31cff38-07af-4d6c-83ba-fcc15b830cbc\") "
Oct 31 01:25:13.697987 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:25:13.697827    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f31cff38-07af-4d6c-83ba-fcc15b830cbc-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f31cff38-07af-4d6c-83ba-fcc15b830cbc-plugins-config-volume\") pod \"f31cff38-07af-4d6c-83ba-fcc15b830cbc\" (UID: \"f31cff38-07af-4d6c-83ba-fcc15b830cbc\") "
Oct 31 01:25:13.698230 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:25:13.698193    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f31cff38-07af-4d6c-83ba-fcc15b830cbc-plugins-config-volume" pod "f31cff38-07af-4d6c-83ba-fcc15b830cbc" (UID: "f31cff38-07af-4d6c-83ba-fcc15b830cbc"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 01:25:13.698422 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:25:13.698393    4284 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f31cff38-07af-4d6c-83ba-fcc15b830cbc-plugins-config-volume\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:25:13.700062 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:25:13.700039    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f31cff38-07af-4d6c-83ba-fcc15b830cbc-kube-api-access-65jlb" pod "f31cff38-07af-4d6c-83ba-fcc15b830cbc" (UID: "f31cff38-07af-4d6c-83ba-fcc15b830cbc"). InnerVolumeSpecName "kube-api-access-65jlb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:25:13.729881 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:25:13.729844    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-3aca3f-67994c4bc5-hwmd2"
Oct 31 01:25:13.799572 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:25:13.799539    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/16fc298a-4f7c-4efd-ac33-5ac97b7d3275-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/16fc298a-4f7c-4efd-ac33-5ac97b7d3275-plugins-config-volume\") pod \"16fc298a-4f7c-4efd-ac33-5ac97b7d3275\" (UID: \"16fc298a-4f7c-4efd-ac33-5ac97b7d3275\") "
Oct 31 01:25:13.799743 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:25:13.799654    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/16fc298a-4f7c-4efd-ac33-5ac97b7d3275-kube-api-access-2wwhj\" (UniqueName: \"kubernetes.io/projected/16fc298a-4f7c-4efd-ac33-5ac97b7d3275-kube-api-access-2wwhj\") pod \"16fc298a-4f7c-4efd-ac33-5ac97b7d3275\" (UID: \"16fc298a-4f7c-4efd-ac33-5ac97b7d3275\") "
Oct 31 01:25:13.799966 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:25:13.799942    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/16fc298a-4f7c-4efd-ac33-5ac97b7d3275-plugins-config-volume" pod "16fc298a-4f7c-4efd-ac33-5ac97b7d3275" (UID: "16fc298a-4f7c-4efd-ac33-5ac97b7d3275"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 01:25:13.800100 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:25:13.800085    4284 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/16fc298a-4f7c-4efd-ac33-5ac97b7d3275-plugins-config-volume\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:25:13.800151 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:25:13.800106    4284 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-65jlb\" (UniqueName: \"kubernetes.io/projected/f31cff38-07af-4d6c-83ba-fcc15b830cbc-kube-api-access-65jlb\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:25:13.801865 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:25:13.801843    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/16fc298a-4f7c-4efd-ac33-5ac97b7d3275-kube-api-access-2wwhj" pod "16fc298a-4f7c-4efd-ac33-5ac97b7d3275" (UID: "16fc298a-4f7c-4efd-ac33-5ac97b7d3275"). InnerVolumeSpecName "kube-api-access-2wwhj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:25:13.880264 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:25:13.880142    4284 generic.go:356] "Generic (PLEG): container finished" podID="16fc298a-4f7c-4efd-ac33-5ac97b7d3275" containerID="7138bb9c84157a17fdcdd52579a072a4c17efaae3a7bebeea3b093e8a992f0f9" exitCode=0
Oct 31 01:25:13.880264 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:25:13.880193    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-3aca3f-67994c4bc5-hwmd2"
Oct 31 01:25:13.880264 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:25:13.880249    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-3aca3f-67994c4bc5-hwmd2" event={"ID":"16fc298a-4f7c-4efd-ac33-5ac97b7d3275","Type":"ContainerDied","Data":"7138bb9c84157a17fdcdd52579a072a4c17efaae3a7bebeea3b093e8a992f0f9"}
Oct 31 01:25:13.880600 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:25:13.880307    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-3aca3f-67994c4bc5-hwmd2" event={"ID":"16fc298a-4f7c-4efd-ac33-5ac97b7d3275","Type":"ContainerDied","Data":"5499da23d0bcae83e804d85f7ac4d0f75473f28b28d2280cecbffc68a7df6e57"}
Oct 31 01:25:13.880671 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:25:13.880652    4284 scope.go:122] "RemoveContainer" containerID="7138bb9c84157a17fdcdd52579a072a4c17efaae3a7bebeea3b093e8a992f0f9"
Oct 31 01:25:13.882259 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:25:13.882179    4284 generic.go:356] "Generic (PLEG): container finished" podID="f31cff38-07af-4d6c-83ba-fcc15b830cbc" containerID="1e2a1519420a7ab584361faf5ec11a0db28214841e72854c92d4546e4304d730" exitCode=0
Oct 31 01:25:13.882352 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:25:13.882291    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-extmod-a-3aca3f-554dd84857-59rv6" event={"ID":"f31cff38-07af-4d6c-83ba-fcc15b830cbc","Type":"ContainerDied","Data":"1e2a1519420a7ab584361faf5ec11a0db28214841e72854c92d4546e4304d730"}
Oct 31 01:25:13.882352 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:25:13.882332    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-extmod-a-3aca3f-554dd84857-59rv6" event={"ID":"f31cff38-07af-4d6c-83ba-fcc15b830cbc","Type":"ContainerDied","Data":"2d65d6c97cf60e16ed20ae4539ec1b062d94c28aa5b37550718667db1486488d"}
Oct 31 01:25:13.882473 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:25:13.882396    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-extmod-a-3aca3f-554dd84857-59rv6"
Oct 31 01:25:13.894031 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:25:13.894014    4284 scope.go:122] "RemoveContainer" containerID="7138bb9c84157a17fdcdd52579a072a4c17efaae3a7bebeea3b093e8a992f0f9"
Oct 31 01:25:13.894391 ip-10-0-132-173 kubenswrapper[4284]: E1031 01:25:13.894370    4284 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7138bb9c84157a17fdcdd52579a072a4c17efaae3a7bebeea3b093e8a992f0f9\": container with ID starting with 7138bb9c84157a17fdcdd52579a072a4c17efaae3a7bebeea3b093e8a992f0f9 not found: ID does not exist" containerID="7138bb9c84157a17fdcdd52579a072a4c17efaae3a7bebeea3b093e8a992f0f9"
Oct 31 01:25:13.894491 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:25:13.894397    4284 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7138bb9c84157a17fdcdd52579a072a4c17efaae3a7bebeea3b093e8a992f0f9"} err="failed to get container status \"7138bb9c84157a17fdcdd52579a072a4c17efaae3a7bebeea3b093e8a992f0f9\": rpc error: code = NotFound desc = could not find container \"7138bb9c84157a17fdcdd52579a072a4c17efaae3a7bebeea3b093e8a992f0f9\": container with ID starting with 7138bb9c84157a17fdcdd52579a072a4c17efaae3a7bebeea3b093e8a992f0f9 not found: ID does not exist"
Oct 31 01:25:13.894491 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:25:13.894416    4284 scope.go:122] "RemoveContainer" containerID="1e2a1519420a7ab584361faf5ec11a0db28214841e72854c92d4546e4304d730"
Oct 31 01:25:13.901194 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:25:13.901166    4284 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2wwhj\" (UniqueName: \"kubernetes.io/projected/16fc298a-4f7c-4efd-ac33-5ac97b7d3275-kube-api-access-2wwhj\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:25:13.903360 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:25:13.903342    4284 scope.go:122] "RemoveContainer" containerID="1e2a1519420a7ab584361faf5ec11a0db28214841e72854c92d4546e4304d730"
Oct 31 01:25:13.903625 ip-10-0-132-173 kubenswrapper[4284]: E1031 01:25:13.903605    4284 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1e2a1519420a7ab584361faf5ec11a0db28214841e72854c92d4546e4304d730\": container with ID starting with 1e2a1519420a7ab584361faf5ec11a0db28214841e72854c92d4546e4304d730 not found: ID does not exist" containerID="1e2a1519420a7ab584361faf5ec11a0db28214841e72854c92d4546e4304d730"
Oct 31 01:25:13.903700 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:25:13.903634    4284 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1e2a1519420a7ab584361faf5ec11a0db28214841e72854c92d4546e4304d730"} err="failed to get container status \"1e2a1519420a7ab584361faf5ec11a0db28214841e72854c92d4546e4304d730\": rpc error: code = NotFound desc = could not find container \"1e2a1519420a7ab584361faf5ec11a0db28214841e72854c92d4546e4304d730\": container with ID starting with 1e2a1519420a7ab584361faf5ec11a0db28214841e72854c92d4546e4304d730 not found: ID does not exist"
Oct 31 01:25:14.039122 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:25:14.039091    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-a-3aca3f-67994c4bc5-hwmd2"]
Oct 31 01:25:14.043339 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:25:14.043310    4284 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-a-3aca3f-67994c4bc5-hwmd2"]
Oct 31 01:25:14.148056 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:25:14.147974    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-a-3aca3f-554dd84857-59rv6"]
Oct 31 01:25:14.167977 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:25:14.167943    4284 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-a-3aca3f-554dd84857-59rv6"]
Oct 31 01:25:14.576366 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:25:14.576318    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:25:15.964087 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:25:15.964054    4284 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="16fc298a-4f7c-4efd-ac33-5ac97b7d3275" path="/var/lib/kubelet/pods/16fc298a-4f7c-4efd-ac33-5ac97b7d3275/volumes"
Oct 31 01:25:15.964497 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:25:15.964478    4284 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f31cff38-07af-4d6c-83ba-fcc15b830cbc" path="/var/lib/kubelet/pods/f31cff38-07af-4d6c-83ba-fcc15b830cbc/volumes"
Oct 31 01:25:17.013905 ip-10-0-132-173 kubenswrapper[4284]: E1031 01:25:17.013872    4284 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fd8sp" containerName="extractor"
Oct 31 01:25:18.521353 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:25:18.521323    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:25:18.677111 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:25:18.677081    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:25:19.408914 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:25:19.408873    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:25:24.283231 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:25:24.283183    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:25:29.185853 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:25:29.185809    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:25:33.941848 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:25:33.941814    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:25:39.099392 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:25:39.099358    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:25:43.905894 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:25:43.905848    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:25:44.817786 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:25:44.817747    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:25:44.964460 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:25:44.964416    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:25:47.014998 ip-10-0-132-173 kubenswrapper[4284]: E1031 01:25:47.014916    4284 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fd8sp" containerName="extractor"
Oct 31 01:25:47.026626 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:25:47.026590    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:25:48.720906 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:25:48.720862    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:25:49.794710 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:25:49.794678    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-a-d067bd-7469974c5f-22k9f"]
Oct 31 01:25:50.066912 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:25:50.066818    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:25:50.221809 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:25:50.221772    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:25:53.551896 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:25:53.551857    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:25:54.795511 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:25:54.795462    4284 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-d067bd-7469974c5f-22k9f" podUID="a71dc36e-312b-4dff-b412-056d51f65731" containerName="payload-pre-processing" containerID="cri-o://607c407d4b9760aab40202699c66e68bad42f08ba96973142df16ede4a406cdb" gracePeriod=25
Oct 31 01:25:55.069909 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:25:55.069885    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-d067bd-7469974c5f-22k9f"
Oct 31 01:25:55.185066 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:25:55.185033    4284 generic.go:356] "Generic (PLEG): container finished" podID="a71dc36e-312b-4dff-b412-056d51f65731" containerID="607c407d4b9760aab40202699c66e68bad42f08ba96973142df16ede4a406cdb" exitCode=0
Oct 31 01:25:55.185261 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:25:55.185094    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-d067bd-7469974c5f-22k9f" event={"ID":"a71dc36e-312b-4dff-b412-056d51f65731","Type":"ContainerDied","Data":"607c407d4b9760aab40202699c66e68bad42f08ba96973142df16ede4a406cdb"}
Oct 31 01:25:55.185261 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:25:55.185113    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-d067bd-7469974c5f-22k9f"
Oct 31 01:25:55.185261 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:25:55.185163    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-d067bd-7469974c5f-22k9f" event={"ID":"a71dc36e-312b-4dff-b412-056d51f65731","Type":"ContainerDied","Data":"6a9a8e3e7eff65fc8efa8679bf4a45f2446f692649af2dcb7b2fce5b9dd2eb3b"}
Oct 31 01:25:55.185261 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:25:55.185199    4284 scope.go:122] "RemoveContainer" containerID="607c407d4b9760aab40202699c66e68bad42f08ba96973142df16ede4a406cdb"
Oct 31 01:25:55.196005 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:25:55.195973    4284 scope.go:122] "RemoveContainer" containerID="607c407d4b9760aab40202699c66e68bad42f08ba96973142df16ede4a406cdb"
Oct 31 01:25:55.197441 ip-10-0-132-173 kubenswrapper[4284]: E1031 01:25:55.197410    4284 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"607c407d4b9760aab40202699c66e68bad42f08ba96973142df16ede4a406cdb\": container with ID starting with 607c407d4b9760aab40202699c66e68bad42f08ba96973142df16ede4a406cdb not found: ID does not exist" containerID="607c407d4b9760aab40202699c66e68bad42f08ba96973142df16ede4a406cdb"
Oct 31 01:25:55.197541 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:25:55.197450    4284 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"607c407d4b9760aab40202699c66e68bad42f08ba96973142df16ede4a406cdb"} err="failed to get container status \"607c407d4b9760aab40202699c66e68bad42f08ba96973142df16ede4a406cdb\": rpc error: code = NotFound desc = could not find container \"607c407d4b9760aab40202699c66e68bad42f08ba96973142df16ede4a406cdb\": container with ID starting with 607c407d4b9760aab40202699c66e68bad42f08ba96973142df16ede4a406cdb not found: ID does not exist"
Oct 31 01:25:55.221056 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:25:55.221021    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a71dc36e-312b-4dff-b412-056d51f65731-kube-api-access-t8lsq\" (UniqueName: \"kubernetes.io/projected/a71dc36e-312b-4dff-b412-056d51f65731-kube-api-access-t8lsq\") pod \"a71dc36e-312b-4dff-b412-056d51f65731\" (UID: \"a71dc36e-312b-4dff-b412-056d51f65731\") "
Oct 31 01:25:55.221309 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:25:55.221256    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a71dc36e-312b-4dff-b412-056d51f65731-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a71dc36e-312b-4dff-b412-056d51f65731-plugins-config-volume\") pod \"a71dc36e-312b-4dff-b412-056d51f65731\" (UID: \"a71dc36e-312b-4dff-b412-056d51f65731\") "
Oct 31 01:25:55.221694 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:25:55.221668    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a71dc36e-312b-4dff-b412-056d51f65731-plugins-config-volume" pod "a71dc36e-312b-4dff-b412-056d51f65731" (UID: "a71dc36e-312b-4dff-b412-056d51f65731"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 01:25:55.221981 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:25:55.221915    4284 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a71dc36e-312b-4dff-b412-056d51f65731-plugins-config-volume\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:25:55.223556 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:25:55.223527    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a71dc36e-312b-4dff-b412-056d51f65731-kube-api-access-t8lsq" pod "a71dc36e-312b-4dff-b412-056d51f65731" (UID: "a71dc36e-312b-4dff-b412-056d51f65731"). InnerVolumeSpecName "kube-api-access-t8lsq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:25:55.323615 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:25:55.323531    4284 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t8lsq\" (UniqueName: \"kubernetes.io/projected/a71dc36e-312b-4dff-b412-056d51f65731-kube-api-access-t8lsq\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:25:55.602341 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:25:55.602252    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-a-d067bd-7469974c5f-22k9f"]
Oct 31 01:25:55.605382 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:25:55.605349    4284 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-a-d067bd-7469974c5f-22k9f"]
Oct 31 01:25:55.963452 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:25:55.963422    4284 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a71dc36e-312b-4dff-b412-056d51f65731" path="/var/lib/kubelet/pods/a71dc36e-312b-4dff-b412-056d51f65731/volumes"
Oct 31 01:25:58.727717 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:25:58.727679    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:26:03.971746 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:26:03.971705    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:26:04.876415 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:26:04.876382    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:26:05.023160 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:26:05.023129    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:26:08.378447 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:26:08.378411    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:26:09.234106 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:26:09.234059    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:26:12.552234 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:26:12.552183    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-a-eb68dd-79b7d8c769-xr7cl"]
Oct 31 01:26:12.924380 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:26:12.924344    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:26:13.071266 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:26:13.071229    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:26:14.369083 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:26:14.369046    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:26:17.014739 ip-10-0-132-173 kubenswrapper[4284]: E1031 01:26:17.014701    4284 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fd8sp" containerName="extractor"
Oct 31 01:26:17.553130 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:26:17.553064    4284 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-extmod-a-eb68dd-79b7d8c769-xr7cl" podUID="b96b200a-f8de-4419-8232-061be10ada45" containerName="payload-processing" containerID="cri-o://e38b8a3fa7a0cf8b0df5b76811063dd4a0b471177ce0dbae6e0ca0c5e8086227" gracePeriod=25
Oct 31 01:26:17.816348 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:26:17.816281    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-extmod-a-eb68dd-79b7d8c769-xr7cl"
Oct 31 01:26:17.964856 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:26:17.964822    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b96b200a-f8de-4419-8232-061be10ada45-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b96b200a-f8de-4419-8232-061be10ada45-plugins-config-volume\") pod \"b96b200a-f8de-4419-8232-061be10ada45\" (UID: \"b96b200a-f8de-4419-8232-061be10ada45\") "
Oct 31 01:26:17.965031 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:26:17.964923    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b96b200a-f8de-4419-8232-061be10ada45-kube-api-access-4q2km\" (UniqueName: \"kubernetes.io/projected/b96b200a-f8de-4419-8232-061be10ada45-kube-api-access-4q2km\") pod \"b96b200a-f8de-4419-8232-061be10ada45\" (UID: \"b96b200a-f8de-4419-8232-061be10ada45\") "
Oct 31 01:26:17.965298 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:26:17.965268    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b96b200a-f8de-4419-8232-061be10ada45-plugins-config-volume" pod "b96b200a-f8de-4419-8232-061be10ada45" (UID: "b96b200a-f8de-4419-8232-061be10ada45"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 01:26:17.965602 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:26:17.965578    4284 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b96b200a-f8de-4419-8232-061be10ada45-plugins-config-volume\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:26:17.967319 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:26:17.967294    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b96b200a-f8de-4419-8232-061be10ada45-kube-api-access-4q2km" pod "b96b200a-f8de-4419-8232-061be10ada45" (UID: "b96b200a-f8de-4419-8232-061be10ada45"). InnerVolumeSpecName "kube-api-access-4q2km". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:26:18.067949 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:26:18.067767    4284 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4q2km\" (UniqueName: \"kubernetes.io/projected/b96b200a-f8de-4419-8232-061be10ada45-kube-api-access-4q2km\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:26:18.362017 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:26:18.361938    4284 generic.go:356] "Generic (PLEG): container finished" podID="b96b200a-f8de-4419-8232-061be10ada45" containerID="e38b8a3fa7a0cf8b0df5b76811063dd4a0b471177ce0dbae6e0ca0c5e8086227" exitCode=0
Oct 31 01:26:18.362017 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:26:18.362001    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-extmod-a-eb68dd-79b7d8c769-xr7cl"
Oct 31 01:26:18.362204 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:26:18.362021    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-extmod-a-eb68dd-79b7d8c769-xr7cl" event={"ID":"b96b200a-f8de-4419-8232-061be10ada45","Type":"ContainerDied","Data":"e38b8a3fa7a0cf8b0df5b76811063dd4a0b471177ce0dbae6e0ca0c5e8086227"}
Oct 31 01:26:18.362204 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:26:18.362103    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-extmod-a-eb68dd-79b7d8c769-xr7cl" event={"ID":"b96b200a-f8de-4419-8232-061be10ada45","Type":"ContainerDied","Data":"e7c11e22d3deced77b411268b320a712fd8e94c5c335d89119a8824782eae369"}
Oct 31 01:26:18.362204 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:26:18.362130    4284 scope.go:122] "RemoveContainer" containerID="e38b8a3fa7a0cf8b0df5b76811063dd4a0b471177ce0dbae6e0ca0c5e8086227"
Oct 31 01:26:18.372097 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:26:18.372077    4284 scope.go:122] "RemoveContainer" containerID="e38b8a3fa7a0cf8b0df5b76811063dd4a0b471177ce0dbae6e0ca0c5e8086227"
Oct 31 01:26:18.372450 ip-10-0-132-173 kubenswrapper[4284]: E1031 01:26:18.372430    4284 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e38b8a3fa7a0cf8b0df5b76811063dd4a0b471177ce0dbae6e0ca0c5e8086227\": container with ID starting with e38b8a3fa7a0cf8b0df5b76811063dd4a0b471177ce0dbae6e0ca0c5e8086227 not found: ID does not exist" containerID="e38b8a3fa7a0cf8b0df5b76811063dd4a0b471177ce0dbae6e0ca0c5e8086227"
Oct 31 01:26:18.372548 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:26:18.372460    4284 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e38b8a3fa7a0cf8b0df5b76811063dd4a0b471177ce0dbae6e0ca0c5e8086227"} err="failed to get container status \"e38b8a3fa7a0cf8b0df5b76811063dd4a0b471177ce0dbae6e0ca0c5e8086227\": rpc error: code = NotFound desc = could not find container \"e38b8a3fa7a0cf8b0df5b76811063dd4a0b471177ce0dbae6e0ca0c5e8086227\": container with ID starting with e38b8a3fa7a0cf8b0df5b76811063dd4a0b471177ce0dbae6e0ca0c5e8086227 not found: ID does not exist"
Oct 31 01:26:18.530422 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:26:18.530229    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-a-eb68dd-79b7d8c769-xr7cl"]
Oct 31 01:26:18.576344 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:26:18.576305    4284 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-a-eb68dd-79b7d8c769-xr7cl"]
Oct 31 01:26:19.196263 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:26:19.196204    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:26:19.963293 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:26:19.963261    4284 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b96b200a-f8de-4419-8232-061be10ada45" path="/var/lib/kubelet/pods/b96b200a-f8de-4419-8232-061be10ada45/volumes"
Oct 31 01:26:24.443205 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:26:24.443164    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:26:28.907876 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:26:28.907839    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:26:30.482578 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:26:30.482540    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:26:30.792846 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:26:30.792756    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:26:30.987565 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:26:30.987518    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:26:31.652834 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:26:31.652800    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:26:36.040631 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:26:36.040590    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:26:37.668859 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:26:37.668823    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:26:37.858080 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:26:37.858047    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:26:38.051885 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:26:38.051794    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:26:41.137072 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:26:41.137039    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:26:46.092671 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:26:46.092636    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:26:47.015394 ip-10-0-132-173 kubenswrapper[4284]: E1031 01:26:47.015357    4284 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fd8sp" containerName="extractor"
Oct 31 01:26:47.183916 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:26:47.183884    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:26:50.899153 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:26:50.899114    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:26:52.432041 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:26:52.432002    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:26:52.580515 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:26:52.580487    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:26:55.686771 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:26:55.686739    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:27:00.489938 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:27:00.489900    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:27:05.200735 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:27:05.200684    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:27:08.689675 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:27:08.689643    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:27:09.273522 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:27:09.273486    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-a-d067bd-7ff8568785-rqbm9"]
Oct 31 01:27:09.273714 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:27:09.273692    4284 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-d067bd-7ff8568785-rqbm9" podUID="178450d7-7cf3-46f9-8e25-e09b73075316" containerName="maas-api" containerID="cri-o://1b87f2b4406fba76ae112312e6e05d181992f56022288b937495e8d04fb92ba1" gracePeriod=30
Oct 31 01:27:09.373632 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:27:09.373595    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:27:09.572055 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:27:09.572031    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-d067bd-7ff8568785-rqbm9"
Oct 31 01:27:09.659176 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:27:09.658735    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/178450d7-7cf3-46f9-8e25-e09b73075316-metrics-certs\" (UniqueName: \"kubernetes.io/secret/178450d7-7cf3-46f9-8e25-e09b73075316-metrics-certs\") pod \"178450d7-7cf3-46f9-8e25-e09b73075316\" (UID: \"178450d7-7cf3-46f9-8e25-e09b73075316\") "
Oct 31 01:27:09.659519 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:27:09.659477    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/178450d7-7cf3-46f9-8e25-e09b73075316-kube-api-access-mxpg7\" (UniqueName: \"kubernetes.io/projected/178450d7-7cf3-46f9-8e25-e09b73075316-kube-api-access-mxpg7\") pod \"178450d7-7cf3-46f9-8e25-e09b73075316\" (UID: \"178450d7-7cf3-46f9-8e25-e09b73075316\") "
Oct 31 01:27:09.660081 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:27:09.660062    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/178450d7-7cf3-46f9-8e25-e09b73075316-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/178450d7-7cf3-46f9-8e25-e09b73075316-openshift-service-ca\") pod \"178450d7-7cf3-46f9-8e25-e09b73075316\" (UID: \"178450d7-7cf3-46f9-8e25-e09b73075316\") "
Oct 31 01:27:09.674242 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:27:09.666322    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/178450d7-7cf3-46f9-8e25-e09b73075316-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/178450d7-7cf3-46f9-8e25-e09b73075316-maas-api-tls\") pod \"178450d7-7cf3-46f9-8e25-e09b73075316\" (UID: \"178450d7-7cf3-46f9-8e25-e09b73075316\") "
Oct 31 01:27:09.674855 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:27:09.674825    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/178450d7-7cf3-46f9-8e25-e09b73075316-openshift-service-ca" pod "178450d7-7cf3-46f9-8e25-e09b73075316" (UID: "178450d7-7cf3-46f9-8e25-e09b73075316"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 01:27:09.685470 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:27:09.685430    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/178450d7-7cf3-46f9-8e25-e09b73075316-kube-api-access-mxpg7" pod "178450d7-7cf3-46f9-8e25-e09b73075316" (UID: "178450d7-7cf3-46f9-8e25-e09b73075316"). InnerVolumeSpecName "kube-api-access-mxpg7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:27:09.693160 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:27:09.689103    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/178450d7-7cf3-46f9-8e25-e09b73075316-metrics-certs" pod "178450d7-7cf3-46f9-8e25-e09b73075316" (UID: "178450d7-7cf3-46f9-8e25-e09b73075316"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 01:27:09.693160 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:27:09.692652    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/178450d7-7cf3-46f9-8e25-e09b73075316-maas-api-tls" pod "178450d7-7cf3-46f9-8e25-e09b73075316" (UID: "178450d7-7cf3-46f9-8e25-e09b73075316"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 01:27:09.700106 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:27:09.700068    4284 generic.go:356] "Generic (PLEG): container finished" podID="178450d7-7cf3-46f9-8e25-e09b73075316" containerID="1b87f2b4406fba76ae112312e6e05d181992f56022288b937495e8d04fb92ba1" exitCode=0
Oct 31 01:27:09.700307 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:27:09.700117    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-d067bd-7ff8568785-rqbm9"
Oct 31 01:27:09.700307 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:27:09.700142    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-d067bd-7ff8568785-rqbm9" event={"ID":"178450d7-7cf3-46f9-8e25-e09b73075316","Type":"ContainerDied","Data":"1b87f2b4406fba76ae112312e6e05d181992f56022288b937495e8d04fb92ba1"}
Oct 31 01:27:09.700307 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:27:09.700198    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-d067bd-7ff8568785-rqbm9" event={"ID":"178450d7-7cf3-46f9-8e25-e09b73075316","Type":"ContainerDied","Data":"66ec309223334d8461962a293482f24506c5ad9a7ed9a03a036fb4a734fd702a"}
Oct 31 01:27:09.700307 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:27:09.700253    4284 scope.go:122] "RemoveContainer" containerID="1b87f2b4406fba76ae112312e6e05d181992f56022288b937495e8d04fb92ba1"
Oct 31 01:27:09.712444 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:27:09.712421    4284 scope.go:122] "RemoveContainer" containerID="1b87f2b4406fba76ae112312e6e05d181992f56022288b937495e8d04fb92ba1"
Oct 31 01:27:09.712804 ip-10-0-132-173 kubenswrapper[4284]: E1031 01:27:09.712778    4284 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1b87f2b4406fba76ae112312e6e05d181992f56022288b937495e8d04fb92ba1\": container with ID starting with 1b87f2b4406fba76ae112312e6e05d181992f56022288b937495e8d04fb92ba1 not found: ID does not exist" containerID="1b87f2b4406fba76ae112312e6e05d181992f56022288b937495e8d04fb92ba1"
Oct 31 01:27:09.712893 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:27:09.712815    4284 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1b87f2b4406fba76ae112312e6e05d181992f56022288b937495e8d04fb92ba1"} err="failed to get container status \"1b87f2b4406fba76ae112312e6e05d181992f56022288b937495e8d04fb92ba1\": rpc error: code = NotFound desc = could not find container \"1b87f2b4406fba76ae112312e6e05d181992f56022288b937495e8d04fb92ba1\": container with ID starting with 1b87f2b4406fba76ae112312e6e05d181992f56022288b937495e8d04fb92ba1 not found: ID does not exist"
Oct 31 01:27:09.767857 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:27:09.767825    4284 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/178450d7-7cf3-46f9-8e25-e09b73075316-maas-api-tls\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:27:09.767857 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:27:09.767851    4284 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/178450d7-7cf3-46f9-8e25-e09b73075316-metrics-certs\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:27:09.767857 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:27:09.767861    4284 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mxpg7\" (UniqueName: \"kubernetes.io/projected/178450d7-7cf3-46f9-8e25-e09b73075316-kube-api-access-mxpg7\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:27:09.768108 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:27:09.767871    4284 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/178450d7-7cf3-46f9-8e25-e09b73075316-openshift-service-ca\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:27:09.882404 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:27:09.882311    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-a-d067bd-7ff8568785-rqbm9"]
Oct 31 01:27:09.885616 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:27:09.885595    4284 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-a-d067bd-7ff8568785-rqbm9"]
Oct 31 01:27:09.963759 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:27:09.963725    4284 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="178450d7-7cf3-46f9-8e25-e09b73075316" path="/var/lib/kubelet/pods/178450d7-7cf3-46f9-8e25-e09b73075316/volumes"
Oct 31 01:27:10.484100 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:27:10.484062    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:27:10.832853 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:27:10.832761    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:27:11.162944 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:27:11.162903    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:27:11.809310 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:27:11.809269    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:27:12.065567 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:27:12.065477    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:27:12.277412 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:27:12.277371    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:27:12.476286 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:27:12.476242    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:27:13.084921 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:27:13.084882    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:27:17.014912 ip-10-0-132-173 kubenswrapper[4284]: E1031 01:27:17.014828    4284 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fd8sp" containerName="extractor"
Oct 31 01:27:17.490958 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:27:17.490922    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:27:18.396855 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:27:18.396822    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:27:18.583885 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:27:18.583844    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:27:18.754473 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:27:18.754438    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-extmod-a-d067bd-openshift-default-766dbd44fc-7zjcd"]
Oct 31 01:27:18.754697 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:27:18.754643    4284 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-extmod-a-d067bd-openshift-default-766dbd44fc-7zjcd" podUID="6ac9b2b9-a157-4f05-8c7e-e7a2987397f6" containerName="istio-proxy" containerID="cri-o://dbaf6050821a1875f066f86ae5c41b61638802ac781c0f784cc15f1df1d56580" gracePeriod=30
Oct 31 01:27:18.762485 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:27:18.762457    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:27:19.844248 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:27:19.844194    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:27:19.994027 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:27:19.993991    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:27:20.194881 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:27:20.194845    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:27:20.757663 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:27:20.757622    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:27:21.796343 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:27:21.796293    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:27:22.229601 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:27:22.229565    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:27:22.859773 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:27:22.859740    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:27:23.117451 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:27:23.117354    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:27:23.281371 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:27:23.281337    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:27:23.490030 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:27:23.489994    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:27:23.760600 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:27:23.760498    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:27:24.077763 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:27:24.077740    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-a-d067bd-openshift-default-766dbd44fc-7zjcd"
Oct 31 01:27:24.122237 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:27:24.120345    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6ac9b2b9-a157-4f05-8c7e-e7a2987397f6-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6ac9b2b9-a157-4f05-8c7e-e7a2987397f6-credential-socket\") pod \"6ac9b2b9-a157-4f05-8c7e-e7a2987397f6\" (UID: \"6ac9b2b9-a157-4f05-8c7e-e7a2987397f6\") "
Oct 31 01:27:24.122237 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:27:24.120497    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6ac9b2b9-a157-4f05-8c7e-e7a2987397f6-istio-token\" (UniqueName: \"kubernetes.io/projected/6ac9b2b9-a157-4f05-8c7e-e7a2987397f6-istio-token\") pod \"6ac9b2b9-a157-4f05-8c7e-e7a2987397f6\" (UID: \"6ac9b2b9-a157-4f05-8c7e-e7a2987397f6\") "
Oct 31 01:27:24.122237 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:27:24.120549    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/6ac9b2b9-a157-4f05-8c7e-e7a2987397f6-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6ac9b2b9-a157-4f05-8c7e-e7a2987397f6-istiod-ca-cert\") pod \"6ac9b2b9-a157-4f05-8c7e-e7a2987397f6\" (UID: \"6ac9b2b9-a157-4f05-8c7e-e7a2987397f6\") "
Oct 31 01:27:24.122237 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:27:24.120617    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6ac9b2b9-a157-4f05-8c7e-e7a2987397f6-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6ac9b2b9-a157-4f05-8c7e-e7a2987397f6-istio-data\") pod \"6ac9b2b9-a157-4f05-8c7e-e7a2987397f6\" (UID: \"6ac9b2b9-a157-4f05-8c7e-e7a2987397f6\") "
Oct 31 01:27:24.122237 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:27:24.120679    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6ac9b2b9-a157-4f05-8c7e-e7a2987397f6-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6ac9b2b9-a157-4f05-8c7e-e7a2987397f6-workload-socket\") pod \"6ac9b2b9-a157-4f05-8c7e-e7a2987397f6\" (UID: \"6ac9b2b9-a157-4f05-8c7e-e7a2987397f6\") "
Oct 31 01:27:24.122237 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:27:24.120737    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6ac9b2b9-a157-4f05-8c7e-e7a2987397f6-kube-api-access-2pgzz\" (UniqueName: \"kubernetes.io/projected/6ac9b2b9-a157-4f05-8c7e-e7a2987397f6-kube-api-access-2pgzz\") pod \"6ac9b2b9-a157-4f05-8c7e-e7a2987397f6\" (UID: \"6ac9b2b9-a157-4f05-8c7e-e7a2987397f6\") "
Oct 31 01:27:24.122237 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:27:24.120770    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6ac9b2b9-a157-4f05-8c7e-e7a2987397f6-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6ac9b2b9-a157-4f05-8c7e-e7a2987397f6-workload-certs\") pod \"6ac9b2b9-a157-4f05-8c7e-e7a2987397f6\" (UID: \"6ac9b2b9-a157-4f05-8c7e-e7a2987397f6\") "
Oct 31 01:27:24.122237 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:27:24.120796    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/6ac9b2b9-a157-4f05-8c7e-e7a2987397f6-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6ac9b2b9-a157-4f05-8c7e-e7a2987397f6-istio-podinfo\") pod \"6ac9b2b9-a157-4f05-8c7e-e7a2987397f6\" (UID: \"6ac9b2b9-a157-4f05-8c7e-e7a2987397f6\") "
Oct 31 01:27:24.122237 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:27:24.120839    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6ac9b2b9-a157-4f05-8c7e-e7a2987397f6-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6ac9b2b9-a157-4f05-8c7e-e7a2987397f6-istio-envoy\") pod \"6ac9b2b9-a157-4f05-8c7e-e7a2987397f6\" (UID: \"6ac9b2b9-a157-4f05-8c7e-e7a2987397f6\") "
Oct 31 01:27:24.139235 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:27:24.133714    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6ac9b2b9-a157-4f05-8c7e-e7a2987397f6-workload-certs" pod "6ac9b2b9-a157-4f05-8c7e-e7a2987397f6" (UID: "6ac9b2b9-a157-4f05-8c7e-e7a2987397f6"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:27:24.139235 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:27:24.134101    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6ac9b2b9-a157-4f05-8c7e-e7a2987397f6-istio-data" pod "6ac9b2b9-a157-4f05-8c7e-e7a2987397f6" (UID: "6ac9b2b9-a157-4f05-8c7e-e7a2987397f6"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:27:24.139235 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:27:24.134461    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6ac9b2b9-a157-4f05-8c7e-e7a2987397f6-istiod-ca-cert" pod "6ac9b2b9-a157-4f05-8c7e-e7a2987397f6" (UID: "6ac9b2b9-a157-4f05-8c7e-e7a2987397f6"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 01:27:24.139235 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:27:24.134683    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6ac9b2b9-a157-4f05-8c7e-e7a2987397f6-credential-socket" pod "6ac9b2b9-a157-4f05-8c7e-e7a2987397f6" (UID: "6ac9b2b9-a157-4f05-8c7e-e7a2987397f6"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:27:24.139235 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:27:24.134847    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6ac9b2b9-a157-4f05-8c7e-e7a2987397f6-workload-socket" pod "6ac9b2b9-a157-4f05-8c7e-e7a2987397f6" (UID: "6ac9b2b9-a157-4f05-8c7e-e7a2987397f6"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:27:24.200913 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:27:24.200656    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/6ac9b2b9-a157-4f05-8c7e-e7a2987397f6-istio-podinfo" pod "6ac9b2b9-a157-4f05-8c7e-e7a2987397f6" (UID: "6ac9b2b9-a157-4f05-8c7e-e7a2987397f6"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 01:27:24.201467 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:27:24.201267    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6ac9b2b9-a157-4f05-8c7e-e7a2987397f6-istio-envoy" pod "6ac9b2b9-a157-4f05-8c7e-e7a2987397f6" (UID: "6ac9b2b9-a157-4f05-8c7e-e7a2987397f6"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:27:24.201467 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:27:24.201310    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6ac9b2b9-a157-4f05-8c7e-e7a2987397f6-kube-api-access-2pgzz" pod "6ac9b2b9-a157-4f05-8c7e-e7a2987397f6" (UID: "6ac9b2b9-a157-4f05-8c7e-e7a2987397f6"). InnerVolumeSpecName "kube-api-access-2pgzz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:27:24.203281 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:27:24.203252    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6ac9b2b9-a157-4f05-8c7e-e7a2987397f6-istio-token" pod "6ac9b2b9-a157-4f05-8c7e-e7a2987397f6" (UID: "6ac9b2b9-a157-4f05-8c7e-e7a2987397f6"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:27:24.222046 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:27:24.222015    4284 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2pgzz\" (UniqueName: \"kubernetes.io/projected/6ac9b2b9-a157-4f05-8c7e-e7a2987397f6-kube-api-access-2pgzz\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:27:24.222046 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:27:24.222046    4284 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6ac9b2b9-a157-4f05-8c7e-e7a2987397f6-workload-certs\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:27:24.222275 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:27:24.222060    4284 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6ac9b2b9-a157-4f05-8c7e-e7a2987397f6-istio-podinfo\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:27:24.222275 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:27:24.222071    4284 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6ac9b2b9-a157-4f05-8c7e-e7a2987397f6-istio-envoy\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:27:24.222275 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:27:24.222084    4284 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6ac9b2b9-a157-4f05-8c7e-e7a2987397f6-credential-socket\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:27:24.222275 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:27:24.222096    4284 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6ac9b2b9-a157-4f05-8c7e-e7a2987397f6-istio-token\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:27:24.222275 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:27:24.222107    4284 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6ac9b2b9-a157-4f05-8c7e-e7a2987397f6-istiod-ca-cert\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:27:24.222275 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:27:24.222118    4284 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6ac9b2b9-a157-4f05-8c7e-e7a2987397f6-istio-data\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:27:24.222275 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:27:24.222131    4284 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6ac9b2b9-a157-4f05-8c7e-e7a2987397f6-workload-socket\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:27:24.408239 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:27:24.408173    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:27:24.796556 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:27:24.796526    4284 generic.go:356] "Generic (PLEG): container finished" podID="6ac9b2b9-a157-4f05-8c7e-e7a2987397f6" containerID="dbaf6050821a1875f066f86ae5c41b61638802ac781c0f784cc15f1df1d56580" exitCode=0
Oct 31 01:27:24.796724 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:27:24.796604    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-a-d067bd-openshift-default-766dbd44fc-7zjcd"
Oct 31 01:27:24.796724 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:27:24.796601    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-d067bd-openshift-default-766dbd44fc-7zjcd" event={"ID":"6ac9b2b9-a157-4f05-8c7e-e7a2987397f6","Type":"ContainerDied","Data":"dbaf6050821a1875f066f86ae5c41b61638802ac781c0f784cc15f1df1d56580"}
Oct 31 01:27:24.796724 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:27:24.796722    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-d067bd-openshift-default-766dbd44fc-7zjcd" event={"ID":"6ac9b2b9-a157-4f05-8c7e-e7a2987397f6","Type":"ContainerDied","Data":"ccb855dbe24faa26a511e2f7328734903e4ba5a06a61dc1a0ed1117e4073045f"}
Oct 31 01:27:24.796865 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:27:24.796744    4284 scope.go:122] "RemoveContainer" containerID="dbaf6050821a1875f066f86ae5c41b61638802ac781c0f784cc15f1df1d56580"
Oct 31 01:27:24.801397 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:27:24.801373    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:27:24.808024 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:27:24.807998    4284 scope.go:122] "RemoveContainer" containerID="58899a926296d61cc7d889798b19a5ec9104930e416b0c0ab3c601e7b9acd728"
Oct 31 01:27:24.817987 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:27:24.817971    4284 scope.go:122] "RemoveContainer" containerID="dbaf6050821a1875f066f86ae5c41b61638802ac781c0f784cc15f1df1d56580"
Oct 31 01:27:24.818256 ip-10-0-132-173 kubenswrapper[4284]: E1031 01:27:24.818237    4284 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dbaf6050821a1875f066f86ae5c41b61638802ac781c0f784cc15f1df1d56580\": container with ID starting with dbaf6050821a1875f066f86ae5c41b61638802ac781c0f784cc15f1df1d56580 not found: ID does not exist" containerID="dbaf6050821a1875f066f86ae5c41b61638802ac781c0f784cc15f1df1d56580"
Oct 31 01:27:24.818302 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:27:24.818266    4284 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dbaf6050821a1875f066f86ae5c41b61638802ac781c0f784cc15f1df1d56580"} err="failed to get container status \"dbaf6050821a1875f066f86ae5c41b61638802ac781c0f784cc15f1df1d56580\": rpc error: code = NotFound desc = could not find container \"dbaf6050821a1875f066f86ae5c41b61638802ac781c0f784cc15f1df1d56580\": container with ID starting with dbaf6050821a1875f066f86ae5c41b61638802ac781c0f784cc15f1df1d56580 not found: ID does not exist"
Oct 31 01:27:24.818302 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:27:24.818283    4284 scope.go:122] "RemoveContainer" containerID="58899a926296d61cc7d889798b19a5ec9104930e416b0c0ab3c601e7b9acd728"
Oct 31 01:27:24.818519 ip-10-0-132-173 kubenswrapper[4284]: E1031 01:27:24.818504    4284 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"58899a926296d61cc7d889798b19a5ec9104930e416b0c0ab3c601e7b9acd728\": container with ID starting with 58899a926296d61cc7d889798b19a5ec9104930e416b0c0ab3c601e7b9acd728 not found: ID does not exist" containerID="58899a926296d61cc7d889798b19a5ec9104930e416b0c0ab3c601e7b9acd728"
Oct 31 01:27:24.818558 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:27:24.818525    4284 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"58899a926296d61cc7d889798b19a5ec9104930e416b0c0ab3c601e7b9acd728"} err="failed to get container status \"58899a926296d61cc7d889798b19a5ec9104930e416b0c0ab3c601e7b9acd728\": rpc error: code = NotFound desc = could not find container \"58899a926296d61cc7d889798b19a5ec9104930e416b0c0ab3c601e7b9acd728\": container with ID starting with 58899a926296d61cc7d889798b19a5ec9104930e416b0c0ab3c601e7b9acd728 not found: ID does not exist"
Oct 31 01:27:24.955925 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:27:24.955893    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-extmod-a-d067bd-openshift-default-766dbd44fc-7zjcd"]
Oct 31 01:27:25.005139 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:27:25.005104    4284 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-extmod-a-d067bd-openshift-default-766dbd44fc-7zjcd"]
Oct 31 01:27:25.014335 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:27:25.014303    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:27:25.159281 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:27:25.159239    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:27:25.808029 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:27:25.807993    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:27:25.952649 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:27:25.952610    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:27:25.964075 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:27:25.964043    4284 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6ac9b2b9-a157-4f05-8c7e-e7a2987397f6" path="/var/lib/kubelet/pods/6ac9b2b9-a157-4f05-8c7e-e7a2987397f6/volumes"
Oct 31 01:27:26.947929 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:27:26.947891    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:27:27.171634 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:27:27.171600    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:27:27.323657 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:27:27.323542    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:27:30.132457 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:27:30.132427    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:27:30.275334 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:27:30.275298    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:27:30.438936 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:27:30.438886    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:27:31.851704 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:27:31.851668    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:27:32.028919 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:27:32.028879    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:27:32.205428 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:27:32.205374    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:27:32.210278 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:27:32.210250    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-extmod-a-eb68dd-openshift-default-98b48c6cc-gfz48"]
Oct 31 01:27:32.210538 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:27:32.210516    4284 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-extmod-a-eb68dd-openshift-default-98b48c6cc-gfz48" podUID="1dde1271-7b40-4dc0-97ca-c1be6b89c477" containerName="istio-proxy" containerID="cri-o://7010ec84037e565c88b766656f2b922c7077c92442970dde7646b0425199d25d" gracePeriod=30
Oct 31 01:27:32.830466 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:27:32.830429    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:27:32.984976 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:27:32.984946    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:27:33.185362 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:27:33.185325    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:27:33.194769 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:27:33.194746    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-extmod-a-08fcc4-openshift-default-7985d77b8f-t22ds"]
Oct 31 01:27:33.194961 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:27:33.194944    4284 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-extmod-a-08fcc4-openshift-default-7985d77b8f-t22ds" podUID="04e2ad9d-c5b9-47b0-ba97-86114775393c" containerName="istio-proxy" containerID="cri-o://308a3655c034978cc676a0a87c53e6476371fba9cb33992a37d64198792752c9" gracePeriod=30
Oct 31 01:27:34.596895 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:27:34.596861    4284 patch_prober.go:29] interesting pod/e2e-extmod-a-eb68dd-openshift-default-98b48c6cc-gfz48 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 31 01:27:34.597323 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:27:34.596919    4284 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-extmod-a-eb68dd-openshift-default-98b48c6cc-gfz48" podUID="1dde1271-7b40-4dc0-97ca-c1be6b89c477" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 31 01:27:35.112253 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:27:35.112202    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w3-dd98f0-7cf4455d8c-979g4"]
Oct 31 01:27:35.317956 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:27:35.317920    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:27:35.578006 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:27:35.577956    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:27:35.750268 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:27:35.750230    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:27:37.475325 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:27:37.475269    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-a-eb68dd-openshift-default-98b48c6cc-gfz48"
Oct 31 01:27:37.548005 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:27:37.547965    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1dde1271-7b40-4dc0-97ca-c1be6b89c477-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1dde1271-7b40-4dc0-97ca-c1be6b89c477-workload-socket\") pod \"1dde1271-7b40-4dc0-97ca-c1be6b89c477\" (UID: \"1dde1271-7b40-4dc0-97ca-c1be6b89c477\") "
Oct 31 01:27:37.548203 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:27:37.548042    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/1dde1271-7b40-4dc0-97ca-c1be6b89c477-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1dde1271-7b40-4dc0-97ca-c1be6b89c477-istio-podinfo\") pod \"1dde1271-7b40-4dc0-97ca-c1be6b89c477\" (UID: \"1dde1271-7b40-4dc0-97ca-c1be6b89c477\") "
Oct 31 01:27:37.548203 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:27:37.548088    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1dde1271-7b40-4dc0-97ca-c1be6b89c477-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1dde1271-7b40-4dc0-97ca-c1be6b89c477-istio-envoy\") pod \"1dde1271-7b40-4dc0-97ca-c1be6b89c477\" (UID: \"1dde1271-7b40-4dc0-97ca-c1be6b89c477\") "
Oct 31 01:27:37.548203 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:27:37.548144    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1dde1271-7b40-4dc0-97ca-c1be6b89c477-istio-token\" (UniqueName: \"kubernetes.io/projected/1dde1271-7b40-4dc0-97ca-c1be6b89c477-istio-token\") pod \"1dde1271-7b40-4dc0-97ca-c1be6b89c477\" (UID: \"1dde1271-7b40-4dc0-97ca-c1be6b89c477\") "
Oct 31 01:27:37.548203 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:27:37.548187    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1dde1271-7b40-4dc0-97ca-c1be6b89c477-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1dde1271-7b40-4dc0-97ca-c1be6b89c477-istio-data\") pod \"1dde1271-7b40-4dc0-97ca-c1be6b89c477\" (UID: \"1dde1271-7b40-4dc0-97ca-c1be6b89c477\") "
Oct 31 01:27:37.548476 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:27:37.548318    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1dde1271-7b40-4dc0-97ca-c1be6b89c477-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1dde1271-7b40-4dc0-97ca-c1be6b89c477-istiod-ca-cert\") pod \"1dde1271-7b40-4dc0-97ca-c1be6b89c477\" (UID: \"1dde1271-7b40-4dc0-97ca-c1be6b89c477\") "
Oct 31 01:27:37.548476 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:27:37.548386    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1dde1271-7b40-4dc0-97ca-c1be6b89c477-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1dde1271-7b40-4dc0-97ca-c1be6b89c477-credential-socket\") pod \"1dde1271-7b40-4dc0-97ca-c1be6b89c477\" (UID: \"1dde1271-7b40-4dc0-97ca-c1be6b89c477\") "
Oct 31 01:27:37.548476 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:27:37.548425    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1dde1271-7b40-4dc0-97ca-c1be6b89c477-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1dde1271-7b40-4dc0-97ca-c1be6b89c477-workload-certs\") pod \"1dde1271-7b40-4dc0-97ca-c1be6b89c477\" (UID: \"1dde1271-7b40-4dc0-97ca-c1be6b89c477\") "
Oct 31 01:27:37.548645 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:27:37.548474    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1dde1271-7b40-4dc0-97ca-c1be6b89c477-kube-api-access-xb7gk\" (UniqueName: \"kubernetes.io/projected/1dde1271-7b40-4dc0-97ca-c1be6b89c477-kube-api-access-xb7gk\") pod \"1dde1271-7b40-4dc0-97ca-c1be6b89c477\" (UID: \"1dde1271-7b40-4dc0-97ca-c1be6b89c477\") "
Oct 31 01:27:37.548645 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:27:37.548311    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1dde1271-7b40-4dc0-97ca-c1be6b89c477-workload-socket" pod "1dde1271-7b40-4dc0-97ca-c1be6b89c477" (UID: "1dde1271-7b40-4dc0-97ca-c1be6b89c477"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:27:37.548645 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:27:37.548533    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1dde1271-7b40-4dc0-97ca-c1be6b89c477-istio-data" pod "1dde1271-7b40-4dc0-97ca-c1be6b89c477" (UID: "1dde1271-7b40-4dc0-97ca-c1be6b89c477"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:27:37.548792 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:27:37.548724    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1dde1271-7b40-4dc0-97ca-c1be6b89c477-credential-socket" pod "1dde1271-7b40-4dc0-97ca-c1be6b89c477" (UID: "1dde1271-7b40-4dc0-97ca-c1be6b89c477"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:27:37.548792 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:27:37.548741    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1dde1271-7b40-4dc0-97ca-c1be6b89c477-workload-certs" pod "1dde1271-7b40-4dc0-97ca-c1be6b89c477" (UID: "1dde1271-7b40-4dc0-97ca-c1be6b89c477"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:27:37.549162 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:27:37.549140    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1dde1271-7b40-4dc0-97ca-c1be6b89c477-istiod-ca-cert" pod "1dde1271-7b40-4dc0-97ca-c1be6b89c477" (UID: "1dde1271-7b40-4dc0-97ca-c1be6b89c477"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 01:27:37.549291 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:27:37.549158    4284 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1dde1271-7b40-4dc0-97ca-c1be6b89c477-credential-socket\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:27:37.549291 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:27:37.549184    4284 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1dde1271-7b40-4dc0-97ca-c1be6b89c477-workload-certs\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:27:37.549291 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:27:37.549200    4284 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1dde1271-7b40-4dc0-97ca-c1be6b89c477-workload-socket\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:27:37.549291 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:27:37.549234    4284 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1dde1271-7b40-4dc0-97ca-c1be6b89c477-istio-data\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:27:37.550554 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:27:37.550531    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/1dde1271-7b40-4dc0-97ca-c1be6b89c477-istio-podinfo" pod "1dde1271-7b40-4dc0-97ca-c1be6b89c477" (UID: "1dde1271-7b40-4dc0-97ca-c1be6b89c477"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 01:27:37.550656 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:27:37.550567    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1dde1271-7b40-4dc0-97ca-c1be6b89c477-istio-envoy" pod "1dde1271-7b40-4dc0-97ca-c1be6b89c477" (UID: "1dde1271-7b40-4dc0-97ca-c1be6b89c477"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:27:37.550942 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:27:37.550924    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1dde1271-7b40-4dc0-97ca-c1be6b89c477-istio-token" pod "1dde1271-7b40-4dc0-97ca-c1be6b89c477" (UID: "1dde1271-7b40-4dc0-97ca-c1be6b89c477"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:27:37.551009 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:27:37.550997    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1dde1271-7b40-4dc0-97ca-c1be6b89c477-kube-api-access-xb7gk" pod "1dde1271-7b40-4dc0-97ca-c1be6b89c477" (UID: "1dde1271-7b40-4dc0-97ca-c1be6b89c477"). InnerVolumeSpecName "kube-api-access-xb7gk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:27:37.650338 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:27:37.650304    4284 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1dde1271-7b40-4dc0-97ca-c1be6b89c477-istio-podinfo\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:27:37.650553 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:27:37.650343    4284 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1dde1271-7b40-4dc0-97ca-c1be6b89c477-istio-envoy\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:27:37.650553 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:27:37.650358    4284 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1dde1271-7b40-4dc0-97ca-c1be6b89c477-istio-token\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:27:37.650553 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:27:37.650373    4284 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1dde1271-7b40-4dc0-97ca-c1be6b89c477-istiod-ca-cert\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:27:37.650553 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:27:37.650387    4284 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xb7gk\" (UniqueName: \"kubernetes.io/projected/1dde1271-7b40-4dc0-97ca-c1be6b89c477-kube-api-access-xb7gk\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:27:37.919885 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:27:37.919847    4284 generic.go:356] "Generic (PLEG): container finished" podID="1dde1271-7b40-4dc0-97ca-c1be6b89c477" containerID="7010ec84037e565c88b766656f2b922c7077c92442970dde7646b0425199d25d" exitCode=0
Oct 31 01:27:37.920073 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:27:37.919903    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-a-eb68dd-openshift-default-98b48c6cc-gfz48"
Oct 31 01:27:37.920073 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:27:37.919917    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-eb68dd-openshift-default-98b48c6cc-gfz48" event={"ID":"1dde1271-7b40-4dc0-97ca-c1be6b89c477","Type":"ContainerDied","Data":"7010ec84037e565c88b766656f2b922c7077c92442970dde7646b0425199d25d"}
Oct 31 01:27:37.920073 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:27:37.919966    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-eb68dd-openshift-default-98b48c6cc-gfz48" event={"ID":"1dde1271-7b40-4dc0-97ca-c1be6b89c477","Type":"ContainerDied","Data":"cbf89d3915a2b260d21d6e2a45430b9a01d6bd1bbb49c144b2e409de1929ce0a"}
Oct 31 01:27:37.920073 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:27:37.919991    4284 scope.go:122] "RemoveContainer" containerID="7010ec84037e565c88b766656f2b922c7077c92442970dde7646b0425199d25d"
Oct 31 01:27:37.931072 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:27:37.931051    4284 scope.go:122] "RemoveContainer" containerID="bbab3f04b06e5edb3eaae4cb48bd2bef652990c808bf7fae3bbbf37a9beabcb1"
Oct 31 01:27:37.939770 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:27:37.939750    4284 scope.go:122] "RemoveContainer" containerID="7010ec84037e565c88b766656f2b922c7077c92442970dde7646b0425199d25d"
Oct 31 01:27:37.940040 ip-10-0-132-173 kubenswrapper[4284]: E1031 01:27:37.940019    4284 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7010ec84037e565c88b766656f2b922c7077c92442970dde7646b0425199d25d\": container with ID starting with 7010ec84037e565c88b766656f2b922c7077c92442970dde7646b0425199d25d not found: ID does not exist" containerID="7010ec84037e565c88b766656f2b922c7077c92442970dde7646b0425199d25d"
Oct 31 01:27:37.940162 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:27:37.940052    4284 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7010ec84037e565c88b766656f2b922c7077c92442970dde7646b0425199d25d"} err="failed to get container status \"7010ec84037e565c88b766656f2b922c7077c92442970dde7646b0425199d25d\": rpc error: code = NotFound desc = could not find container \"7010ec84037e565c88b766656f2b922c7077c92442970dde7646b0425199d25d\": container with ID starting with 7010ec84037e565c88b766656f2b922c7077c92442970dde7646b0425199d25d not found: ID does not exist"
Oct 31 01:27:37.940162 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:27:37.940079    4284 scope.go:122] "RemoveContainer" containerID="bbab3f04b06e5edb3eaae4cb48bd2bef652990c808bf7fae3bbbf37a9beabcb1"
Oct 31 01:27:37.940352 ip-10-0-132-173 kubenswrapper[4284]: E1031 01:27:37.940333    4284 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bbab3f04b06e5edb3eaae4cb48bd2bef652990c808bf7fae3bbbf37a9beabcb1\": container with ID starting with bbab3f04b06e5edb3eaae4cb48bd2bef652990c808bf7fae3bbbf37a9beabcb1 not found: ID does not exist" containerID="bbab3f04b06e5edb3eaae4cb48bd2bef652990c808bf7fae3bbbf37a9beabcb1"
Oct 31 01:27:37.940402 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:27:37.940357    4284 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bbab3f04b06e5edb3eaae4cb48bd2bef652990c808bf7fae3bbbf37a9beabcb1"} err="failed to get container status \"bbab3f04b06e5edb3eaae4cb48bd2bef652990c808bf7fae3bbbf37a9beabcb1\": rpc error: code = NotFound desc = could not find container \"bbab3f04b06e5edb3eaae4cb48bd2bef652990c808bf7fae3bbbf37a9beabcb1\": container with ID starting with bbab3f04b06e5edb3eaae4cb48bd2bef652990c808bf7fae3bbbf37a9beabcb1 not found: ID does not exist"
Oct 31 01:27:38.078595 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:27:38.078558    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-extmod-a-eb68dd-openshift-default-98b48c6cc-gfz48"]
Oct 31 01:27:38.126227 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:27:38.126178    4284 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-extmod-a-eb68dd-openshift-default-98b48c6cc-gfz48"]
Oct 31 01:27:38.466457 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:27:38.466394    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-a-08fcc4-openshift-default-7985d77b8f-t22ds"
Oct 31 01:27:38.561112 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:27:38.561079    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/04e2ad9d-c5b9-47b0-ba97-86114775393c-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/04e2ad9d-c5b9-47b0-ba97-86114775393c-credential-socket\") pod \"04e2ad9d-c5b9-47b0-ba97-86114775393c\" (UID: \"04e2ad9d-c5b9-47b0-ba97-86114775393c\") "
Oct 31 01:27:38.561112 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:27:38.561116    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/04e2ad9d-c5b9-47b0-ba97-86114775393c-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/04e2ad9d-c5b9-47b0-ba97-86114775393c-workload-socket\") pod \"04e2ad9d-c5b9-47b0-ba97-86114775393c\" (UID: \"04e2ad9d-c5b9-47b0-ba97-86114775393c\") "
Oct 31 01:27:38.561707 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:27:38.561207    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/04e2ad9d-c5b9-47b0-ba97-86114775393c-istio-token\" (UniqueName: \"kubernetes.io/projected/04e2ad9d-c5b9-47b0-ba97-86114775393c-istio-token\") pod \"04e2ad9d-c5b9-47b0-ba97-86114775393c\" (UID: \"04e2ad9d-c5b9-47b0-ba97-86114775393c\") "
Oct 31 01:27:38.561707 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:27:38.561309    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/04e2ad9d-c5b9-47b0-ba97-86114775393c-kube-api-access-x8nqn\" (UniqueName: \"kubernetes.io/projected/04e2ad9d-c5b9-47b0-ba97-86114775393c-kube-api-access-x8nqn\") pod \"04e2ad9d-c5b9-47b0-ba97-86114775393c\" (UID: \"04e2ad9d-c5b9-47b0-ba97-86114775393c\") "
Oct 31 01:27:38.561707 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:27:38.561350    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/04e2ad9d-c5b9-47b0-ba97-86114775393c-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/04e2ad9d-c5b9-47b0-ba97-86114775393c-istiod-ca-cert\") pod \"04e2ad9d-c5b9-47b0-ba97-86114775393c\" (UID: \"04e2ad9d-c5b9-47b0-ba97-86114775393c\") "
Oct 31 01:27:38.561707 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:27:38.561400    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/04e2ad9d-c5b9-47b0-ba97-86114775393c-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/04e2ad9d-c5b9-47b0-ba97-86114775393c-istio-data\") pod \"04e2ad9d-c5b9-47b0-ba97-86114775393c\" (UID: \"04e2ad9d-c5b9-47b0-ba97-86114775393c\") "
Oct 31 01:27:38.561707 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:27:38.561435    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/04e2ad9d-c5b9-47b0-ba97-86114775393c-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/04e2ad9d-c5b9-47b0-ba97-86114775393c-workload-certs\") pod \"04e2ad9d-c5b9-47b0-ba97-86114775393c\" (UID: \"04e2ad9d-c5b9-47b0-ba97-86114775393c\") "
Oct 31 01:27:38.561707 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:27:38.561453    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/04e2ad9d-c5b9-47b0-ba97-86114775393c-credential-socket" pod "04e2ad9d-c5b9-47b0-ba97-86114775393c" (UID: "04e2ad9d-c5b9-47b0-ba97-86114775393c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:27:38.561707 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:27:38.561470    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/04e2ad9d-c5b9-47b0-ba97-86114775393c-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/04e2ad9d-c5b9-47b0-ba97-86114775393c-istio-podinfo\") pod \"04e2ad9d-c5b9-47b0-ba97-86114775393c\" (UID: \"04e2ad9d-c5b9-47b0-ba97-86114775393c\") "
Oct 31 01:27:38.561707 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:27:38.561483    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/04e2ad9d-c5b9-47b0-ba97-86114775393c-workload-socket" pod "04e2ad9d-c5b9-47b0-ba97-86114775393c" (UID: "04e2ad9d-c5b9-47b0-ba97-86114775393c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:27:38.561707 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:27:38.561496    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/04e2ad9d-c5b9-47b0-ba97-86114775393c-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/04e2ad9d-c5b9-47b0-ba97-86114775393c-istio-envoy\") pod \"04e2ad9d-c5b9-47b0-ba97-86114775393c\" (UID: \"04e2ad9d-c5b9-47b0-ba97-86114775393c\") "
Oct 31 01:27:38.562133 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:27:38.561741    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/04e2ad9d-c5b9-47b0-ba97-86114775393c-istio-data" pod "04e2ad9d-c5b9-47b0-ba97-86114775393c" (UID: "04e2ad9d-c5b9-47b0-ba97-86114775393c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:27:38.562133 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:27:38.561758    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/04e2ad9d-c5b9-47b0-ba97-86114775393c-workload-certs" pod "04e2ad9d-c5b9-47b0-ba97-86114775393c" (UID: "04e2ad9d-c5b9-47b0-ba97-86114775393c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:27:38.562133 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:27:38.561925    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/04e2ad9d-c5b9-47b0-ba97-86114775393c-istiod-ca-cert" pod "04e2ad9d-c5b9-47b0-ba97-86114775393c" (UID: "04e2ad9d-c5b9-47b0-ba97-86114775393c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 01:27:38.562133 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:27:38.562125    4284 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/04e2ad9d-c5b9-47b0-ba97-86114775393c-credential-socket\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:27:38.562311 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:27:38.562145    4284 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/04e2ad9d-c5b9-47b0-ba97-86114775393c-workload-socket\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:27:38.562311 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:27:38.562160    4284 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/04e2ad9d-c5b9-47b0-ba97-86114775393c-istiod-ca-cert\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:27:38.562311 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:27:38.562173    4284 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/04e2ad9d-c5b9-47b0-ba97-86114775393c-istio-data\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:27:38.562414 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:27:38.562370    4284 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/04e2ad9d-c5b9-47b0-ba97-86114775393c-workload-certs\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:27:38.563651 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:27:38.563625    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/04e2ad9d-c5b9-47b0-ba97-86114775393c-istio-token" pod "04e2ad9d-c5b9-47b0-ba97-86114775393c" (UID: "04e2ad9d-c5b9-47b0-ba97-86114775393c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:27:38.563806 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:27:38.563786    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/04e2ad9d-c5b9-47b0-ba97-86114775393c-istio-podinfo" pod "04e2ad9d-c5b9-47b0-ba97-86114775393c" (UID: "04e2ad9d-c5b9-47b0-ba97-86114775393c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 01:27:38.564108 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:27:38.564079    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/04e2ad9d-c5b9-47b0-ba97-86114775393c-kube-api-access-x8nqn" pod "04e2ad9d-c5b9-47b0-ba97-86114775393c" (UID: "04e2ad9d-c5b9-47b0-ba97-86114775393c"). InnerVolumeSpecName "kube-api-access-x8nqn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:27:38.564265 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:27:38.564108    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/04e2ad9d-c5b9-47b0-ba97-86114775393c-istio-envoy" pod "04e2ad9d-c5b9-47b0-ba97-86114775393c" (UID: "04e2ad9d-c5b9-47b0-ba97-86114775393c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:27:38.664160 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:27:38.664128    4284 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/04e2ad9d-c5b9-47b0-ba97-86114775393c-istio-token\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:27:38.664160 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:27:38.664156    4284 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x8nqn\" (UniqueName: \"kubernetes.io/projected/04e2ad9d-c5b9-47b0-ba97-86114775393c-kube-api-access-x8nqn\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:27:38.664160 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:27:38.664166    4284 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/04e2ad9d-c5b9-47b0-ba97-86114775393c-istio-podinfo\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:27:38.664432 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:27:38.664174    4284 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/04e2ad9d-c5b9-47b0-ba97-86114775393c-istio-envoy\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:27:38.928472 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:27:38.928434    4284 generic.go:356] "Generic (PLEG): container finished" podID="04e2ad9d-c5b9-47b0-ba97-86114775393c" containerID="308a3655c034978cc676a0a87c53e6476371fba9cb33992a37d64198792752c9" exitCode=0
Oct 31 01:27:38.928674 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:27:38.928489    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-a-08fcc4-openshift-default-7985d77b8f-t22ds"
Oct 31 01:27:38.928674 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:27:38.928520    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-08fcc4-openshift-default-7985d77b8f-t22ds" event={"ID":"04e2ad9d-c5b9-47b0-ba97-86114775393c","Type":"ContainerDied","Data":"308a3655c034978cc676a0a87c53e6476371fba9cb33992a37d64198792752c9"}
Oct 31 01:27:38.928674 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:27:38.928573    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-08fcc4-openshift-default-7985d77b8f-t22ds" event={"ID":"04e2ad9d-c5b9-47b0-ba97-86114775393c","Type":"ContainerDied","Data":"42a7f69d84606ef4dc540f248bc38cce6697aad2067783da937c44d1ee556bc3"}
Oct 31 01:27:38.928674 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:27:38.928595    4284 scope.go:122] "RemoveContainer" containerID="308a3655c034978cc676a0a87c53e6476371fba9cb33992a37d64198792752c9"
Oct 31 01:27:38.939271 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:27:38.939247    4284 scope.go:122] "RemoveContainer" containerID="549889e4e2ee33f009fd5e45398cc88baf53ae9706354a3075d755a1b45735e4"
Oct 31 01:27:38.948284 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:27:38.948267    4284 scope.go:122] "RemoveContainer" containerID="308a3655c034978cc676a0a87c53e6476371fba9cb33992a37d64198792752c9"
Oct 31 01:27:38.948564 ip-10-0-132-173 kubenswrapper[4284]: E1031 01:27:38.948525    4284 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"308a3655c034978cc676a0a87c53e6476371fba9cb33992a37d64198792752c9\": container with ID starting with 308a3655c034978cc676a0a87c53e6476371fba9cb33992a37d64198792752c9 not found: ID does not exist" containerID="308a3655c034978cc676a0a87c53e6476371fba9cb33992a37d64198792752c9"
Oct 31 01:27:38.948609 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:27:38.948571    4284 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"308a3655c034978cc676a0a87c53e6476371fba9cb33992a37d64198792752c9"} err="failed to get container status \"308a3655c034978cc676a0a87c53e6476371fba9cb33992a37d64198792752c9\": rpc error: code = NotFound desc = could not find container \"308a3655c034978cc676a0a87c53e6476371fba9cb33992a37d64198792752c9\": container with ID starting with 308a3655c034978cc676a0a87c53e6476371fba9cb33992a37d64198792752c9 not found: ID does not exist"
Oct 31 01:27:38.948609 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:27:38.948590    4284 scope.go:122] "RemoveContainer" containerID="549889e4e2ee33f009fd5e45398cc88baf53ae9706354a3075d755a1b45735e4"
Oct 31 01:27:38.948831 ip-10-0-132-173 kubenswrapper[4284]: E1031 01:27:38.948812    4284 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"549889e4e2ee33f009fd5e45398cc88baf53ae9706354a3075d755a1b45735e4\": container with ID starting with 549889e4e2ee33f009fd5e45398cc88baf53ae9706354a3075d755a1b45735e4 not found: ID does not exist" containerID="549889e4e2ee33f009fd5e45398cc88baf53ae9706354a3075d755a1b45735e4"
Oct 31 01:27:38.948873 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:27:38.948839    4284 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"549889e4e2ee33f009fd5e45398cc88baf53ae9706354a3075d755a1b45735e4"} err="failed to get container status \"549889e4e2ee33f009fd5e45398cc88baf53ae9706354a3075d755a1b45735e4\": rpc error: code = NotFound desc = could not find container \"549889e4e2ee33f009fd5e45398cc88baf53ae9706354a3075d755a1b45735e4\": container with ID starting with 549889e4e2ee33f009fd5e45398cc88baf53ae9706354a3075d755a1b45735e4 not found: ID does not exist"
Oct 31 01:27:39.086557 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:27:39.086524    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-extmod-a-08fcc4-openshift-default-7985d77b8f-t22ds"]
Oct 31 01:27:39.134041 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:27:39.134010    4284 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-extmod-a-08fcc4-openshift-default-7985d77b8f-t22ds"]
Oct 31 01:27:39.970915 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:27:39.970876    4284 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="04e2ad9d-c5b9-47b0-ba97-86114775393c" path="/var/lib/kubelet/pods/04e2ad9d-c5b9-47b0-ba97-86114775393c/volumes"
Oct 31 01:27:39.971577 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:27:39.971556    4284 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1dde1271-7b40-4dc0-97ca-c1be6b89c477" path="/var/lib/kubelet/pods/1dde1271-7b40-4dc0-97ca-c1be6b89c477/volumes"
Oct 31 01:27:40.113225 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:27:40.113129    4284 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-worker-w3-dd98f0-7cf4455d8c-979g4" podUID="1a294983-e4ba-4253-8e51-24481d476dd6" containerName="payload-processing" containerID="cri-o://4cd17abeec3e7f3281c3a48b75a8eaa53bba63c3524ff6079dbbf23588047847" gracePeriod=25
Oct 31 01:27:40.536278 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:27:40.536203    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:27:42.689787 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:27:42.689747    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:27:43.310002 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:27:43.309956    4284 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-w3-dd98f0-7cf4455d8c-979g4" podUID="1a294983-e4ba-4253-8e51-24481d476dd6" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 31 01:27:44.901583 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:27:44.901546    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:27:45.050276 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:27:45.050242    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:27:45.696329 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:27:45.696291    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:27:47.013267 ip-10-0-132-173 kubenswrapper[4284]: E1031 01:27:47.013229    4284 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fd8sp" containerName="extractor"
Oct 31 01:27:47.438693 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:27:47.438660    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:27:47.635979 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:27:47.635885    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:27:50.482910 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:27:50.482872    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:27:50.623783 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:27:50.623744    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:27:53.310422 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:27:53.310385    4284 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-w3-dd98f0-7cf4455d8c-979g4" podUID="1a294983-e4ba-4253-8e51-24481d476dd6" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 31 01:27:55.273349 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:27:55.273311    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:27:59.336678 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:27:59.336646    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-ba34eb-6c74885c96-8smj9"]
Oct 31 01:27:59.337098 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:27:59.336841    4284 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-ba34eb-6c74885c96-8smj9" podUID="a11c7b2d-901c-4c69-9be2-a0bd6eeaf9c0" containerName="maas-api" containerID="cri-o://46a2b477abe13c9c91c8aa486dd689a73a9eb40e0667fbd36e846a708023d212" gracePeriod=30
Oct 31 01:27:59.438698 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:27:59.438669    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:27:59.639085 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:27:59.639062    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-ba34eb-6c74885c96-8smj9"
Oct 31 01:27:59.744237 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:27:59.739344    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a11c7b2d-901c-4c69-9be2-a0bd6eeaf9c0-kube-api-access-zjxd6\" (UniqueName: \"kubernetes.io/projected/a11c7b2d-901c-4c69-9be2-a0bd6eeaf9c0-kube-api-access-zjxd6\") pod \"a11c7b2d-901c-4c69-9be2-a0bd6eeaf9c0\" (UID: \"a11c7b2d-901c-4c69-9be2-a0bd6eeaf9c0\") "
Oct 31 01:27:59.744237 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:27:59.739419    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a11c7b2d-901c-4c69-9be2-a0bd6eeaf9c0-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a11c7b2d-901c-4c69-9be2-a0bd6eeaf9c0-maas-api-tls\") pod \"a11c7b2d-901c-4c69-9be2-a0bd6eeaf9c0\" (UID: \"a11c7b2d-901c-4c69-9be2-a0bd6eeaf9c0\") "
Oct 31 01:27:59.744237 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:27:59.739522    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a11c7b2d-901c-4c69-9be2-a0bd6eeaf9c0-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a11c7b2d-901c-4c69-9be2-a0bd6eeaf9c0-openshift-service-ca\") pod \"a11c7b2d-901c-4c69-9be2-a0bd6eeaf9c0\" (UID: \"a11c7b2d-901c-4c69-9be2-a0bd6eeaf9c0\") "
Oct 31 01:27:59.744237 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:27:59.739572    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a11c7b2d-901c-4c69-9be2-a0bd6eeaf9c0-metrics-certs\" (UniqueName: \"kubernetes.io/secret/a11c7b2d-901c-4c69-9be2-a0bd6eeaf9c0-metrics-certs\") pod \"a11c7b2d-901c-4c69-9be2-a0bd6eeaf9c0\" (UID: \"a11c7b2d-901c-4c69-9be2-a0bd6eeaf9c0\") "
Oct 31 01:27:59.744237 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:27:59.741726    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a11c7b2d-901c-4c69-9be2-a0bd6eeaf9c0-openshift-service-ca" pod "a11c7b2d-901c-4c69-9be2-a0bd6eeaf9c0" (UID: "a11c7b2d-901c-4c69-9be2-a0bd6eeaf9c0"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 01:27:59.744237 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:27:59.743780    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a11c7b2d-901c-4c69-9be2-a0bd6eeaf9c0-maas-api-tls" pod "a11c7b2d-901c-4c69-9be2-a0bd6eeaf9c0" (UID: "a11c7b2d-901c-4c69-9be2-a0bd6eeaf9c0"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 01:27:59.744694 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:27:59.744481    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a11c7b2d-901c-4c69-9be2-a0bd6eeaf9c0-metrics-certs" pod "a11c7b2d-901c-4c69-9be2-a0bd6eeaf9c0" (UID: "a11c7b2d-901c-4c69-9be2-a0bd6eeaf9c0"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 01:27:59.749540 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:27:59.748863    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a11c7b2d-901c-4c69-9be2-a0bd6eeaf9c0-kube-api-access-zjxd6" pod "a11c7b2d-901c-4c69-9be2-a0bd6eeaf9c0" (UID: "a11c7b2d-901c-4c69-9be2-a0bd6eeaf9c0"). InnerVolumeSpecName "kube-api-access-zjxd6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:27:59.841779 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:27:59.841734    4284 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zjxd6\" (UniqueName: \"kubernetes.io/projected/a11c7b2d-901c-4c69-9be2-a0bd6eeaf9c0-kube-api-access-zjxd6\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:27:59.841779 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:27:59.841772    4284 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a11c7b2d-901c-4c69-9be2-a0bd6eeaf9c0-maas-api-tls\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:27:59.841779 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:27:59.841787    4284 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a11c7b2d-901c-4c69-9be2-a0bd6eeaf9c0-openshift-service-ca\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:27:59.842027 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:27:59.841799    4284 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a11c7b2d-901c-4c69-9be2-a0bd6eeaf9c0-metrics-certs\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:28:00.058578 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:28:00.058550    4284 generic.go:356] "Generic (PLEG): container finished" podID="a11c7b2d-901c-4c69-9be2-a0bd6eeaf9c0" containerID="46a2b477abe13c9c91c8aa486dd689a73a9eb40e0667fbd36e846a708023d212" exitCode=0
Oct 31 01:28:00.058743 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:28:00.058600    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-ba34eb-6c74885c96-8smj9"
Oct 31 01:28:00.058743 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:28:00.058643    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-ba34eb-6c74885c96-8smj9" event={"ID":"a11c7b2d-901c-4c69-9be2-a0bd6eeaf9c0","Type":"ContainerDied","Data":"46a2b477abe13c9c91c8aa486dd689a73a9eb40e0667fbd36e846a708023d212"}
Oct 31 01:28:00.058743 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:28:00.058704    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-ba34eb-6c74885c96-8smj9" event={"ID":"a11c7b2d-901c-4c69-9be2-a0bd6eeaf9c0","Type":"ContainerDied","Data":"fbd7bed1dae58dc6212172f8cef516caaa17f99dab18254f456a5deb01130fe7"}
Oct 31 01:28:00.058743 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:28:00.058731    4284 scope.go:122] "RemoveContainer" containerID="46a2b477abe13c9c91c8aa486dd689a73a9eb40e0667fbd36e846a708023d212"
Oct 31 01:28:00.068083 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:28:00.068062    4284 scope.go:122] "RemoveContainer" containerID="46a2b477abe13c9c91c8aa486dd689a73a9eb40e0667fbd36e846a708023d212"
Oct 31 01:28:00.068366 ip-10-0-132-173 kubenswrapper[4284]: E1031 01:28:00.068342    4284 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"46a2b477abe13c9c91c8aa486dd689a73a9eb40e0667fbd36e846a708023d212\": container with ID starting with 46a2b477abe13c9c91c8aa486dd689a73a9eb40e0667fbd36e846a708023d212 not found: ID does not exist" containerID="46a2b477abe13c9c91c8aa486dd689a73a9eb40e0667fbd36e846a708023d212"
Oct 31 01:28:00.068476 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:28:00.068371    4284 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"46a2b477abe13c9c91c8aa486dd689a73a9eb40e0667fbd36e846a708023d212"} err="failed to get container status \"46a2b477abe13c9c91c8aa486dd689a73a9eb40e0667fbd36e846a708023d212\": rpc error: code = NotFound desc = could not find container \"46a2b477abe13c9c91c8aa486dd689a73a9eb40e0667fbd36e846a708023d212\": container with ID starting with 46a2b477abe13c9c91c8aa486dd689a73a9eb40e0667fbd36e846a708023d212 not found: ID does not exist"
Oct 31 01:28:00.216888 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:28:00.216855    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-ba34eb-6c74885c96-8smj9"]
Oct 31 01:28:00.297263 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:28:00.294424    4284 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-ba34eb-6c74885c96-8smj9"]
Oct 31 01:28:00.447978 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:28:00.447956    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w3-dd98f0-7cf4455d8c-979g4"
Oct 31 01:28:00.577099 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:28:00.577019    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1a294983-e4ba-4253-8e51-24481d476dd6-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1a294983-e4ba-4253-8e51-24481d476dd6-plugins-config-volume\") pod \"1a294983-e4ba-4253-8e51-24481d476dd6\" (UID: \"1a294983-e4ba-4253-8e51-24481d476dd6\") "
Oct 31 01:28:00.577275 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:28:00.577183    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1a294983-e4ba-4253-8e51-24481d476dd6-kube-api-access-6m98s\" (UniqueName: \"kubernetes.io/projected/1a294983-e4ba-4253-8e51-24481d476dd6-kube-api-access-6m98s\") pod \"1a294983-e4ba-4253-8e51-24481d476dd6\" (UID: \"1a294983-e4ba-4253-8e51-24481d476dd6\") "
Oct 31 01:28:00.577475 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:28:00.577454    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1a294983-e4ba-4253-8e51-24481d476dd6-plugins-config-volume" pod "1a294983-e4ba-4253-8e51-24481d476dd6" (UID: "1a294983-e4ba-4253-8e51-24481d476dd6"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 01:28:00.577679 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:28:00.577652    4284 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1a294983-e4ba-4253-8e51-24481d476dd6-plugins-config-volume\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:28:00.579482 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:28:00.579456    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1a294983-e4ba-4253-8e51-24481d476dd6-kube-api-access-6m98s" pod "1a294983-e4ba-4253-8e51-24481d476dd6" (UID: "1a294983-e4ba-4253-8e51-24481d476dd6"). InnerVolumeSpecName "kube-api-access-6m98s". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:28:00.678545 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:28:00.678509    4284 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6m98s\" (UniqueName: \"kubernetes.io/projected/1a294983-e4ba-4253-8e51-24481d476dd6-kube-api-access-6m98s\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:28:01.066148 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:28:01.066111    4284 generic.go:356] "Generic (PLEG): container finished" podID="1a294983-e4ba-4253-8e51-24481d476dd6" containerID="4cd17abeec3e7f3281c3a48b75a8eaa53bba63c3524ff6079dbbf23588047847" exitCode=0
Oct 31 01:28:01.066358 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:28:01.066161    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w3-dd98f0-7cf4455d8c-979g4"
Oct 31 01:28:01.066358 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:28:01.066199    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w3-dd98f0-7cf4455d8c-979g4" event={"ID":"1a294983-e4ba-4253-8e51-24481d476dd6","Type":"ContainerDied","Data":"4cd17abeec3e7f3281c3a48b75a8eaa53bba63c3524ff6079dbbf23588047847"}
Oct 31 01:28:01.066358 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:28:01.066273    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w3-dd98f0-7cf4455d8c-979g4" event={"ID":"1a294983-e4ba-4253-8e51-24481d476dd6","Type":"ContainerDied","Data":"8d95411bdd4fdf355955af0dfba470741d2300bc948439a0550aa8186c6224db"}
Oct 31 01:28:01.066358 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:28:01.066297    4284 scope.go:122] "RemoveContainer" containerID="4cd17abeec3e7f3281c3a48b75a8eaa53bba63c3524ff6079dbbf23588047847"
Oct 31 01:28:01.076071 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:28:01.076049    4284 scope.go:122] "RemoveContainer" containerID="4cd17abeec3e7f3281c3a48b75a8eaa53bba63c3524ff6079dbbf23588047847"
Oct 31 01:28:01.076361 ip-10-0-132-173 kubenswrapper[4284]: E1031 01:28:01.076340    4284 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4cd17abeec3e7f3281c3a48b75a8eaa53bba63c3524ff6079dbbf23588047847\": container with ID starting with 4cd17abeec3e7f3281c3a48b75a8eaa53bba63c3524ff6079dbbf23588047847 not found: ID does not exist" containerID="4cd17abeec3e7f3281c3a48b75a8eaa53bba63c3524ff6079dbbf23588047847"
Oct 31 01:28:01.076416 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:28:01.076369    4284 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4cd17abeec3e7f3281c3a48b75a8eaa53bba63c3524ff6079dbbf23588047847"} err="failed to get container status \"4cd17abeec3e7f3281c3a48b75a8eaa53bba63c3524ff6079dbbf23588047847\": rpc error: code = NotFound desc = could not find container \"4cd17abeec3e7f3281c3a48b75a8eaa53bba63c3524ff6079dbbf23588047847\": container with ID starting with 4cd17abeec3e7f3281c3a48b75a8eaa53bba63c3524ff6079dbbf23588047847 not found: ID does not exist"
Oct 31 01:28:01.164284 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:28:01.164244    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:28:01.229038 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:28:01.223981    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w3-dd98f0-7cf4455d8c-979g4"]
Oct 31 01:28:01.247238 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:28:01.237926    4284 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w3-dd98f0-7cf4455d8c-979g4"]
Oct 31 01:28:01.397515 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:28:01.397485    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:28:01.717879 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:28:01.717840    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:28:01.964115 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:28:01.964081    4284 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1a294983-e4ba-4253-8e51-24481d476dd6" path="/var/lib/kubelet/pods/1a294983-e4ba-4253-8e51-24481d476dd6/volumes"
Oct 31 01:28:01.964784 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:28:01.964759    4284 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a11c7b2d-901c-4c69-9be2-a0bd6eeaf9c0" path="/var/lib/kubelet/pods/a11c7b2d-901c-4c69-9be2-a0bd6eeaf9c0/volumes"
Oct 31 01:28:02.404766 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:28:02.404725    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:28:03.099114 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:28:03.099072    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:28:03.258282 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:28:03.258244    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:28:04.615224 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:28:04.615187    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:28:06.379747 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:28:06.379701    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:28:08.729647 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:28:08.729611    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:28:08.876590 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:28:08.876551    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:28:09.032990 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:28:09.032890    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:28:09.119390 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:28:09.119360    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w1-ba34eb-openshift-default-98cfdc4c5-f7gpd"]
Oct 31 01:28:09.119604 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:28:09.119583    4284 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-w1-ba34eb-openshift-default-98cfdc4c5-f7gpd" podUID="6aa40f46-c2b1-4c37-a341-d230158a7db9" containerName="istio-proxy" containerID="cri-o://08a063371a0f3fb4e2cf6ac1b9f2a14878be6ec744e71b10aa82481b76895771" gracePeriod=30
Oct 31 01:28:09.175372 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:28:09.175336    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:28:11.308385 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:28:11.308353    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:28:11.447673 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:28:11.447632    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:28:11.775593 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:28:11.775558    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:28:14.393005 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:28:14.392940    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w1-ba34eb-openshift-default-98cfdc4c5-f7gpd"
Oct 31 01:28:14.454380 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:28:14.454349    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/6aa40f46-c2b1-4c37-a341-d230158a7db9-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6aa40f46-c2b1-4c37-a341-d230158a7db9-istiod-ca-cert\") pod \"6aa40f46-c2b1-4c37-a341-d230158a7db9\" (UID: \"6aa40f46-c2b1-4c37-a341-d230158a7db9\") "
Oct 31 01:28:14.454542 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:28:14.454422    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6aa40f46-c2b1-4c37-a341-d230158a7db9-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6aa40f46-c2b1-4c37-a341-d230158a7db9-workload-certs\") pod \"6aa40f46-c2b1-4c37-a341-d230158a7db9\" (UID: \"6aa40f46-c2b1-4c37-a341-d230158a7db9\") "
Oct 31 01:28:14.454542 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:28:14.454455    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/6aa40f46-c2b1-4c37-a341-d230158a7db9-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6aa40f46-c2b1-4c37-a341-d230158a7db9-istio-podinfo\") pod \"6aa40f46-c2b1-4c37-a341-d230158a7db9\" (UID: \"6aa40f46-c2b1-4c37-a341-d230158a7db9\") "
Oct 31 01:28:14.454542 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:28:14.454496    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6aa40f46-c2b1-4c37-a341-d230158a7db9-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6aa40f46-c2b1-4c37-a341-d230158a7db9-credential-socket\") pod \"6aa40f46-c2b1-4c37-a341-d230158a7db9\" (UID: \"6aa40f46-c2b1-4c37-a341-d230158a7db9\") "
Oct 31 01:28:14.454695 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:28:14.454548    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6aa40f46-c2b1-4c37-a341-d230158a7db9-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6aa40f46-c2b1-4c37-a341-d230158a7db9-istio-data\") pod \"6aa40f46-c2b1-4c37-a341-d230158a7db9\" (UID: \"6aa40f46-c2b1-4c37-a341-d230158a7db9\") "
Oct 31 01:28:14.454695 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:28:14.454649    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6aa40f46-c2b1-4c37-a341-d230158a7db9-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6aa40f46-c2b1-4c37-a341-d230158a7db9-istio-envoy\") pod \"6aa40f46-c2b1-4c37-a341-d230158a7db9\" (UID: \"6aa40f46-c2b1-4c37-a341-d230158a7db9\") "
Oct 31 01:28:14.454803 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:28:14.454707    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6aa40f46-c2b1-4c37-a341-d230158a7db9-istio-token\" (UniqueName: \"kubernetes.io/projected/6aa40f46-c2b1-4c37-a341-d230158a7db9-istio-token\") pod \"6aa40f46-c2b1-4c37-a341-d230158a7db9\" (UID: \"6aa40f46-c2b1-4c37-a341-d230158a7db9\") "
Oct 31 01:28:14.454803 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:28:14.454735    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6aa40f46-c2b1-4c37-a341-d230158a7db9-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6aa40f46-c2b1-4c37-a341-d230158a7db9-workload-socket\") pod \"6aa40f46-c2b1-4c37-a341-d230158a7db9\" (UID: \"6aa40f46-c2b1-4c37-a341-d230158a7db9\") "
Oct 31 01:28:14.454803 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:28:14.454734    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6aa40f46-c2b1-4c37-a341-d230158a7db9-workload-certs" pod "6aa40f46-c2b1-4c37-a341-d230158a7db9" (UID: "6aa40f46-c2b1-4c37-a341-d230158a7db9"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:28:14.454803 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:28:14.454770    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6aa40f46-c2b1-4c37-a341-d230158a7db9-istiod-ca-cert" pod "6aa40f46-c2b1-4c37-a341-d230158a7db9" (UID: "6aa40f46-c2b1-4c37-a341-d230158a7db9"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 01:28:14.455002 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:28:14.454857    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6aa40f46-c2b1-4c37-a341-d230158a7db9-istio-data" pod "6aa40f46-c2b1-4c37-a341-d230158a7db9" (UID: "6aa40f46-c2b1-4c37-a341-d230158a7db9"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:28:14.455002 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:28:14.454785    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6aa40f46-c2b1-4c37-a341-d230158a7db9-kube-api-access-nsstt\" (UniqueName: \"kubernetes.io/projected/6aa40f46-c2b1-4c37-a341-d230158a7db9-kube-api-access-nsstt\") pod \"6aa40f46-c2b1-4c37-a341-d230158a7db9\" (UID: \"6aa40f46-c2b1-4c37-a341-d230158a7db9\") "
Oct 31 01:28:14.455085 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:28:14.455063    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6aa40f46-c2b1-4c37-a341-d230158a7db9-credential-socket" pod "6aa40f46-c2b1-4c37-a341-d230158a7db9" (UID: "6aa40f46-c2b1-4c37-a341-d230158a7db9"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:28:14.455154 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:28:14.455078    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6aa40f46-c2b1-4c37-a341-d230158a7db9-workload-socket" pod "6aa40f46-c2b1-4c37-a341-d230158a7db9" (UID: "6aa40f46-c2b1-4c37-a341-d230158a7db9"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:28:14.455666 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:28:14.455601    4284 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6aa40f46-c2b1-4c37-a341-d230158a7db9-workload-certs\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:28:14.455666 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:28:14.455631    4284 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6aa40f46-c2b1-4c37-a341-d230158a7db9-credential-socket\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:28:14.455666 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:28:14.455646    4284 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6aa40f46-c2b1-4c37-a341-d230158a7db9-istio-data\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:28:14.455666 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:28:14.455661    4284 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6aa40f46-c2b1-4c37-a341-d230158a7db9-workload-socket\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:28:14.455981 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:28:14.455675    4284 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6aa40f46-c2b1-4c37-a341-d230158a7db9-istiod-ca-cert\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:28:14.457056 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:28:14.457028    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/6aa40f46-c2b1-4c37-a341-d230158a7db9-istio-podinfo" pod "6aa40f46-c2b1-4c37-a341-d230158a7db9" (UID: "6aa40f46-c2b1-4c37-a341-d230158a7db9"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 01:28:14.457177 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:28:14.457159    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6aa40f46-c2b1-4c37-a341-d230158a7db9-istio-envoy" pod "6aa40f46-c2b1-4c37-a341-d230158a7db9" (UID: "6aa40f46-c2b1-4c37-a341-d230158a7db9"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:28:14.457177 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:28:14.457159    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6aa40f46-c2b1-4c37-a341-d230158a7db9-istio-token" pod "6aa40f46-c2b1-4c37-a341-d230158a7db9" (UID: "6aa40f46-c2b1-4c37-a341-d230158a7db9"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:28:14.457342 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:28:14.457319    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6aa40f46-c2b1-4c37-a341-d230158a7db9-kube-api-access-nsstt" pod "6aa40f46-c2b1-4c37-a341-d230158a7db9" (UID: "6aa40f46-c2b1-4c37-a341-d230158a7db9"). InnerVolumeSpecName "kube-api-access-nsstt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:28:14.557173 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:28:14.557141    4284 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6aa40f46-c2b1-4c37-a341-d230158a7db9-istio-envoy\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:28:14.557173 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:28:14.557167    4284 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6aa40f46-c2b1-4c37-a341-d230158a7db9-istio-token\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:28:14.557173 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:28:14.557177    4284 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nsstt\" (UniqueName: \"kubernetes.io/projected/6aa40f46-c2b1-4c37-a341-d230158a7db9-kube-api-access-nsstt\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:28:14.557517 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:28:14.557187    4284 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6aa40f46-c2b1-4c37-a341-d230158a7db9-istio-podinfo\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:28:15.155917 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:28:15.155887    4284 generic.go:356] "Generic (PLEG): container finished" podID="6aa40f46-c2b1-4c37-a341-d230158a7db9" containerID="08a063371a0f3fb4e2cf6ac1b9f2a14878be6ec744e71b10aa82481b76895771" exitCode=0
Oct 31 01:28:15.156088 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:28:15.155939    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w1-ba34eb-openshift-default-98cfdc4c5-f7gpd"
Oct 31 01:28:15.156088 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:28:15.155966    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w1-ba34eb-openshift-default-98cfdc4c5-f7gpd" event={"ID":"6aa40f46-c2b1-4c37-a341-d230158a7db9","Type":"ContainerDied","Data":"08a063371a0f3fb4e2cf6ac1b9f2a14878be6ec744e71b10aa82481b76895771"}
Oct 31 01:28:15.156088 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:28:15.156025    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w1-ba34eb-openshift-default-98cfdc4c5-f7gpd" event={"ID":"6aa40f46-c2b1-4c37-a341-d230158a7db9","Type":"ContainerDied","Data":"9c9f0c711b27dc2cd3ad27444a0b333b91f7e109cd583393359e99262d944a90"}
Oct 31 01:28:15.156088 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:28:15.156048    4284 scope.go:122] "RemoveContainer" containerID="08a063371a0f3fb4e2cf6ac1b9f2a14878be6ec744e71b10aa82481b76895771"
Oct 31 01:28:15.166740 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:28:15.166721    4284 scope.go:122] "RemoveContainer" containerID="f4d91cbb71f4601158caf9415ba4d9cb617951c3cda297a3c341c334e23db70b"
Oct 31 01:28:15.176032 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:28:15.176009    4284 scope.go:122] "RemoveContainer" containerID="08a063371a0f3fb4e2cf6ac1b9f2a14878be6ec744e71b10aa82481b76895771"
Oct 31 01:28:15.176330 ip-10-0-132-173 kubenswrapper[4284]: E1031 01:28:15.176311    4284 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"08a063371a0f3fb4e2cf6ac1b9f2a14878be6ec744e71b10aa82481b76895771\": container with ID starting with 08a063371a0f3fb4e2cf6ac1b9f2a14878be6ec744e71b10aa82481b76895771 not found: ID does not exist" containerID="08a063371a0f3fb4e2cf6ac1b9f2a14878be6ec744e71b10aa82481b76895771"
Oct 31 01:28:15.176382 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:28:15.176339    4284 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"08a063371a0f3fb4e2cf6ac1b9f2a14878be6ec744e71b10aa82481b76895771"} err="failed to get container status \"08a063371a0f3fb4e2cf6ac1b9f2a14878be6ec744e71b10aa82481b76895771\": rpc error: code = NotFound desc = could not find container \"08a063371a0f3fb4e2cf6ac1b9f2a14878be6ec744e71b10aa82481b76895771\": container with ID starting with 08a063371a0f3fb4e2cf6ac1b9f2a14878be6ec744e71b10aa82481b76895771 not found: ID does not exist"
Oct 31 01:28:15.176382 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:28:15.176357    4284 scope.go:122] "RemoveContainer" containerID="f4d91cbb71f4601158caf9415ba4d9cb617951c3cda297a3c341c334e23db70b"
Oct 31 01:28:15.176620 ip-10-0-132-173 kubenswrapper[4284]: E1031 01:28:15.176603    4284 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f4d91cbb71f4601158caf9415ba4d9cb617951c3cda297a3c341c334e23db70b\": container with ID starting with f4d91cbb71f4601158caf9415ba4d9cb617951c3cda297a3c341c334e23db70b not found: ID does not exist" containerID="f4d91cbb71f4601158caf9415ba4d9cb617951c3cda297a3c341c334e23db70b"
Oct 31 01:28:15.176661 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:28:15.176627    4284 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f4d91cbb71f4601158caf9415ba4d9cb617951c3cda297a3c341c334e23db70b"} err="failed to get container status \"f4d91cbb71f4601158caf9415ba4d9cb617951c3cda297a3c341c334e23db70b\": rpc error: code = NotFound desc = could not find container \"f4d91cbb71f4601158caf9415ba4d9cb617951c3cda297a3c341c334e23db70b\": container with ID starting with f4d91cbb71f4601158caf9415ba4d9cb617951c3cda297a3c341c334e23db70b not found: ID does not exist"
Oct 31 01:28:15.363258 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:28:15.363225    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w1-ba34eb-openshift-default-98cfdc4c5-f7gpd"]
Oct 31 01:28:15.410806 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:28:15.410731    4284 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-w1-ba34eb-openshift-default-98cfdc4c5-f7gpd"]
Oct 31 01:28:15.962491 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:28:15.962461    4284 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6aa40f46-c2b1-4c37-a341-d230158a7db9" path="/var/lib/kubelet/pods/6aa40f46-c2b1-4c37-a341-d230158a7db9/volumes"
Oct 31 01:28:17.023715 ip-10-0-132-173 kubenswrapper[4284]: E1031 01:28:17.023675    4284 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fd8sp" containerName="extractor"
Oct 31 01:28:17.029276 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:28:17.029227    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:28:21.914955 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:28:21.914924    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:28:22.109405 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:28:22.109357    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:28:26.657536 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:28:26.657497    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:28:28.257890 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:28:28.257850    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:28:28.484008 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:28:28.483967    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:28:28.673586 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:28:28.673547    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:28:29.142490 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:28:29.142396    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:28:33.719306 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:28:33.719271    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:28:33.923817 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:28:33.923774    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:28:34.070737 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:28:34.070651    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:28:35.533535 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:28:35.533491    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:28:35.681688 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:28:35.681648    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:28:35.902435 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:28:35.902347    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-4397fc-openshift-default-6b458f575f-tvgdj"]
Oct 31 01:28:35.902642 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:28:35.902614    4284 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-w2-4397fc-openshift-default-6b458f575f-tvgdj" podUID="fd46495b-8c51-476d-91f9-750529060bbd" containerName="istio-proxy" containerID="cri-o://e9ebf0b02fcd8550bde2f0e6cae0ad98fe7c51b171e1397767f0556648bdb0b5" gracePeriod=30
Oct 31 01:28:35.921432 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:28:35.921392    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:28:37.106985 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:28:37.106947    4284 patch_prober.go:29] interesting pod/e2e-shared-a-w2-4397fc-openshift-default-6b458f575f-tvgdj container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 31 01:28:37.107402 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:28:37.107037    4284 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-w2-4397fc-openshift-default-6b458f575f-tvgdj" podUID="fd46495b-8c51-476d-91f9-750529060bbd" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 31 01:28:38.999077 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:28:38.999044    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:28:41.176840 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:28:41.176768    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-4397fc-openshift-default-6b458f575f-tvgdj"
Oct 31 01:28:41.273363 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:28:41.273329    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fd46495b-8c51-476d-91f9-750529060bbd-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fd46495b-8c51-476d-91f9-750529060bbd-istio-envoy\") pod \"fd46495b-8c51-476d-91f9-750529060bbd\" (UID: \"fd46495b-8c51-476d-91f9-750529060bbd\") "
Oct 31 01:28:41.273548 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:28:41.273427    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/fd46495b-8c51-476d-91f9-750529060bbd-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fd46495b-8c51-476d-91f9-750529060bbd-istiod-ca-cert\") pod \"fd46495b-8c51-476d-91f9-750529060bbd\" (UID: \"fd46495b-8c51-476d-91f9-750529060bbd\") "
Oct 31 01:28:41.273548 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:28:41.273455    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fd46495b-8c51-476d-91f9-750529060bbd-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fd46495b-8c51-476d-91f9-750529060bbd-workload-socket\") pod \"fd46495b-8c51-476d-91f9-750529060bbd\" (UID: \"fd46495b-8c51-476d-91f9-750529060bbd\") "
Oct 31 01:28:41.273548 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:28:41.273487    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/fd46495b-8c51-476d-91f9-750529060bbd-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fd46495b-8c51-476d-91f9-750529060bbd-istio-podinfo\") pod \"fd46495b-8c51-476d-91f9-750529060bbd\" (UID: \"fd46495b-8c51-476d-91f9-750529060bbd\") "
Oct 31 01:28:41.273548 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:28:41.273506    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fd46495b-8c51-476d-91f9-750529060bbd-kube-api-access-jcbsm\" (UniqueName: \"kubernetes.io/projected/fd46495b-8c51-476d-91f9-750529060bbd-kube-api-access-jcbsm\") pod \"fd46495b-8c51-476d-91f9-750529060bbd\" (UID: \"fd46495b-8c51-476d-91f9-750529060bbd\") "
Oct 31 01:28:41.273745 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:28:41.273596    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fd46495b-8c51-476d-91f9-750529060bbd-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fd46495b-8c51-476d-91f9-750529060bbd-workload-certs\") pod \"fd46495b-8c51-476d-91f9-750529060bbd\" (UID: \"fd46495b-8c51-476d-91f9-750529060bbd\") "
Oct 31 01:28:41.273745 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:28:41.273623    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fd46495b-8c51-476d-91f9-750529060bbd-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fd46495b-8c51-476d-91f9-750529060bbd-credential-socket\") pod \"fd46495b-8c51-476d-91f9-750529060bbd\" (UID: \"fd46495b-8c51-476d-91f9-750529060bbd\") "
Oct 31 01:28:41.273745 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:28:41.273651    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fd46495b-8c51-476d-91f9-750529060bbd-istio-token\" (UniqueName: \"kubernetes.io/projected/fd46495b-8c51-476d-91f9-750529060bbd-istio-token\") pod \"fd46495b-8c51-476d-91f9-750529060bbd\" (UID: \"fd46495b-8c51-476d-91f9-750529060bbd\") "
Oct 31 01:28:41.273914 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:28:41.273745    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fd46495b-8c51-476d-91f9-750529060bbd-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fd46495b-8c51-476d-91f9-750529060bbd-istio-data\") pod \"fd46495b-8c51-476d-91f9-750529060bbd\" (UID: \"fd46495b-8c51-476d-91f9-750529060bbd\") "
Oct 31 01:28:41.273914 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:28:41.273772    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fd46495b-8c51-476d-91f9-750529060bbd-workload-socket" pod "fd46495b-8c51-476d-91f9-750529060bbd" (UID: "fd46495b-8c51-476d-91f9-750529060bbd"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:28:41.274034 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:28:41.273927    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fd46495b-8c51-476d-91f9-750529060bbd-istiod-ca-cert" pod "fd46495b-8c51-476d-91f9-750529060bbd" (UID: "fd46495b-8c51-476d-91f9-750529060bbd"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 01:28:41.274034 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:28:41.273989    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fd46495b-8c51-476d-91f9-750529060bbd-credential-socket" pod "fd46495b-8c51-476d-91f9-750529060bbd" (UID: "fd46495b-8c51-476d-91f9-750529060bbd"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:28:41.274277 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:28:41.274252    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fd46495b-8c51-476d-91f9-750529060bbd-workload-certs" pod "fd46495b-8c51-476d-91f9-750529060bbd" (UID: "fd46495b-8c51-476d-91f9-750529060bbd"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:28:41.274392 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:28:41.274367    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fd46495b-8c51-476d-91f9-750529060bbd-istio-data" pod "fd46495b-8c51-476d-91f9-750529060bbd" (UID: "fd46495b-8c51-476d-91f9-750529060bbd"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:28:41.274525 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:28:41.274506    4284 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fd46495b-8c51-476d-91f9-750529060bbd-istio-data\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:28:41.274588 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:28:41.274532    4284 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fd46495b-8c51-476d-91f9-750529060bbd-istiod-ca-cert\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:28:41.274588 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:28:41.274546    4284 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fd46495b-8c51-476d-91f9-750529060bbd-workload-socket\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:28:41.274588 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:28:41.274562    4284 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fd46495b-8c51-476d-91f9-750529060bbd-workload-certs\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:28:41.274588 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:28:41.274578    4284 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fd46495b-8c51-476d-91f9-750529060bbd-credential-socket\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:28:41.275777 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:28:41.275756    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fd46495b-8c51-476d-91f9-750529060bbd-istio-envoy" pod "fd46495b-8c51-476d-91f9-750529060bbd" (UID: "fd46495b-8c51-476d-91f9-750529060bbd"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:28:41.276064 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:28:41.276044    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fd46495b-8c51-476d-91f9-750529060bbd-kube-api-access-jcbsm" pod "fd46495b-8c51-476d-91f9-750529060bbd" (UID: "fd46495b-8c51-476d-91f9-750529060bbd"). InnerVolumeSpecName "kube-api-access-jcbsm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:28:41.276382 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:28:41.276364    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/fd46495b-8c51-476d-91f9-750529060bbd-istio-podinfo" pod "fd46495b-8c51-476d-91f9-750529060bbd" (UID: "fd46495b-8c51-476d-91f9-750529060bbd"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 01:28:41.276465 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:28:41.276384    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fd46495b-8c51-476d-91f9-750529060bbd-istio-token" pod "fd46495b-8c51-476d-91f9-750529060bbd" (UID: "fd46495b-8c51-476d-91f9-750529060bbd"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:28:41.301120 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:28:41.301083    4284 generic.go:356] "Generic (PLEG): container finished" podID="fd46495b-8c51-476d-91f9-750529060bbd" containerID="e9ebf0b02fcd8550bde2f0e6cae0ad98fe7c51b171e1397767f0556648bdb0b5" exitCode=0
Oct 31 01:28:41.301276 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:28:41.301131    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-4397fc-openshift-default-6b458f575f-tvgdj"
Oct 31 01:28:41.301276 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:28:41.301144    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-4397fc-openshift-default-6b458f575f-tvgdj" event={"ID":"fd46495b-8c51-476d-91f9-750529060bbd","Type":"ContainerDied","Data":"e9ebf0b02fcd8550bde2f0e6cae0ad98fe7c51b171e1397767f0556648bdb0b5"}
Oct 31 01:28:41.301276 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:28:41.301247    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-4397fc-openshift-default-6b458f575f-tvgdj" event={"ID":"fd46495b-8c51-476d-91f9-750529060bbd","Type":"ContainerDied","Data":"b417862b33116ed4bb863f8c3144aef2a173736adbcbf30ca6bf23c5dc0134e2"}
Oct 31 01:28:41.301460 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:28:41.301281    4284 scope.go:122] "RemoveContainer" containerID="e9ebf0b02fcd8550bde2f0e6cae0ad98fe7c51b171e1397767f0556648bdb0b5"
Oct 31 01:28:41.311248 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:28:41.311232    4284 scope.go:122] "RemoveContainer" containerID="1c21af988df8b2ba6536445f69f14974cbba7dbb136bb74eb436fca4bf48fefe"
Oct 31 01:28:41.319532 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:28:41.319513    4284 scope.go:122] "RemoveContainer" containerID="e9ebf0b02fcd8550bde2f0e6cae0ad98fe7c51b171e1397767f0556648bdb0b5"
Oct 31 01:28:41.319795 ip-10-0-132-173 kubenswrapper[4284]: E1031 01:28:41.319778    4284 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e9ebf0b02fcd8550bde2f0e6cae0ad98fe7c51b171e1397767f0556648bdb0b5\": container with ID starting with e9ebf0b02fcd8550bde2f0e6cae0ad98fe7c51b171e1397767f0556648bdb0b5 not found: ID does not exist" containerID="e9ebf0b02fcd8550bde2f0e6cae0ad98fe7c51b171e1397767f0556648bdb0b5"
Oct 31 01:28:41.319853 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:28:41.319801    4284 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e9ebf0b02fcd8550bde2f0e6cae0ad98fe7c51b171e1397767f0556648bdb0b5"} err="failed to get container status \"e9ebf0b02fcd8550bde2f0e6cae0ad98fe7c51b171e1397767f0556648bdb0b5\": rpc error: code = NotFound desc = could not find container \"e9ebf0b02fcd8550bde2f0e6cae0ad98fe7c51b171e1397767f0556648bdb0b5\": container with ID starting with e9ebf0b02fcd8550bde2f0e6cae0ad98fe7c51b171e1397767f0556648bdb0b5 not found: ID does not exist"
Oct 31 01:28:41.319853 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:28:41.319820    4284 scope.go:122] "RemoveContainer" containerID="1c21af988df8b2ba6536445f69f14974cbba7dbb136bb74eb436fca4bf48fefe"
Oct 31 01:28:41.320047 ip-10-0-132-173 kubenswrapper[4284]: E1031 01:28:41.320032    4284 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1c21af988df8b2ba6536445f69f14974cbba7dbb136bb74eb436fca4bf48fefe\": container with ID starting with 1c21af988df8b2ba6536445f69f14974cbba7dbb136bb74eb436fca4bf48fefe not found: ID does not exist" containerID="1c21af988df8b2ba6536445f69f14974cbba7dbb136bb74eb436fca4bf48fefe"
Oct 31 01:28:41.320085 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:28:41.320050    4284 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1c21af988df8b2ba6536445f69f14974cbba7dbb136bb74eb436fca4bf48fefe"} err="failed to get container status \"1c21af988df8b2ba6536445f69f14974cbba7dbb136bb74eb436fca4bf48fefe\": rpc error: code = NotFound desc = could not find container \"1c21af988df8b2ba6536445f69f14974cbba7dbb136bb74eb436fca4bf48fefe\": container with ID starting with 1c21af988df8b2ba6536445f69f14974cbba7dbb136bb74eb436fca4bf48fefe not found: ID does not exist"
Oct 31 01:28:41.375663 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:28:41.375627    4284 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fd46495b-8c51-476d-91f9-750529060bbd-istio-envoy\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:28:41.375663 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:28:41.375659    4284 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fd46495b-8c51-476d-91f9-750529060bbd-istio-podinfo\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:28:41.375869 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:28:41.375675    4284 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jcbsm\" (UniqueName: \"kubernetes.io/projected/fd46495b-8c51-476d-91f9-750529060bbd-kube-api-access-jcbsm\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:28:41.375869 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:28:41.375688    4284 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fd46495b-8c51-476d-91f9-750529060bbd-istio-token\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:28:41.449054 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:28:41.448975    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-4397fc-openshift-default-6b458f575f-tvgdj"]
Oct 31 01:28:41.453026 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:28:41.453002    4284 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-4397fc-openshift-default-6b458f575f-tvgdj"]
Oct 31 01:28:41.962566 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:28:41.962531    4284 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="fd46495b-8c51-476d-91f9-750529060bbd" path="/var/lib/kubelet/pods/fd46495b-8c51-476d-91f9-750529060bbd/volumes"
Oct 31 01:28:43.197852 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:28:43.197822    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-b-9332a5-5bd6d599b-t2454"]
Oct 31 01:28:43.198291 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:28:43.198032    4284 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-9332a5-5bd6d599b-t2454" podUID="40b79195-6d51-4041-9cbb-5acc6f1e7345" containerName="maas-api" containerID="cri-o://9e768c9b62eb3cef7402630ab5bea547457dc52b05a44e4fa76dfefb0cd4b2f2" gracePeriod=30
Oct 31 01:28:43.299721 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:28:43.299678    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:28:43.315092 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:28:43.315065    4284 generic.go:356] "Generic (PLEG): container finished" podID="40b79195-6d51-4041-9cbb-5acc6f1e7345" containerID="9e768c9b62eb3cef7402630ab5bea547457dc52b05a44e4fa76dfefb0cd4b2f2" exitCode=0
Oct 31 01:28:43.315294 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:28:43.315135    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-9332a5-5bd6d599b-t2454" event={"ID":"40b79195-6d51-4041-9cbb-5acc6f1e7345","Type":"ContainerDied","Data":"9e768c9b62eb3cef7402630ab5bea547457dc52b05a44e4fa76dfefb0cd4b2f2"}
Oct 31 01:28:43.471688 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:28:43.471665    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-9332a5-5bd6d599b-t2454"
Oct 31 01:28:43.610078 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:28:43.610038    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/40b79195-6d51-4041-9cbb-5acc6f1e7345-metrics-certs\" (UniqueName: \"kubernetes.io/secret/40b79195-6d51-4041-9cbb-5acc6f1e7345-metrics-certs\") pod \"40b79195-6d51-4041-9cbb-5acc6f1e7345\" (UID: \"40b79195-6d51-4041-9cbb-5acc6f1e7345\") "
Oct 31 01:28:43.610319 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:28:43.610110    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/40b79195-6d51-4041-9cbb-5acc6f1e7345-kube-api-access-8jmpc\" (UniqueName: \"kubernetes.io/projected/40b79195-6d51-4041-9cbb-5acc6f1e7345-kube-api-access-8jmpc\") pod \"40b79195-6d51-4041-9cbb-5acc6f1e7345\" (UID: \"40b79195-6d51-4041-9cbb-5acc6f1e7345\") "
Oct 31 01:28:43.610395 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:28:43.610328    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/40b79195-6d51-4041-9cbb-5acc6f1e7345-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/40b79195-6d51-4041-9cbb-5acc6f1e7345-openshift-service-ca\") pod \"40b79195-6d51-4041-9cbb-5acc6f1e7345\" (UID: \"40b79195-6d51-4041-9cbb-5acc6f1e7345\") "
Oct 31 01:28:43.610395 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:28:43.610359    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/40b79195-6d51-4041-9cbb-5acc6f1e7345-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/40b79195-6d51-4041-9cbb-5acc6f1e7345-maas-api-tls\") pod \"40b79195-6d51-4041-9cbb-5acc6f1e7345\" (UID: \"40b79195-6d51-4041-9cbb-5acc6f1e7345\") "
Oct 31 01:28:43.610719 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:28:43.610688    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/40b79195-6d51-4041-9cbb-5acc6f1e7345-openshift-service-ca" pod "40b79195-6d51-4041-9cbb-5acc6f1e7345" (UID: "40b79195-6d51-4041-9cbb-5acc6f1e7345"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 01:28:43.610979 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:28:43.610960    4284 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/40b79195-6d51-4041-9cbb-5acc6f1e7345-openshift-service-ca\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:28:43.612654 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:28:43.612630    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/40b79195-6d51-4041-9cbb-5acc6f1e7345-metrics-certs" pod "40b79195-6d51-4041-9cbb-5acc6f1e7345" (UID: "40b79195-6d51-4041-9cbb-5acc6f1e7345"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 01:28:43.612950 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:28:43.612931    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/40b79195-6d51-4041-9cbb-5acc6f1e7345-kube-api-access-8jmpc" pod "40b79195-6d51-4041-9cbb-5acc6f1e7345" (UID: "40b79195-6d51-4041-9cbb-5acc6f1e7345"). InnerVolumeSpecName "kube-api-access-8jmpc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:28:43.614420 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:28:43.614403    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/40b79195-6d51-4041-9cbb-5acc6f1e7345-maas-api-tls" pod "40b79195-6d51-4041-9cbb-5acc6f1e7345" (UID: "40b79195-6d51-4041-9cbb-5acc6f1e7345"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 01:28:43.713607 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:28:43.713011    4284 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/40b79195-6d51-4041-9cbb-5acc6f1e7345-metrics-certs\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:28:43.713607 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:28:43.713053    4284 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8jmpc\" (UniqueName: \"kubernetes.io/projected/40b79195-6d51-4041-9cbb-5acc6f1e7345-kube-api-access-8jmpc\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:28:43.713607 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:28:43.713068    4284 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/40b79195-6d51-4041-9cbb-5acc6f1e7345-maas-api-tls\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:28:44.322312 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:28:44.322200    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-9332a5-5bd6d599b-t2454" event={"ID":"40b79195-6d51-4041-9cbb-5acc6f1e7345","Type":"ContainerDied","Data":"d5e163e8926b8066516bed3928c90c8353ba1264a4d5851d54015b70937855d5"}
Oct 31 01:28:44.322312 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:28:44.322289    4284 scope.go:122] "RemoveContainer" containerID="9e768c9b62eb3cef7402630ab5bea547457dc52b05a44e4fa76dfefb0cd4b2f2"
Oct 31 01:28:44.322884 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:28:44.322240    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-9332a5-5bd6d599b-t2454"
Oct 31 01:28:44.480147 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:28:44.480116    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-b-9332a5-5bd6d599b-t2454"]
Oct 31 01:28:44.530733 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:28:44.530691    4284 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-b-9332a5-5bd6d599b-t2454"]
Oct 31 01:28:44.767728 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:28:44.767686    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:28:44.982511 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:28:44.982477    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:28:45.126043 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:28:45.125959    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:28:45.670504 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:28:45.670470    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:28:45.963121 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:28:45.963048    4284 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="40b79195-6d51-4041-9cbb-5acc6f1e7345" path="/var/lib/kubelet/pods/40b79195-6d51-4041-9cbb-5acc6f1e7345/volumes"
Oct 31 01:28:47.015231 ip-10-0-132-173 kubenswrapper[4284]: E1031 01:28:47.015142    4284 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fd8sp" containerName="extractor"
Oct 31 01:28:48.707573 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:28:48.707534    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:28:48.893310 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:28:48.893280    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:28:50.184691 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:28:50.184657    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:28:52.190521 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:28:52.190482    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:28:52.333449 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:28:52.333418    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:28:52.559257 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:28:52.559137    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:28:55.308352 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:28:55.308319    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:29:00.599435 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:00.599384    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:29:05.043947 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:05.043910    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w4-8fb7f0-5889f6f4ff-r76rq"]
Oct 31 01:29:05.156875 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:05.156839    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w4-8fb7f0-796d4996ff-25jt7"]
Oct 31 01:29:05.415663 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:05.415632    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:29:05.611800 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:05.611758    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:29:05.766347 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:05.766239    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:29:09.364931 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:09.364894    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:29:10.044495 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:10.044435    4284 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-a-w4-8fb7f0-5889f6f4ff-r76rq" podUID="ddda98fe-7ad3-4592-81bb-2b6ce526b882" containerName="payload-processing" containerID="cri-o://e61a9cf1a0f18e656164387b56c018d2c3d74bea74a088f205fbe2f7b5f069e2" gracePeriod=25
Oct 31 01:29:10.157387 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:10.157338    4284 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w4-8fb7f0-796d4996ff-25jt7" podUID="0e3a22bb-0915-4025-af10-c9c3b01011f1" containerName="payload-pre-processing" containerID="cri-o://22fb4eaee3719a5c3488797ccf2689eb080cd33bba6ccaed7ef117650b76b950" gracePeriod=25
Oct 31 01:29:10.920809 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:10.920769    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:29:11.746192 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:11.746147    4284 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w4-8fb7f0-796d4996ff-25jt7" podUID="0e3a22bb-0915-4025-af10-c9c3b01011f1" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 31 01:29:11.746399 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:11.746147    4284 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-shared-a-w4-8fb7f0-5889f6f4ff-r76rq" podUID="ddda98fe-7ad3-4592-81bb-2b6ce526b882" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 31 01:29:15.684005 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:15.683958    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:29:17.013392 ip-10-0-132-173 kubenswrapper[4284]: E1031 01:29:17.013358    4284 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fd8sp" containerName="extractor"
Oct 31 01:29:19.747653 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:19.747617    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-3be6fa-5db9dc4847-46thz"]
Oct 31 01:29:19.748105 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:19.747871    4284 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-3be6fa-5db9dc4847-46thz" podUID="e38c82a4-98f4-48a6-b888-f95877335bd1" containerName="maas-api" containerID="cri-o://cbd2003858448580b240b709299c93bf893fd47219bd3c1409101c48ee7603eb" gracePeriod=30
Oct 31 01:29:19.851899 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:19.851840    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:29:20.038005 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:20.037981    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-3be6fa-5db9dc4847-46thz"
Oct 31 01:29:20.165893 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:20.165859    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/e38c82a4-98f4-48a6-b888-f95877335bd1-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e38c82a4-98f4-48a6-b888-f95877335bd1-maas-api-tls\") pod \"e38c82a4-98f4-48a6-b888-f95877335bd1\" (UID: \"e38c82a4-98f4-48a6-b888-f95877335bd1\") "
Oct 31 01:29:20.166091 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:20.165954    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e38c82a4-98f4-48a6-b888-f95877335bd1-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e38c82a4-98f4-48a6-b888-f95877335bd1-openshift-service-ca\") pod \"e38c82a4-98f4-48a6-b888-f95877335bd1\" (UID: \"e38c82a4-98f4-48a6-b888-f95877335bd1\") "
Oct 31 01:29:20.166091 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:20.166009    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e38c82a4-98f4-48a6-b888-f95877335bd1-kube-api-access-6vqcf\" (UniqueName: \"kubernetes.io/projected/e38c82a4-98f4-48a6-b888-f95877335bd1-kube-api-access-6vqcf\") pod \"e38c82a4-98f4-48a6-b888-f95877335bd1\" (UID: \"e38c82a4-98f4-48a6-b888-f95877335bd1\") "
Oct 31 01:29:20.166242 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:20.166156    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/e38c82a4-98f4-48a6-b888-f95877335bd1-metrics-certs\" (UniqueName: \"kubernetes.io/secret/e38c82a4-98f4-48a6-b888-f95877335bd1-metrics-certs\") pod \"e38c82a4-98f4-48a6-b888-f95877335bd1\" (UID: \"e38c82a4-98f4-48a6-b888-f95877335bd1\") "
Oct 31 01:29:20.166386 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:20.166354    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e38c82a4-98f4-48a6-b888-f95877335bd1-openshift-service-ca" pod "e38c82a4-98f4-48a6-b888-f95877335bd1" (UID: "e38c82a4-98f4-48a6-b888-f95877335bd1"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 01:29:20.166681 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:20.166618    4284 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e38c82a4-98f4-48a6-b888-f95877335bd1-openshift-service-ca\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:29:20.168255 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:20.168230    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e38c82a4-98f4-48a6-b888-f95877335bd1-maas-api-tls" pod "e38c82a4-98f4-48a6-b888-f95877335bd1" (UID: "e38c82a4-98f4-48a6-b888-f95877335bd1"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 01:29:20.168725 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:20.168692    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e38c82a4-98f4-48a6-b888-f95877335bd1-kube-api-access-6vqcf" pod "e38c82a4-98f4-48a6-b888-f95877335bd1" (UID: "e38c82a4-98f4-48a6-b888-f95877335bd1"). InnerVolumeSpecName "kube-api-access-6vqcf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:29:20.168844 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:20.168745    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e38c82a4-98f4-48a6-b888-f95877335bd1-metrics-certs" pod "e38c82a4-98f4-48a6-b888-f95877335bd1" (UID: "e38c82a4-98f4-48a6-b888-f95877335bd1"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 01:29:20.279177 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:20.279040    4284 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e38c82a4-98f4-48a6-b888-f95877335bd1-maas-api-tls\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:29:20.279177 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:20.279076    4284 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6vqcf\" (UniqueName: \"kubernetes.io/projected/e38c82a4-98f4-48a6-b888-f95877335bd1-kube-api-access-6vqcf\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:29:20.279177 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:20.279093    4284 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e38c82a4-98f4-48a6-b888-f95877335bd1-metrics-certs\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:29:20.525721 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:20.525684    4284 generic.go:356] "Generic (PLEG): container finished" podID="e38c82a4-98f4-48a6-b888-f95877335bd1" containerID="cbd2003858448580b240b709299c93bf893fd47219bd3c1409101c48ee7603eb" exitCode=0
Oct 31 01:29:20.525906 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:20.525720    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-3be6fa-5db9dc4847-46thz" event={"ID":"e38c82a4-98f4-48a6-b888-f95877335bd1","Type":"ContainerDied","Data":"cbd2003858448580b240b709299c93bf893fd47219bd3c1409101c48ee7603eb"}
Oct 31 01:29:20.525906 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:20.525743    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-3be6fa-5db9dc4847-46thz"
Oct 31 01:29:20.525906 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:20.525773    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-3be6fa-5db9dc4847-46thz" event={"ID":"e38c82a4-98f4-48a6-b888-f95877335bd1","Type":"ContainerDied","Data":"972b3a4deb45dbe9851879f9d2ecb4f2e6fd45e5f1725d50df6471902aa467d4"}
Oct 31 01:29:20.525906 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:20.525804    4284 scope.go:122] "RemoveContainer" containerID="cbd2003858448580b240b709299c93bf893fd47219bd3c1409101c48ee7603eb"
Oct 31 01:29:20.536734 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:20.536717    4284 scope.go:122] "RemoveContainer" containerID="cbd2003858448580b240b709299c93bf893fd47219bd3c1409101c48ee7603eb"
Oct 31 01:29:20.536983 ip-10-0-132-173 kubenswrapper[4284]: E1031 01:29:20.536957    4284 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cbd2003858448580b240b709299c93bf893fd47219bd3c1409101c48ee7603eb\": container with ID starting with cbd2003858448580b240b709299c93bf893fd47219bd3c1409101c48ee7603eb not found: ID does not exist" containerID="cbd2003858448580b240b709299c93bf893fd47219bd3c1409101c48ee7603eb"
Oct 31 01:29:20.537069 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:20.536993    4284 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cbd2003858448580b240b709299c93bf893fd47219bd3c1409101c48ee7603eb"} err="failed to get container status \"cbd2003858448580b240b709299c93bf893fd47219bd3c1409101c48ee7603eb\": rpc error: code = NotFound desc = could not find container \"cbd2003858448580b240b709299c93bf893fd47219bd3c1409101c48ee7603eb\": container with ID starting with cbd2003858448580b240b709299c93bf893fd47219bd3c1409101c48ee7603eb not found: ID does not exist"
Oct 31 01:29:20.688969 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:20.688932    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-3be6fa-5db9dc4847-46thz"]
Oct 31 01:29:20.691782 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:20.691714    4284 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-3be6fa-5db9dc4847-46thz"]
Oct 31 01:29:21.365685 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:21.365652    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:29:21.573974 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:21.573930    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:29:21.717519 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:21.717471    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:29:21.746146 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:21.746099    4284 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-shared-a-w4-8fb7f0-5889f6f4ff-r76rq" podUID="ddda98fe-7ad3-4592-81bb-2b6ce526b882" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 31 01:29:21.746341 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:21.746107    4284 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w4-8fb7f0-796d4996ff-25jt7" podUID="0e3a22bb-0915-4025-af10-c9c3b01011f1" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 31 01:29:21.962312 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:21.962278    4284 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e38c82a4-98f4-48a6-b888-f95877335bd1" path="/var/lib/kubelet/pods/e38c82a4-98f4-48a6-b888-f95877335bd1/volumes"
Oct 31 01:29:22.342592 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:22.342561    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:29:26.003485 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:26.003447    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:29:27.002654 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:27.002620    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-f9925a-7b8b48d8d7-z2n6r"]
Oct 31 01:29:27.002962 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:27.002821    4284 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-f9925a-7b8b48d8d7-z2n6r" podUID="f763b240-930c-4044-8c59-02cc2bd614d7" containerName="maas-api" containerID="cri-o://4e35d3294b9cb82929171716d4ec2d5f43a1cd3db233576e758f80fd189d631b" gracePeriod=30
Oct 31 01:29:27.100266 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:27.100231    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:29:27.287064 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:27.287037    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-f9925a-7b8b48d8d7-z2n6r"
Oct 31 01:29:27.355297 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:27.355248    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f763b240-930c-4044-8c59-02cc2bd614d7-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f763b240-930c-4044-8c59-02cc2bd614d7-openshift-service-ca\") pod \"f763b240-930c-4044-8c59-02cc2bd614d7\" (UID: \"f763b240-930c-4044-8c59-02cc2bd614d7\") "
Oct 31 01:29:27.356238 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:27.356062    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/f763b240-930c-4044-8c59-02cc2bd614d7-metrics-certs\" (UniqueName: \"kubernetes.io/secret/f763b240-930c-4044-8c59-02cc2bd614d7-metrics-certs\") pod \"f763b240-930c-4044-8c59-02cc2bd614d7\" (UID: \"f763b240-930c-4044-8c59-02cc2bd614d7\") "
Oct 31 01:29:27.356238 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:27.356104    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/f763b240-930c-4044-8c59-02cc2bd614d7-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f763b240-930c-4044-8c59-02cc2bd614d7-maas-api-tls\") pod \"f763b240-930c-4044-8c59-02cc2bd614d7\" (UID: \"f763b240-930c-4044-8c59-02cc2bd614d7\") "
Oct 31 01:29:27.356238 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:27.356138    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f763b240-930c-4044-8c59-02cc2bd614d7-kube-api-access-r4mds\" (UniqueName: \"kubernetes.io/projected/f763b240-930c-4044-8c59-02cc2bd614d7-kube-api-access-r4mds\") pod \"f763b240-930c-4044-8c59-02cc2bd614d7\" (UID: \"f763b240-930c-4044-8c59-02cc2bd614d7\") "
Oct 31 01:29:27.357051 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:27.355917    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f763b240-930c-4044-8c59-02cc2bd614d7-openshift-service-ca" pod "f763b240-930c-4044-8c59-02cc2bd614d7" (UID: "f763b240-930c-4044-8c59-02cc2bd614d7"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 01:29:27.358761 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:27.357909    4284 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f763b240-930c-4044-8c59-02cc2bd614d7-openshift-service-ca\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:29:27.386470 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:27.385018    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f763b240-930c-4044-8c59-02cc2bd614d7-maas-api-tls" pod "f763b240-930c-4044-8c59-02cc2bd614d7" (UID: "f763b240-930c-4044-8c59-02cc2bd614d7"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 01:29:27.386470 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:27.385136    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f763b240-930c-4044-8c59-02cc2bd614d7-kube-api-access-r4mds" pod "f763b240-930c-4044-8c59-02cc2bd614d7" (UID: "f763b240-930c-4044-8c59-02cc2bd614d7"). InnerVolumeSpecName "kube-api-access-r4mds". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:29:27.386737 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:27.386479    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f763b240-930c-4044-8c59-02cc2bd614d7-metrics-certs" pod "f763b240-930c-4044-8c59-02cc2bd614d7" (UID: "f763b240-930c-4044-8c59-02cc2bd614d7"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 01:29:27.456169 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:27.456134    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:29:27.459392 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:27.459366    4284 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f763b240-930c-4044-8c59-02cc2bd614d7-metrics-certs\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:29:27.459392 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:27.459390    4284 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f763b240-930c-4044-8c59-02cc2bd614d7-maas-api-tls\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:29:27.459572 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:27.459400    4284 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r4mds\" (UniqueName: \"kubernetes.io/projected/f763b240-930c-4044-8c59-02cc2bd614d7-kube-api-access-r4mds\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:29:27.564954 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:27.564863    4284 generic.go:356] "Generic (PLEG): container finished" podID="f763b240-930c-4044-8c59-02cc2bd614d7" containerID="4e35d3294b9cb82929171716d4ec2d5f43a1cd3db233576e758f80fd189d631b" exitCode=0
Oct 31 01:29:27.564954 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:27.564908    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-f9925a-7b8b48d8d7-z2n6r"
Oct 31 01:29:27.565237 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:27.564948    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-f9925a-7b8b48d8d7-z2n6r" event={"ID":"f763b240-930c-4044-8c59-02cc2bd614d7","Type":"ContainerDied","Data":"4e35d3294b9cb82929171716d4ec2d5f43a1cd3db233576e758f80fd189d631b"}
Oct 31 01:29:27.565237 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:27.565006    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-f9925a-7b8b48d8d7-z2n6r" event={"ID":"f763b240-930c-4044-8c59-02cc2bd614d7","Type":"ContainerDied","Data":"5710bcb191778bb8aa957cf698fa86e83024bb7a4e34d2380f9a01e412373502"}
Oct 31 01:29:27.565237 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:27.565040    4284 scope.go:122] "RemoveContainer" containerID="4e35d3294b9cb82929171716d4ec2d5f43a1cd3db233576e758f80fd189d631b"
Oct 31 01:29:27.575598 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:27.575581    4284 scope.go:122] "RemoveContainer" containerID="4e35d3294b9cb82929171716d4ec2d5f43a1cd3db233576e758f80fd189d631b"
Oct 31 01:29:27.575871 ip-10-0-132-173 kubenswrapper[4284]: E1031 01:29:27.575852    4284 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4e35d3294b9cb82929171716d4ec2d5f43a1cd3db233576e758f80fd189d631b\": container with ID starting with 4e35d3294b9cb82929171716d4ec2d5f43a1cd3db233576e758f80fd189d631b not found: ID does not exist" containerID="4e35d3294b9cb82929171716d4ec2d5f43a1cd3db233576e758f80fd189d631b"
Oct 31 01:29:27.575930 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:27.575883    4284 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4e35d3294b9cb82929171716d4ec2d5f43a1cd3db233576e758f80fd189d631b"} err="failed to get container status \"4e35d3294b9cb82929171716d4ec2d5f43a1cd3db233576e758f80fd189d631b\": rpc error: code = NotFound desc = could not find container \"4e35d3294b9cb82929171716d4ec2d5f43a1cd3db233576e758f80fd189d631b\": container with ID starting with 4e35d3294b9cb82929171716d4ec2d5f43a1cd3db233576e758f80fd189d631b not found: ID does not exist"
Oct 31 01:29:27.722851 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:27.722818    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-f9925a-7b8b48d8d7-z2n6r"]
Oct 31 01:29:27.725061 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:27.725036    4284 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-f9925a-7b8b48d8d7-z2n6r"]
Oct 31 01:29:27.725395 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:27.725376    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:29:27.963346 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:27.963316    4284 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f763b240-930c-4044-8c59-02cc2bd614d7" path="/var/lib/kubelet/pods/f763b240-930c-4044-8c59-02cc2bd614d7/volumes"
Oct 31 01:29:28.402188 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:28.402085    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:29:28.623814 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:28.623780    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:29:28.878880 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:28.878845    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:29:29.078733 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:29.078697    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:29:29.227335 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:29.227296    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:29:29.398329 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:29.398297    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:29:29.546171 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:29.546073    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:29:29.943311 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:29.943277    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:29:30.327053 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:30.327027    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w4-8fb7f0-5889f6f4ff-r76rq"
Oct 31 01:29:30.414370 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:30.414340    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ddda98fe-7ad3-4592-81bb-2b6ce526b882-kube-api-access-ggl82\" (UniqueName: \"kubernetes.io/projected/ddda98fe-7ad3-4592-81bb-2b6ce526b882-kube-api-access-ggl82\") pod \"ddda98fe-7ad3-4592-81bb-2b6ce526b882\" (UID: \"ddda98fe-7ad3-4592-81bb-2b6ce526b882\") "
Oct 31 01:29:30.414531 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:30.414427    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ddda98fe-7ad3-4592-81bb-2b6ce526b882-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ddda98fe-7ad3-4592-81bb-2b6ce526b882-plugins-config-volume\") pod \"ddda98fe-7ad3-4592-81bb-2b6ce526b882\" (UID: \"ddda98fe-7ad3-4592-81bb-2b6ce526b882\") "
Oct 31 01:29:30.414870 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:30.414845    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ddda98fe-7ad3-4592-81bb-2b6ce526b882-plugins-config-volume" pod "ddda98fe-7ad3-4592-81bb-2b6ce526b882" (UID: "ddda98fe-7ad3-4592-81bb-2b6ce526b882"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 01:29:30.417104 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:30.417077    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ddda98fe-7ad3-4592-81bb-2b6ce526b882-kube-api-access-ggl82" pod "ddda98fe-7ad3-4592-81bb-2b6ce526b882" (UID: "ddda98fe-7ad3-4592-81bb-2b6ce526b882"). InnerVolumeSpecName "kube-api-access-ggl82". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:29:30.434737 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:30.434718    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w4-8fb7f0-796d4996ff-25jt7"
Oct 31 01:29:30.516288 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:30.516195    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0e3a22bb-0915-4025-af10-c9c3b01011f1-kube-api-access-dwxhj\" (UniqueName: \"kubernetes.io/projected/0e3a22bb-0915-4025-af10-c9c3b01011f1-kube-api-access-dwxhj\") pod \"0e3a22bb-0915-4025-af10-c9c3b01011f1\" (UID: \"0e3a22bb-0915-4025-af10-c9c3b01011f1\") "
Oct 31 01:29:30.516288 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:30.516269    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0e3a22bb-0915-4025-af10-c9c3b01011f1-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0e3a22bb-0915-4025-af10-c9c3b01011f1-plugins-config-volume\") pod \"0e3a22bb-0915-4025-af10-c9c3b01011f1\" (UID: \"0e3a22bb-0915-4025-af10-c9c3b01011f1\") "
Oct 31 01:29:30.516569 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:30.516554    4284 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ggl82\" (UniqueName: \"kubernetes.io/projected/ddda98fe-7ad3-4592-81bb-2b6ce526b882-kube-api-access-ggl82\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:29:30.516621 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:30.516573    4284 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ddda98fe-7ad3-4592-81bb-2b6ce526b882-plugins-config-volume\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:29:30.516621 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:30.516601    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0e3a22bb-0915-4025-af10-c9c3b01011f1-plugins-config-volume" pod "0e3a22bb-0915-4025-af10-c9c3b01011f1" (UID: "0e3a22bb-0915-4025-af10-c9c3b01011f1"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 01:29:30.518371 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:30.518352    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0e3a22bb-0915-4025-af10-c9c3b01011f1-kube-api-access-dwxhj" pod "0e3a22bb-0915-4025-af10-c9c3b01011f1" (UID: "0e3a22bb-0915-4025-af10-c9c3b01011f1"). InnerVolumeSpecName "kube-api-access-dwxhj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:29:30.584923 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:30.584892    4284 generic.go:356] "Generic (PLEG): container finished" podID="ddda98fe-7ad3-4592-81bb-2b6ce526b882" containerID="e61a9cf1a0f18e656164387b56c018d2c3d74bea74a088f205fbe2f7b5f069e2" exitCode=0
Oct 31 01:29:30.585419 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:30.584936    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w4-8fb7f0-5889f6f4ff-r76rq"
Oct 31 01:29:30.585419 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:30.584977    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w4-8fb7f0-5889f6f4ff-r76rq" event={"ID":"ddda98fe-7ad3-4592-81bb-2b6ce526b882","Type":"ContainerDied","Data":"e61a9cf1a0f18e656164387b56c018d2c3d74bea74a088f205fbe2f7b5f069e2"}
Oct 31 01:29:30.585419 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:30.585071    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w4-8fb7f0-5889f6f4ff-r76rq" event={"ID":"ddda98fe-7ad3-4592-81bb-2b6ce526b882","Type":"ContainerDied","Data":"e11a398c65639729e018aa685b60345eb7903d068f4ecebdf17f358c46603a67"}
Oct 31 01:29:30.585419 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:30.585106    4284 scope.go:122] "RemoveContainer" containerID="e61a9cf1a0f18e656164387b56c018d2c3d74bea74a088f205fbe2f7b5f069e2"
Oct 31 01:29:30.586537 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:30.586515    4284 generic.go:356] "Generic (PLEG): container finished" podID="0e3a22bb-0915-4025-af10-c9c3b01011f1" containerID="22fb4eaee3719a5c3488797ccf2689eb080cd33bba6ccaed7ef117650b76b950" exitCode=0
Oct 31 01:29:30.586695 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:30.586567    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w4-8fb7f0-796d4996ff-25jt7"
Oct 31 01:29:30.586695 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:30.586564    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w4-8fb7f0-796d4996ff-25jt7" event={"ID":"0e3a22bb-0915-4025-af10-c9c3b01011f1","Type":"ContainerDied","Data":"22fb4eaee3719a5c3488797ccf2689eb080cd33bba6ccaed7ef117650b76b950"}
Oct 31 01:29:30.586695 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:30.586624    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w4-8fb7f0-796d4996ff-25jt7" event={"ID":"0e3a22bb-0915-4025-af10-c9c3b01011f1","Type":"ContainerDied","Data":"e21a82c12b20917dd49f2e7fd02e073fd0c13b176b885ee465b98ef4ee9e12ac"}
Oct 31 01:29:30.598494 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:30.598475    4284 scope.go:122] "RemoveContainer" containerID="e61a9cf1a0f18e656164387b56c018d2c3d74bea74a088f205fbe2f7b5f069e2"
Oct 31 01:29:30.598995 ip-10-0-132-173 kubenswrapper[4284]: E1031 01:29:30.598795    4284 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e61a9cf1a0f18e656164387b56c018d2c3d74bea74a088f205fbe2f7b5f069e2\": container with ID starting with e61a9cf1a0f18e656164387b56c018d2c3d74bea74a088f205fbe2f7b5f069e2 not found: ID does not exist" containerID="e61a9cf1a0f18e656164387b56c018d2c3d74bea74a088f205fbe2f7b5f069e2"
Oct 31 01:29:30.598995 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:30.598819    4284 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e61a9cf1a0f18e656164387b56c018d2c3d74bea74a088f205fbe2f7b5f069e2"} err="failed to get container status \"e61a9cf1a0f18e656164387b56c018d2c3d74bea74a088f205fbe2f7b5f069e2\": rpc error: code = NotFound desc = could not find container \"e61a9cf1a0f18e656164387b56c018d2c3d74bea74a088f205fbe2f7b5f069e2\": container with ID starting with e61a9cf1a0f18e656164387b56c018d2c3d74bea74a088f205fbe2f7b5f069e2 not found: ID does not exist"
Oct 31 01:29:30.598995 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:30.598835    4284 scope.go:122] "RemoveContainer" containerID="22fb4eaee3719a5c3488797ccf2689eb080cd33bba6ccaed7ef117650b76b950"
Oct 31 01:29:30.607332 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:30.607308    4284 scope.go:122] "RemoveContainer" containerID="22fb4eaee3719a5c3488797ccf2689eb080cd33bba6ccaed7ef117650b76b950"
Oct 31 01:29:30.607635 ip-10-0-132-173 kubenswrapper[4284]: E1031 01:29:30.607613    4284 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"22fb4eaee3719a5c3488797ccf2689eb080cd33bba6ccaed7ef117650b76b950\": container with ID starting with 22fb4eaee3719a5c3488797ccf2689eb080cd33bba6ccaed7ef117650b76b950 not found: ID does not exist" containerID="22fb4eaee3719a5c3488797ccf2689eb080cd33bba6ccaed7ef117650b76b950"
Oct 31 01:29:30.607683 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:30.607643    4284 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"22fb4eaee3719a5c3488797ccf2689eb080cd33bba6ccaed7ef117650b76b950"} err="failed to get container status \"22fb4eaee3719a5c3488797ccf2689eb080cd33bba6ccaed7ef117650b76b950\": rpc error: code = NotFound desc = could not find container \"22fb4eaee3719a5c3488797ccf2689eb080cd33bba6ccaed7ef117650b76b950\": container with ID starting with 22fb4eaee3719a5c3488797ccf2689eb080cd33bba6ccaed7ef117650b76b950 not found: ID does not exist"
Oct 31 01:29:30.617272 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:30.617253    4284 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dwxhj\" (UniqueName: \"kubernetes.io/projected/0e3a22bb-0915-4025-af10-c9c3b01011f1-kube-api-access-dwxhj\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:29:30.617349 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:30.617274    4284 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0e3a22bb-0915-4025-af10-c9c3b01011f1-plugins-config-volume\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:29:30.742653 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:30.742624    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w4-8fb7f0-796d4996ff-25jt7"]
Oct 31 01:29:30.748407 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:30.748367    4284 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w4-8fb7f0-796d4996ff-25jt7"]
Oct 31 01:29:30.852744 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:30.852658    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w4-8fb7f0-5889f6f4ff-r76rq"]
Oct 31 01:29:30.857205 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:30.857177    4284 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w4-8fb7f0-5889f6f4ff-r76rq"]
Oct 31 01:29:31.963803 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:31.963769    4284 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0e3a22bb-0915-4025-af10-c9c3b01011f1" path="/var/lib/kubelet/pods/0e3a22bb-0915-4025-af10-c9c3b01011f1/volumes"
Oct 31 01:29:31.964387 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:31.964229    4284 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ddda98fe-7ad3-4592-81bb-2b6ce526b882" path="/var/lib/kubelet/pods/ddda98fe-7ad3-4592-81bb-2b6ce526b882/volumes"
Oct 31 01:29:33.469718 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:33.469683    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:29:34.854565 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:34.854532    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:29:34.962734 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:34.962704    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w1-b63c24/premium-w1-b63c24-kserve-549f786d9-k7g8j"]
Oct 31 01:29:35.079997 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:35.079951    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:29:35.223669 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:35.223632    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:29:35.373750 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:35.373708    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:29:35.515821 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:35.515729    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:29:35.693347 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:35.693307    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:29:36.071174 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:36.071136    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:29:36.320883 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:36.320840    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:29:36.977635 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:36.977599    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:29:37.034422 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:37.034382    4284 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-extmod-b-6bd7c8-openshift-default-5b6745476f-7xkcx"]
Oct 31 01:29:37.034889 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:37.034871    4284 cpu_manager.go:414] "containerMap: removing container" podUID="6aa40f46-c2b1-4c37-a341-d230158a7db9" containerName="istio-proxy"
Oct 31 01:29:37.034948 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:37.034891    4284 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6aa40f46-c2b1-4c37-a341-d230158a7db9" containerName="istio-proxy"
Oct 31 01:29:37.034948 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:37.034905    4284 cpu_manager.go:414] "containerMap: removing container" podUID="f31cff38-07af-4d6c-83ba-fcc15b830cbc" containerName="payload-processing"
Oct 31 01:29:37.034948 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:37.034911    4284 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f31cff38-07af-4d6c-83ba-fcc15b830cbc" containerName="payload-processing"
Oct 31 01:29:37.034948 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:37.034924    4284 cpu_manager.go:414] "containerMap: removing container" podUID="fd46495b-8c51-476d-91f9-750529060bbd" containerName="istio-proxy"
Oct 31 01:29:37.034948 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:37.034930    4284 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="fd46495b-8c51-476d-91f9-750529060bbd" containerName="istio-proxy"
Oct 31 01:29:37.034948 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:37.034936    4284 cpu_manager.go:414] "containerMap: removing container" podUID="b96b200a-f8de-4419-8232-061be10ada45" containerName="payload-processing"
Oct 31 01:29:37.034948 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:37.034941    4284 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b96b200a-f8de-4419-8232-061be10ada45" containerName="payload-processing"
Oct 31 01:29:37.034948 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:37.034948    4284 cpu_manager.go:414] "containerMap: removing container" podUID="178450d7-7cf3-46f9-8e25-e09b73075316" containerName="maas-api"
Oct 31 01:29:37.035333 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:37.034953    4284 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="178450d7-7cf3-46f9-8e25-e09b73075316" containerName="maas-api"
Oct 31 01:29:37.035333 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:37.034966    4284 cpu_manager.go:414] "containerMap: removing container" podUID="fd46495b-8c51-476d-91f9-750529060bbd" containerName="istio-proxy"
Oct 31 01:29:37.035333 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:37.034973    4284 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="fd46495b-8c51-476d-91f9-750529060bbd" containerName="istio-proxy"
Oct 31 01:29:37.035333 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:37.034980    4284 cpu_manager.go:414] "containerMap: removing container" podUID="e38c82a4-98f4-48a6-b888-f95877335bd1" containerName="maas-api"
Oct 31 01:29:37.035333 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:37.034984    4284 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e38c82a4-98f4-48a6-b888-f95877335bd1" containerName="maas-api"
Oct 31 01:29:37.035333 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:37.034989    4284 cpu_manager.go:414] "containerMap: removing container" podUID="40b79195-6d51-4041-9cbb-5acc6f1e7345" containerName="maas-api"
Oct 31 01:29:37.035333 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:37.034994    4284 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="40b79195-6d51-4041-9cbb-5acc6f1e7345" containerName="maas-api"
Oct 31 01:29:37.035333 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:37.035000    4284 cpu_manager.go:414] "containerMap: removing container" podUID="1dde1271-7b40-4dc0-97ca-c1be6b89c477" containerName="istio-proxy"
Oct 31 01:29:37.035333 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:37.035004    4284 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1dde1271-7b40-4dc0-97ca-c1be6b89c477" containerName="istio-proxy"
Oct 31 01:29:37.035333 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:37.035012    4284 cpu_manager.go:414] "containerMap: removing container" podUID="a71dc36e-312b-4dff-b412-056d51f65731" containerName="payload-pre-processing"
Oct 31 01:29:37.035333 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:37.035016    4284 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a71dc36e-312b-4dff-b412-056d51f65731" containerName="payload-pre-processing"
Oct 31 01:29:37.035333 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:37.035023    4284 cpu_manager.go:414] "containerMap: removing container" podUID="1dde1271-7b40-4dc0-97ca-c1be6b89c477" containerName="istio-proxy"
Oct 31 01:29:37.035333 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:37.035028    4284 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1dde1271-7b40-4dc0-97ca-c1be6b89c477" containerName="istio-proxy"
Oct 31 01:29:37.035333 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:37.035038    4284 cpu_manager.go:414] "containerMap: removing container" podUID="a11c7b2d-901c-4c69-9be2-a0bd6eeaf9c0" containerName="maas-api"
Oct 31 01:29:37.035333 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:37.035045    4284 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a11c7b2d-901c-4c69-9be2-a0bd6eeaf9c0" containerName="maas-api"
Oct 31 01:29:37.035333 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:37.035057    4284 cpu_manager.go:414] "containerMap: removing container" podUID="f763b240-930c-4044-8c59-02cc2bd614d7" containerName="maas-api"
Oct 31 01:29:37.035333 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:37.035063    4284 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f763b240-930c-4044-8c59-02cc2bd614d7" containerName="maas-api"
Oct 31 01:29:37.035333 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:37.035074    4284 cpu_manager.go:414] "containerMap: removing container" podUID="6ac9b2b9-a157-4f05-8c7e-e7a2987397f6" containerName="istio-proxy"
Oct 31 01:29:37.035333 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:37.035080    4284 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6ac9b2b9-a157-4f05-8c7e-e7a2987397f6" containerName="istio-proxy"
Oct 31 01:29:37.035333 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:37.035094    4284 cpu_manager.go:414] "containerMap: removing container" podUID="04e2ad9d-c5b9-47b0-ba97-86114775393c" containerName="istio-proxy"
Oct 31 01:29:37.035333 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:37.035105    4284 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="04e2ad9d-c5b9-47b0-ba97-86114775393c" containerName="istio-proxy"
Oct 31 01:29:37.035333 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:37.035118    4284 cpu_manager.go:414] "containerMap: removing container" podUID="6ac9b2b9-a157-4f05-8c7e-e7a2987397f6" containerName="istio-proxy"
Oct 31 01:29:37.035333 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:37.035126    4284 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6ac9b2b9-a157-4f05-8c7e-e7a2987397f6" containerName="istio-proxy"
Oct 31 01:29:37.035333 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:37.035140    4284 cpu_manager.go:414] "containerMap: removing container" podUID="04e2ad9d-c5b9-47b0-ba97-86114775393c" containerName="istio-proxy"
Oct 31 01:29:37.035333 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:37.035147    4284 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="04e2ad9d-c5b9-47b0-ba97-86114775393c" containerName="istio-proxy"
Oct 31 01:29:37.035333 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:37.035160    4284 cpu_manager.go:414] "containerMap: removing container" podUID="1a294983-e4ba-4253-8e51-24481d476dd6" containerName="payload-processing"
Oct 31 01:29:37.035333 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:37.035167    4284 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1a294983-e4ba-4253-8e51-24481d476dd6" containerName="payload-processing"
Oct 31 01:29:37.035333 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:37.035179    4284 cpu_manager.go:414] "containerMap: removing container" podUID="16fc298a-4f7c-4efd-ac33-5ac97b7d3275" containerName="payload-pre-processing"
Oct 31 01:29:37.035333 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:37.035184    4284 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="16fc298a-4f7c-4efd-ac33-5ac97b7d3275" containerName="payload-pre-processing"
Oct 31 01:29:37.035333 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:37.035201    4284 cpu_manager.go:414] "containerMap: removing container" podUID="0e3a22bb-0915-4025-af10-c9c3b01011f1" containerName="payload-pre-processing"
Oct 31 01:29:37.035333 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:37.035223    4284 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0e3a22bb-0915-4025-af10-c9c3b01011f1" containerName="payload-pre-processing"
Oct 31 01:29:37.035333 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:37.035241    4284 cpu_manager.go:414] "containerMap: removing container" podUID="ddda98fe-7ad3-4592-81bb-2b6ce526b882" containerName="payload-processing"
Oct 31 01:29:37.035333 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:37.035249    4284 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ddda98fe-7ad3-4592-81bb-2b6ce526b882" containerName="payload-processing"
Oct 31 01:29:37.035333 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:37.035350    4284 memory_manager.go:365] "RemoveStaleState removing state" podUID="a11c7b2d-901c-4c69-9be2-a0bd6eeaf9c0" containerName="maas-api"
Oct 31 01:29:37.036430 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:37.035361    4284 memory_manager.go:365] "RemoveStaleState removing state" podUID="f763b240-930c-4044-8c59-02cc2bd614d7" containerName="maas-api"
Oct 31 01:29:37.036430 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:37.035368    4284 memory_manager.go:365] "RemoveStaleState removing state" podUID="6aa40f46-c2b1-4c37-a341-d230158a7db9" containerName="istio-proxy"
Oct 31 01:29:37.036430 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:37.035374    4284 memory_manager.go:365] "RemoveStaleState removing state" podUID="6ac9b2b9-a157-4f05-8c7e-e7a2987397f6" containerName="istio-proxy"
Oct 31 01:29:37.036430 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:37.035381    4284 memory_manager.go:365] "RemoveStaleState removing state" podUID="04e2ad9d-c5b9-47b0-ba97-86114775393c" containerName="istio-proxy"
Oct 31 01:29:37.036430 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:37.035386    4284 memory_manager.go:365] "RemoveStaleState removing state" podUID="6ac9b2b9-a157-4f05-8c7e-e7a2987397f6" containerName="istio-proxy"
Oct 31 01:29:37.036430 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:37.035395    4284 memory_manager.go:365] "RemoveStaleState removing state" podUID="1a294983-e4ba-4253-8e51-24481d476dd6" containerName="payload-processing"
Oct 31 01:29:37.036430 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:37.035406    4284 memory_manager.go:365] "RemoveStaleState removing state" podUID="04e2ad9d-c5b9-47b0-ba97-86114775393c" containerName="istio-proxy"
Oct 31 01:29:37.036430 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:37.035414    4284 memory_manager.go:365] "RemoveStaleState removing state" podUID="16fc298a-4f7c-4efd-ac33-5ac97b7d3275" containerName="payload-pre-processing"
Oct 31 01:29:37.036430 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:37.035422    4284 memory_manager.go:365] "RemoveStaleState removing state" podUID="0e3a22bb-0915-4025-af10-c9c3b01011f1" containerName="payload-pre-processing"
Oct 31 01:29:37.036430 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:37.035428    4284 memory_manager.go:365] "RemoveStaleState removing state" podUID="ddda98fe-7ad3-4592-81bb-2b6ce526b882" containerName="payload-processing"
Oct 31 01:29:37.036430 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:37.035433    4284 memory_manager.go:365] "RemoveStaleState removing state" podUID="6aa40f46-c2b1-4c37-a341-d230158a7db9" containerName="istio-proxy"
Oct 31 01:29:37.036430 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:37.035439    4284 memory_manager.go:365] "RemoveStaleState removing state" podUID="f31cff38-07af-4d6c-83ba-fcc15b830cbc" containerName="payload-processing"
Oct 31 01:29:37.036430 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:37.035445    4284 memory_manager.go:365] "RemoveStaleState removing state" podUID="fd46495b-8c51-476d-91f9-750529060bbd" containerName="istio-proxy"
Oct 31 01:29:37.036430 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:37.035450    4284 memory_manager.go:365] "RemoveStaleState removing state" podUID="b96b200a-f8de-4419-8232-061be10ada45" containerName="payload-processing"
Oct 31 01:29:37.036430 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:37.035455    4284 memory_manager.go:365] "RemoveStaleState removing state" podUID="178450d7-7cf3-46f9-8e25-e09b73075316" containerName="maas-api"
Oct 31 01:29:37.036430 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:37.035461    4284 memory_manager.go:365] "RemoveStaleState removing state" podUID="fd46495b-8c51-476d-91f9-750529060bbd" containerName="istio-proxy"
Oct 31 01:29:37.036430 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:37.035466    4284 memory_manager.go:365] "RemoveStaleState removing state" podUID="e38c82a4-98f4-48a6-b888-f95877335bd1" containerName="maas-api"
Oct 31 01:29:37.036430 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:37.035471    4284 memory_manager.go:365] "RemoveStaleState removing state" podUID="40b79195-6d51-4041-9cbb-5acc6f1e7345" containerName="maas-api"
Oct 31 01:29:37.036430 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:37.035476    4284 memory_manager.go:365] "RemoveStaleState removing state" podUID="1dde1271-7b40-4dc0-97ca-c1be6b89c477" containerName="istio-proxy"
Oct 31 01:29:37.036430 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:37.035485    4284 memory_manager.go:365] "RemoveStaleState removing state" podUID="a71dc36e-312b-4dff-b412-056d51f65731" containerName="payload-pre-processing"
Oct 31 01:29:37.036430 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:37.035494    4284 memory_manager.go:365] "RemoveStaleState removing state" podUID="1dde1271-7b40-4dc0-97ca-c1be6b89c477" containerName="istio-proxy"
Oct 31 01:29:37.039799 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:37.039780    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-b-6bd7c8-openshift-default-5b6745476f-7xkcx"
Oct 31 01:29:37.089990 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:37.089964    4284 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-extmod-b-6bd7c8-openshift-default-dockercfg-7n6jz\""
Oct 31 01:29:37.099563 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:37.099533    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eed2968a-68fc-44a9-beb1-669b189009de-workload-certs\") pod \"e2e-extmod-b-6bd7c8-openshift-default-5b6745476f-7xkcx\" (UID: \"eed2968a-68fc-44a9-beb1-669b189009de\") " pod="openshift-ingress/e2e-extmod-b-6bd7c8-openshift-default-5b6745476f-7xkcx"
Oct 31 01:29:37.099872 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:37.099844    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eed2968a-68fc-44a9-beb1-669b189009de-credential-socket\") pod \"e2e-extmod-b-6bd7c8-openshift-default-5b6745476f-7xkcx\" (UID: \"eed2968a-68fc-44a9-beb1-669b189009de\") " pod="openshift-ingress/e2e-extmod-b-6bd7c8-openshift-default-5b6745476f-7xkcx"
Oct 31 01:29:37.099994 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:37.099886    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-674s4\" (UniqueName: \"kubernetes.io/projected/eed2968a-68fc-44a9-beb1-669b189009de-kube-api-access-674s4\") pod \"e2e-extmod-b-6bd7c8-openshift-default-5b6745476f-7xkcx\" (UID: \"eed2968a-68fc-44a9-beb1-669b189009de\") " pod="openshift-ingress/e2e-extmod-b-6bd7c8-openshift-default-5b6745476f-7xkcx"
Oct 31 01:29:37.099994 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:37.099918    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eed2968a-68fc-44a9-beb1-669b189009de-istio-token\") pod \"e2e-extmod-b-6bd7c8-openshift-default-5b6745476f-7xkcx\" (UID: \"eed2968a-68fc-44a9-beb1-669b189009de\") " pod="openshift-ingress/e2e-extmod-b-6bd7c8-openshift-default-5b6745476f-7xkcx"
Oct 31 01:29:37.099994 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:37.099972    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eed2968a-68fc-44a9-beb1-669b189009de-istio-podinfo\") pod \"e2e-extmod-b-6bd7c8-openshift-default-5b6745476f-7xkcx\" (UID: \"eed2968a-68fc-44a9-beb1-669b189009de\") " pod="openshift-ingress/e2e-extmod-b-6bd7c8-openshift-default-5b6745476f-7xkcx"
Oct 31 01:29:37.100530 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:37.100084    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eed2968a-68fc-44a9-beb1-669b189009de-istiod-ca-cert\") pod \"e2e-extmod-b-6bd7c8-openshift-default-5b6745476f-7xkcx\" (UID: \"eed2968a-68fc-44a9-beb1-669b189009de\") " pod="openshift-ingress/e2e-extmod-b-6bd7c8-openshift-default-5b6745476f-7xkcx"
Oct 31 01:29:37.100530 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:37.100141    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eed2968a-68fc-44a9-beb1-669b189009de-istio-data\") pod \"e2e-extmod-b-6bd7c8-openshift-default-5b6745476f-7xkcx\" (UID: \"eed2968a-68fc-44a9-beb1-669b189009de\") " pod="openshift-ingress/e2e-extmod-b-6bd7c8-openshift-default-5b6745476f-7xkcx"
Oct 31 01:29:37.100530 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:37.100273    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eed2968a-68fc-44a9-beb1-669b189009de-workload-socket\") pod \"e2e-extmod-b-6bd7c8-openshift-default-5b6745476f-7xkcx\" (UID: \"eed2968a-68fc-44a9-beb1-669b189009de\") " pod="openshift-ingress/e2e-extmod-b-6bd7c8-openshift-default-5b6745476f-7xkcx"
Oct 31 01:29:37.100530 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:37.100315    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eed2968a-68fc-44a9-beb1-669b189009de-istio-envoy\") pod \"e2e-extmod-b-6bd7c8-openshift-default-5b6745476f-7xkcx\" (UID: \"eed2968a-68fc-44a9-beb1-669b189009de\") " pod="openshift-ingress/e2e-extmod-b-6bd7c8-openshift-default-5b6745476f-7xkcx"
Oct 31 01:29:37.139710 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:37.139679    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-extmod-b-6bd7c8-openshift-default-5b6745476f-7xkcx"]
Oct 31 01:29:37.173107 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:37.173051    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:29:37.201502 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:37.201459    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eed2968a-68fc-44a9-beb1-669b189009de-istio-data\") pod \"e2e-extmod-b-6bd7c8-openshift-default-5b6745476f-7xkcx\" (UID: \"eed2968a-68fc-44a9-beb1-669b189009de\") " pod="openshift-ingress/e2e-extmod-b-6bd7c8-openshift-default-5b6745476f-7xkcx"
Oct 31 01:29:37.201701 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:37.201512    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eed2968a-68fc-44a9-beb1-669b189009de-workload-socket\") pod \"e2e-extmod-b-6bd7c8-openshift-default-5b6745476f-7xkcx\" (UID: \"eed2968a-68fc-44a9-beb1-669b189009de\") " pod="openshift-ingress/e2e-extmod-b-6bd7c8-openshift-default-5b6745476f-7xkcx"
Oct 31 01:29:37.201701 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:37.201539    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eed2968a-68fc-44a9-beb1-669b189009de-istio-envoy\") pod \"e2e-extmod-b-6bd7c8-openshift-default-5b6745476f-7xkcx\" (UID: \"eed2968a-68fc-44a9-beb1-669b189009de\") " pod="openshift-ingress/e2e-extmod-b-6bd7c8-openshift-default-5b6745476f-7xkcx"
Oct 31 01:29:37.201701 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:37.201608    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eed2968a-68fc-44a9-beb1-669b189009de-workload-certs\") pod \"e2e-extmod-b-6bd7c8-openshift-default-5b6745476f-7xkcx\" (UID: \"eed2968a-68fc-44a9-beb1-669b189009de\") " pod="openshift-ingress/e2e-extmod-b-6bd7c8-openshift-default-5b6745476f-7xkcx"
Oct 31 01:29:37.201701 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:37.201653    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eed2968a-68fc-44a9-beb1-669b189009de-credential-socket\") pod \"e2e-extmod-b-6bd7c8-openshift-default-5b6745476f-7xkcx\" (UID: \"eed2968a-68fc-44a9-beb1-669b189009de\") " pod="openshift-ingress/e2e-extmod-b-6bd7c8-openshift-default-5b6745476f-7xkcx"
Oct 31 01:29:37.201701 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:37.201678    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-674s4\" (UniqueName: \"kubernetes.io/projected/eed2968a-68fc-44a9-beb1-669b189009de-kube-api-access-674s4\") pod \"e2e-extmod-b-6bd7c8-openshift-default-5b6745476f-7xkcx\" (UID: \"eed2968a-68fc-44a9-beb1-669b189009de\") " pod="openshift-ingress/e2e-extmod-b-6bd7c8-openshift-default-5b6745476f-7xkcx"
Oct 31 01:29:37.201970 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:37.201709    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eed2968a-68fc-44a9-beb1-669b189009de-istio-token\") pod \"e2e-extmod-b-6bd7c8-openshift-default-5b6745476f-7xkcx\" (UID: \"eed2968a-68fc-44a9-beb1-669b189009de\") " pod="openshift-ingress/e2e-extmod-b-6bd7c8-openshift-default-5b6745476f-7xkcx"
Oct 31 01:29:37.201970 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:37.201736    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eed2968a-68fc-44a9-beb1-669b189009de-istio-podinfo\") pod \"e2e-extmod-b-6bd7c8-openshift-default-5b6745476f-7xkcx\" (UID: \"eed2968a-68fc-44a9-beb1-669b189009de\") " pod="openshift-ingress/e2e-extmod-b-6bd7c8-openshift-default-5b6745476f-7xkcx"
Oct 31 01:29:37.201970 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:37.201785    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eed2968a-68fc-44a9-beb1-669b189009de-istiod-ca-cert\") pod \"e2e-extmod-b-6bd7c8-openshift-default-5b6745476f-7xkcx\" (UID: \"eed2968a-68fc-44a9-beb1-669b189009de\") " pod="openshift-ingress/e2e-extmod-b-6bd7c8-openshift-default-5b6745476f-7xkcx"
Oct 31 01:29:37.201970 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:37.201912    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eed2968a-68fc-44a9-beb1-669b189009de-istio-data\") pod \"e2e-extmod-b-6bd7c8-openshift-default-5b6745476f-7xkcx\" (UID: \"eed2968a-68fc-44a9-beb1-669b189009de\") " pod="openshift-ingress/e2e-extmod-b-6bd7c8-openshift-default-5b6745476f-7xkcx"
Oct 31 01:29:37.201970 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:37.201912    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eed2968a-68fc-44a9-beb1-669b189009de-workload-socket\") pod \"e2e-extmod-b-6bd7c8-openshift-default-5b6745476f-7xkcx\" (UID: \"eed2968a-68fc-44a9-beb1-669b189009de\") " pod="openshift-ingress/e2e-extmod-b-6bd7c8-openshift-default-5b6745476f-7xkcx"
Oct 31 01:29:37.202206 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:37.202001    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eed2968a-68fc-44a9-beb1-669b189009de-workload-certs\") pod \"e2e-extmod-b-6bd7c8-openshift-default-5b6745476f-7xkcx\" (UID: \"eed2968a-68fc-44a9-beb1-669b189009de\") " pod="openshift-ingress/e2e-extmod-b-6bd7c8-openshift-default-5b6745476f-7xkcx"
Oct 31 01:29:37.202206 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:37.202096    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eed2968a-68fc-44a9-beb1-669b189009de-credential-socket\") pod \"e2e-extmod-b-6bd7c8-openshift-default-5b6745476f-7xkcx\" (UID: \"eed2968a-68fc-44a9-beb1-669b189009de\") " pod="openshift-ingress/e2e-extmod-b-6bd7c8-openshift-default-5b6745476f-7xkcx"
Oct 31 01:29:37.202503 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:37.202487    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eed2968a-68fc-44a9-beb1-669b189009de-istiod-ca-cert\") pod \"e2e-extmod-b-6bd7c8-openshift-default-5b6745476f-7xkcx\" (UID: \"eed2968a-68fc-44a9-beb1-669b189009de\") " pod="openshift-ingress/e2e-extmod-b-6bd7c8-openshift-default-5b6745476f-7xkcx"
Oct 31 01:29:37.204464 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:37.204442    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eed2968a-68fc-44a9-beb1-669b189009de-istio-podinfo\") pod \"e2e-extmod-b-6bd7c8-openshift-default-5b6745476f-7xkcx\" (UID: \"eed2968a-68fc-44a9-beb1-669b189009de\") " pod="openshift-ingress/e2e-extmod-b-6bd7c8-openshift-default-5b6745476f-7xkcx"
Oct 31 01:29:37.204583 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:37.204489    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eed2968a-68fc-44a9-beb1-669b189009de-istio-envoy\") pod \"e2e-extmod-b-6bd7c8-openshift-default-5b6745476f-7xkcx\" (UID: \"eed2968a-68fc-44a9-beb1-669b189009de\") " pod="openshift-ingress/e2e-extmod-b-6bd7c8-openshift-default-5b6745476f-7xkcx"
Oct 31 01:29:37.266826 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:37.266714    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-674s4\" (UniqueName: \"kubernetes.io/projected/eed2968a-68fc-44a9-beb1-669b189009de-kube-api-access-674s4\") pod \"e2e-extmod-b-6bd7c8-openshift-default-5b6745476f-7xkcx\" (UID: \"eed2968a-68fc-44a9-beb1-669b189009de\") " pod="openshift-ingress/e2e-extmod-b-6bd7c8-openshift-default-5b6745476f-7xkcx"
Oct 31 01:29:37.268395 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:37.268121    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eed2968a-68fc-44a9-beb1-669b189009de-istio-token\") pod \"e2e-extmod-b-6bd7c8-openshift-default-5b6745476f-7xkcx\" (UID: \"eed2968a-68fc-44a9-beb1-669b189009de\") " pod="openshift-ingress/e2e-extmod-b-6bd7c8-openshift-default-5b6745476f-7xkcx"
Oct 31 01:29:37.296079 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:37.296044    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-f9925a-openshift-default-6d99c6d9c-nfzmj"]
Oct 31 01:29:37.296385 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:37.296337    4284 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-w2-f9925a-openshift-default-6d99c6d9c-nfzmj" podUID="ed93ba84-d871-4811-ade6-5581127ca3dc" containerName="istio-proxy" containerID="cri-o://e229ba4a03c1cf9651c6d521874f8615a689cf88eeeaf9c4dc279abbd70d4673" gracePeriod=30
Oct 31 01:29:37.345687 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:37.345629    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:29:37.352888 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:37.352859    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-b-6bd7c8-openshift-default-5b6745476f-7xkcx"
Oct 31 01:29:37.493535 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:37.493500    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:29:37.810395 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:37.810364    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-extmod-b-6bd7c8-openshift-default-5b6745476f-7xkcx"]
Oct 31 01:29:37.812007 ip-10-0-132-173 kubenswrapper[4284]: W1031 01:29:37.811978    4284 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeed2968a_68fc_44a9_beb1_669b189009de.slice/crio-683a0b6ca2d78692032147204521d1f63a780caff4e25859ce6e67101ed8686c WatchSource:0}: Error finding container 683a0b6ca2d78692032147204521d1f63a780caff4e25859ce6e67101ed8686c: Status 404 returned error can't find the container with id 683a0b6ca2d78692032147204521d1f63a780caff4e25859ce6e67101ed8686c
Oct 31 01:29:37.813087 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:37.813065    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:29:37.814665 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:37.814637    4284 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 01:29:37.814746 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:37.814697    4284 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 01:29:37.814746 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:37.814725    4284 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 01:29:38.633824 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:38.633785    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-b-6bd7c8-openshift-default-5b6745476f-7xkcx" event={"ID":"eed2968a-68fc-44a9-beb1-669b189009de","Type":"ContainerStarted","Data":"b7868070546d71cabda47b5a4b65ad5a91663b9d0c698b49d9a5f99301ba3488"}
Oct 31 01:29:38.634239 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:38.633845    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-b-6bd7c8-openshift-default-5b6745476f-7xkcx" event={"ID":"eed2968a-68fc-44a9-beb1-669b189009de","Type":"ContainerStarted","Data":"683a0b6ca2d78692032147204521d1f63a780caff4e25859ce6e67101ed8686c"}
Oct 31 01:29:38.798741 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:38.798675    4284 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-extmod-b-6bd7c8-openshift-default-5b6745476f-7xkcx" podStartSLOduration=2.798652901 podStartE2EDuration="2.798652901s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 01:29:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 01:29:38.797091789 +0000 UTC m=+3692.942098709" watchObservedRunningTime="2026-10-31 01:29:38.798652901 +0000 UTC m=+3692.943659824"
Oct 31 01:29:38.883684 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:38.883651    4284 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-extmod-a-c61da9-openshift-default-76f5864d45-7gh6h"]
Oct 31 01:29:38.884248 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:38.884171    4284 cpu_manager.go:414] "containerMap: removing container" podUID="6aa40f46-c2b1-4c37-a341-d230158a7db9" containerName="istio-proxy"
Oct 31 01:29:38.884248 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:38.884189    4284 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6aa40f46-c2b1-4c37-a341-d230158a7db9" containerName="istio-proxy"
Oct 31 01:29:38.902022 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:38.901987    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-a-c61da9-openshift-default-76f5864d45-7gh6h"
Oct 31 01:29:38.948246 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:38.948199    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-extmod-a-c61da9-openshift-default-76f5864d45-7gh6h"]
Oct 31 01:29:38.973205 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:38.973161    4284 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-extmod-a-c61da9-openshift-default-dockercfg-56sw9\""
Oct 31 01:29:39.030900 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:39.030865    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/164bb74b-a95d-4b77-97b6-f2ca2eb2cc09-workload-socket\") pod \"e2e-extmod-a-c61da9-openshift-default-76f5864d45-7gh6h\" (UID: \"164bb74b-a95d-4b77-97b6-f2ca2eb2cc09\") " pod="openshift-ingress/e2e-extmod-a-c61da9-openshift-default-76f5864d45-7gh6h"
Oct 31 01:29:39.031140 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:39.030928    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/164bb74b-a95d-4b77-97b6-f2ca2eb2cc09-istio-token\") pod \"e2e-extmod-a-c61da9-openshift-default-76f5864d45-7gh6h\" (UID: \"164bb74b-a95d-4b77-97b6-f2ca2eb2cc09\") " pod="openshift-ingress/e2e-extmod-a-c61da9-openshift-default-76f5864d45-7gh6h"
Oct 31 01:29:39.031140 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:39.031030    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/164bb74b-a95d-4b77-97b6-f2ca2eb2cc09-workload-certs\") pod \"e2e-extmod-a-c61da9-openshift-default-76f5864d45-7gh6h\" (UID: \"164bb74b-a95d-4b77-97b6-f2ca2eb2cc09\") " pod="openshift-ingress/e2e-extmod-a-c61da9-openshift-default-76f5864d45-7gh6h"
Oct 31 01:29:39.031140 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:39.031059    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cdk2p\" (UniqueName: \"kubernetes.io/projected/164bb74b-a95d-4b77-97b6-f2ca2eb2cc09-kube-api-access-cdk2p\") pod \"e2e-extmod-a-c61da9-openshift-default-76f5864d45-7gh6h\" (UID: \"164bb74b-a95d-4b77-97b6-f2ca2eb2cc09\") " pod="openshift-ingress/e2e-extmod-a-c61da9-openshift-default-76f5864d45-7gh6h"
Oct 31 01:29:39.031140 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:39.031076    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/164bb74b-a95d-4b77-97b6-f2ca2eb2cc09-istio-data\") pod \"e2e-extmod-a-c61da9-openshift-default-76f5864d45-7gh6h\" (UID: \"164bb74b-a95d-4b77-97b6-f2ca2eb2cc09\") " pod="openshift-ingress/e2e-extmod-a-c61da9-openshift-default-76f5864d45-7gh6h"
Oct 31 01:29:39.031140 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:39.031093    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/164bb74b-a95d-4b77-97b6-f2ca2eb2cc09-credential-socket\") pod \"e2e-extmod-a-c61da9-openshift-default-76f5864d45-7gh6h\" (UID: \"164bb74b-a95d-4b77-97b6-f2ca2eb2cc09\") " pod="openshift-ingress/e2e-extmod-a-c61da9-openshift-default-76f5864d45-7gh6h"
Oct 31 01:29:39.031439 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:39.031196    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/164bb74b-a95d-4b77-97b6-f2ca2eb2cc09-istiod-ca-cert\") pod \"e2e-extmod-a-c61da9-openshift-default-76f5864d45-7gh6h\" (UID: \"164bb74b-a95d-4b77-97b6-f2ca2eb2cc09\") " pod="openshift-ingress/e2e-extmod-a-c61da9-openshift-default-76f5864d45-7gh6h"
Oct 31 01:29:39.031439 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:39.031317    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/164bb74b-a95d-4b77-97b6-f2ca2eb2cc09-istio-podinfo\") pod \"e2e-extmod-a-c61da9-openshift-default-76f5864d45-7gh6h\" (UID: \"164bb74b-a95d-4b77-97b6-f2ca2eb2cc09\") " pod="openshift-ingress/e2e-extmod-a-c61da9-openshift-default-76f5864d45-7gh6h"
Oct 31 01:29:39.031439 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:39.031370    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/164bb74b-a95d-4b77-97b6-f2ca2eb2cc09-istio-envoy\") pod \"e2e-extmod-a-c61da9-openshift-default-76f5864d45-7gh6h\" (UID: \"164bb74b-a95d-4b77-97b6-f2ca2eb2cc09\") " pod="openshift-ingress/e2e-extmod-a-c61da9-openshift-default-76f5864d45-7gh6h"
Oct 31 01:29:39.132712 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:39.132679    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/164bb74b-a95d-4b77-97b6-f2ca2eb2cc09-istio-podinfo\") pod \"e2e-extmod-a-c61da9-openshift-default-76f5864d45-7gh6h\" (UID: \"164bb74b-a95d-4b77-97b6-f2ca2eb2cc09\") " pod="openshift-ingress/e2e-extmod-a-c61da9-openshift-default-76f5864d45-7gh6h"
Oct 31 01:29:39.132916 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:39.132732    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/164bb74b-a95d-4b77-97b6-f2ca2eb2cc09-istio-envoy\") pod \"e2e-extmod-a-c61da9-openshift-default-76f5864d45-7gh6h\" (UID: \"164bb74b-a95d-4b77-97b6-f2ca2eb2cc09\") " pod="openshift-ingress/e2e-extmod-a-c61da9-openshift-default-76f5864d45-7gh6h"
Oct 31 01:29:39.132916 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:39.132789    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/164bb74b-a95d-4b77-97b6-f2ca2eb2cc09-workload-socket\") pod \"e2e-extmod-a-c61da9-openshift-default-76f5864d45-7gh6h\" (UID: \"164bb74b-a95d-4b77-97b6-f2ca2eb2cc09\") " pod="openshift-ingress/e2e-extmod-a-c61da9-openshift-default-76f5864d45-7gh6h"
Oct 31 01:29:39.132916 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:39.132845    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/164bb74b-a95d-4b77-97b6-f2ca2eb2cc09-istio-token\") pod \"e2e-extmod-a-c61da9-openshift-default-76f5864d45-7gh6h\" (UID: \"164bb74b-a95d-4b77-97b6-f2ca2eb2cc09\") " pod="openshift-ingress/e2e-extmod-a-c61da9-openshift-default-76f5864d45-7gh6h"
Oct 31 01:29:39.134308 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:39.133152    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/164bb74b-a95d-4b77-97b6-f2ca2eb2cc09-workload-certs\") pod \"e2e-extmod-a-c61da9-openshift-default-76f5864d45-7gh6h\" (UID: \"164bb74b-a95d-4b77-97b6-f2ca2eb2cc09\") " pod="openshift-ingress/e2e-extmod-a-c61da9-openshift-default-76f5864d45-7gh6h"
Oct 31 01:29:39.134308 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:39.133192    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cdk2p\" (UniqueName: \"kubernetes.io/projected/164bb74b-a95d-4b77-97b6-f2ca2eb2cc09-kube-api-access-cdk2p\") pod \"e2e-extmod-a-c61da9-openshift-default-76f5864d45-7gh6h\" (UID: \"164bb74b-a95d-4b77-97b6-f2ca2eb2cc09\") " pod="openshift-ingress/e2e-extmod-a-c61da9-openshift-default-76f5864d45-7gh6h"
Oct 31 01:29:39.134308 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:39.133252    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/164bb74b-a95d-4b77-97b6-f2ca2eb2cc09-istio-data\") pod \"e2e-extmod-a-c61da9-openshift-default-76f5864d45-7gh6h\" (UID: \"164bb74b-a95d-4b77-97b6-f2ca2eb2cc09\") " pod="openshift-ingress/e2e-extmod-a-c61da9-openshift-default-76f5864d45-7gh6h"
Oct 31 01:29:39.134308 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:39.133278    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/164bb74b-a95d-4b77-97b6-f2ca2eb2cc09-workload-socket\") pod \"e2e-extmod-a-c61da9-openshift-default-76f5864d45-7gh6h\" (UID: \"164bb74b-a95d-4b77-97b6-f2ca2eb2cc09\") " pod="openshift-ingress/e2e-extmod-a-c61da9-openshift-default-76f5864d45-7gh6h"
Oct 31 01:29:39.134308 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:39.133280    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/164bb74b-a95d-4b77-97b6-f2ca2eb2cc09-credential-socket\") pod \"e2e-extmod-a-c61da9-openshift-default-76f5864d45-7gh6h\" (UID: \"164bb74b-a95d-4b77-97b6-f2ca2eb2cc09\") " pod="openshift-ingress/e2e-extmod-a-c61da9-openshift-default-76f5864d45-7gh6h"
Oct 31 01:29:39.134308 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:39.133380    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/164bb74b-a95d-4b77-97b6-f2ca2eb2cc09-istiod-ca-cert\") pod \"e2e-extmod-a-c61da9-openshift-default-76f5864d45-7gh6h\" (UID: \"164bb74b-a95d-4b77-97b6-f2ca2eb2cc09\") " pod="openshift-ingress/e2e-extmod-a-c61da9-openshift-default-76f5864d45-7gh6h"
Oct 31 01:29:39.134308 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:39.133504    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/164bb74b-a95d-4b77-97b6-f2ca2eb2cc09-workload-certs\") pod \"e2e-extmod-a-c61da9-openshift-default-76f5864d45-7gh6h\" (UID: \"164bb74b-a95d-4b77-97b6-f2ca2eb2cc09\") " pod="openshift-ingress/e2e-extmod-a-c61da9-openshift-default-76f5864d45-7gh6h"
Oct 31 01:29:39.134308 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:39.133779    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/164bb74b-a95d-4b77-97b6-f2ca2eb2cc09-istio-data\") pod \"e2e-extmod-a-c61da9-openshift-default-76f5864d45-7gh6h\" (UID: \"164bb74b-a95d-4b77-97b6-f2ca2eb2cc09\") " pod="openshift-ingress/e2e-extmod-a-c61da9-openshift-default-76f5864d45-7gh6h"
Oct 31 01:29:39.134308 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:39.134032    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/164bb74b-a95d-4b77-97b6-f2ca2eb2cc09-istiod-ca-cert\") pod \"e2e-extmod-a-c61da9-openshift-default-76f5864d45-7gh6h\" (UID: \"164bb74b-a95d-4b77-97b6-f2ca2eb2cc09\") " pod="openshift-ingress/e2e-extmod-a-c61da9-openshift-default-76f5864d45-7gh6h"
Oct 31 01:29:39.134308 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:39.134124    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/164bb74b-a95d-4b77-97b6-f2ca2eb2cc09-credential-socket\") pod \"e2e-extmod-a-c61da9-openshift-default-76f5864d45-7gh6h\" (UID: \"164bb74b-a95d-4b77-97b6-f2ca2eb2cc09\") " pod="openshift-ingress/e2e-extmod-a-c61da9-openshift-default-76f5864d45-7gh6h"
Oct 31 01:29:39.135881 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:39.135858    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/164bb74b-a95d-4b77-97b6-f2ca2eb2cc09-istio-podinfo\") pod \"e2e-extmod-a-c61da9-openshift-default-76f5864d45-7gh6h\" (UID: \"164bb74b-a95d-4b77-97b6-f2ca2eb2cc09\") " pod="openshift-ingress/e2e-extmod-a-c61da9-openshift-default-76f5864d45-7gh6h"
Oct 31 01:29:39.135958 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:39.135913    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/164bb74b-a95d-4b77-97b6-f2ca2eb2cc09-istio-envoy\") pod \"e2e-extmod-a-c61da9-openshift-default-76f5864d45-7gh6h\" (UID: \"164bb74b-a95d-4b77-97b6-f2ca2eb2cc09\") " pod="openshift-ingress/e2e-extmod-a-c61da9-openshift-default-76f5864d45-7gh6h"
Oct 31 01:29:39.191821 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:39.191794    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/164bb74b-a95d-4b77-97b6-f2ca2eb2cc09-istio-token\") pod \"e2e-extmod-a-c61da9-openshift-default-76f5864d45-7gh6h\" (UID: \"164bb74b-a95d-4b77-97b6-f2ca2eb2cc09\") " pod="openshift-ingress/e2e-extmod-a-c61da9-openshift-default-76f5864d45-7gh6h"
Oct 31 01:29:39.192027 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:39.192001    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cdk2p\" (UniqueName: \"kubernetes.io/projected/164bb74b-a95d-4b77-97b6-f2ca2eb2cc09-kube-api-access-cdk2p\") pod \"e2e-extmod-a-c61da9-openshift-default-76f5864d45-7gh6h\" (UID: \"164bb74b-a95d-4b77-97b6-f2ca2eb2cc09\") " pod="openshift-ingress/e2e-extmod-a-c61da9-openshift-default-76f5864d45-7gh6h"
Oct 31 01:29:39.231898 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:39.231863    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-a-c61da9-openshift-default-76f5864d45-7gh6h"
Oct 31 01:29:39.354043 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:39.354013    4284 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-extmod-b-6bd7c8-openshift-default-5b6745476f-7xkcx"
Oct 31 01:29:39.359809 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:39.359778    4284 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-extmod-b-6bd7c8-openshift-default-5b6745476f-7xkcx"
Oct 31 01:29:39.639612 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:39.639581    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-extmod-b-6bd7c8-openshift-default-5b6745476f-7xkcx"
Oct 31 01:29:39.640523 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:39.640504    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-extmod-b-6bd7c8-openshift-default-5b6745476f-7xkcx"
Oct 31 01:29:39.651559 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:39.651531    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-extmod-a-c61da9-openshift-default-76f5864d45-7gh6h"]
Oct 31 01:29:39.653142 ip-10-0-132-173 kubenswrapper[4284]: W1031 01:29:39.653118    4284 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod164bb74b_a95d_4b77_97b6_f2ca2eb2cc09.slice/crio-6c1d6853cdc348aafddac529f46c3d140849d16717917bd83a5229dacbdea95b WatchSource:0}: Error finding container 6c1d6853cdc348aafddac529f46c3d140849d16717917bd83a5229dacbdea95b: Status 404 returned error can't find the container with id 6c1d6853cdc348aafddac529f46c3d140849d16717917bd83a5229dacbdea95b
Oct 31 01:29:39.656055 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:39.656024    4284 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 01:29:39.656122 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:39.656090    4284 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 01:29:39.656122 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:39.656117    4284 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 01:29:40.649593 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:40.649555    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-c61da9-openshift-default-76f5864d45-7gh6h" event={"ID":"164bb74b-a95d-4b77-97b6-f2ca2eb2cc09","Type":"ContainerStarted","Data":"c26f97643d9e04d41f19ba84e12bf4a369dd9a0b62d12c705f68f2decc1301d1"}
Oct 31 01:29:40.650094 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:40.649615    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-c61da9-openshift-default-76f5864d45-7gh6h" event={"ID":"164bb74b-a95d-4b77-97b6-f2ca2eb2cc09","Type":"ContainerStarted","Data":"6c1d6853cdc348aafddac529f46c3d140849d16717917bd83a5229dacbdea95b"}
Oct 31 01:29:40.768244 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:40.768162    4284 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-extmod-a-c61da9-openshift-default-76f5864d45-7gh6h" podStartSLOduration=2.768146754 podStartE2EDuration="2.768146754s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 01:29:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 01:29:40.765064678 +0000 UTC m=+3694.910071595" watchObservedRunningTime="2026-10-31 01:29:40.768146754 +0000 UTC m=+3694.913153669"
Oct 31 01:29:40.839304 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:40.839268    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:29:41.232907 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:41.232871    4284 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-extmod-a-c61da9-openshift-default-76f5864d45-7gh6h"
Oct 31 01:29:41.234527 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:41.234498    4284 patch_prober.go:29] interesting pod/e2e-extmod-a-c61da9-openshift-default-76f5864d45-7gh6h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.190:15021/healthz/ready\": dial tcp 10.134.0.190:15021: connect: connection refused" start-of-body=
Oct 31 01:29:41.234667 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:41.234546    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-c61da9-openshift-default-76f5864d45-7gh6h" podUID="164bb74b-a95d-4b77-97b6-f2ca2eb2cc09" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.190:15021/healthz/ready\": dial tcp 10.134.0.190:15021: connect: connection refused"
Oct 31 01:29:41.635173 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:41.635094    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:29:41.809944 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:41.809908    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:29:41.951719 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:41.951674    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:29:42.174972 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:42.174924    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w1-b63c24-75cd4d9679-cpvtw"]
Oct 31 01:29:42.232842 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:42.232748    4284 patch_prober.go:29] interesting pod/e2e-extmod-a-c61da9-openshift-default-76f5864d45-7gh6h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.190:15021/healthz/ready\": dial tcp 10.134.0.190:15021: connect: connection refused" start-of-body=
Oct 31 01:29:42.232842 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:42.232834    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-c61da9-openshift-default-76f5864d45-7gh6h" podUID="164bb74b-a95d-4b77-97b6-f2ca2eb2cc09" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.190:15021/healthz/ready\": dial tcp 10.134.0.190:15021: connect: connection refused"
Oct 31 01:29:42.410238 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:42.407854    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:29:42.554182 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:42.554084    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:29:42.662428 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:42.662399    4284 generic.go:356] "Generic (PLEG): container finished" podID="ed93ba84-d871-4811-ade6-5581127ca3dc" containerID="e229ba4a03c1cf9651c6d521874f8615a689cf88eeeaf9c4dc279abbd70d4673" exitCode=0
Oct 31 01:29:42.662588 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:42.662471    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-f9925a-openshift-default-6d99c6d9c-nfzmj" event={"ID":"ed93ba84-d871-4811-ade6-5581127ca3dc","Type":"ContainerDied","Data":"e229ba4a03c1cf9651c6d521874f8615a689cf88eeeaf9c4dc279abbd70d4673"}
Oct 31 01:29:42.662588 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:42.662530    4284 scope.go:122] "RemoveContainer" containerID="1e11a386dd30b7efa3351e22ec346f09bfc78fcefd6e2657c82f6c7e3a9d8dff"
Oct 31 01:29:42.966493 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:42.966404    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:29:43.047004 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:43.046978    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-f9925a-openshift-default-6d99c6d9c-nfzmj"
Oct 31 01:29:43.161847 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:43.161813    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:29:43.179659 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:43.179631    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ed93ba84-d871-4811-ade6-5581127ca3dc-kube-api-access-cgll2\" (UniqueName: \"kubernetes.io/projected/ed93ba84-d871-4811-ade6-5581127ca3dc-kube-api-access-cgll2\") pod \"ed93ba84-d871-4811-ade6-5581127ca3dc\" (UID: \"ed93ba84-d871-4811-ade6-5581127ca3dc\") "
Oct 31 01:29:43.179859 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:43.179691    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ed93ba84-d871-4811-ade6-5581127ca3dc-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ed93ba84-d871-4811-ade6-5581127ca3dc-istio-data\") pod \"ed93ba84-d871-4811-ade6-5581127ca3dc\" (UID: \"ed93ba84-d871-4811-ade6-5581127ca3dc\") "
Oct 31 01:29:43.179859 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:43.179732    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/ed93ba84-d871-4811-ade6-5581127ca3dc-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ed93ba84-d871-4811-ade6-5581127ca3dc-istio-podinfo\") pod \"ed93ba84-d871-4811-ade6-5581127ca3dc\" (UID: \"ed93ba84-d871-4811-ade6-5581127ca3dc\") "
Oct 31 01:29:43.179859 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:43.179777    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ed93ba84-d871-4811-ade6-5581127ca3dc-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ed93ba84-d871-4811-ade6-5581127ca3dc-credential-socket\") pod \"ed93ba84-d871-4811-ade6-5581127ca3dc\" (UID: \"ed93ba84-d871-4811-ade6-5581127ca3dc\") "
Oct 31 01:29:43.179859 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:43.179841    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ed93ba84-d871-4811-ade6-5581127ca3dc-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ed93ba84-d871-4811-ade6-5581127ca3dc-workload-certs\") pod \"ed93ba84-d871-4811-ade6-5581127ca3dc\" (UID: \"ed93ba84-d871-4811-ade6-5581127ca3dc\") "
Oct 31 01:29:43.180094 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:43.179882    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ed93ba84-d871-4811-ade6-5581127ca3dc-istio-token\" (UniqueName: \"kubernetes.io/projected/ed93ba84-d871-4811-ade6-5581127ca3dc-istio-token\") pod \"ed93ba84-d871-4811-ade6-5581127ca3dc\" (UID: \"ed93ba84-d871-4811-ade6-5581127ca3dc\") "
Oct 31 01:29:43.180094 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:43.179937    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ed93ba84-d871-4811-ade6-5581127ca3dc-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ed93ba84-d871-4811-ade6-5581127ca3dc-istiod-ca-cert\") pod \"ed93ba84-d871-4811-ade6-5581127ca3dc\" (UID: \"ed93ba84-d871-4811-ade6-5581127ca3dc\") "
Oct 31 01:29:43.180094 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:43.180075    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ed93ba84-d871-4811-ade6-5581127ca3dc-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ed93ba84-d871-4811-ade6-5581127ca3dc-istio-envoy\") pod \"ed93ba84-d871-4811-ade6-5581127ca3dc\" (UID: \"ed93ba84-d871-4811-ade6-5581127ca3dc\") "
Oct 31 01:29:43.180299 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:43.180129    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ed93ba84-d871-4811-ade6-5581127ca3dc-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ed93ba84-d871-4811-ade6-5581127ca3dc-workload-socket\") pod \"ed93ba84-d871-4811-ade6-5581127ca3dc\" (UID: \"ed93ba84-d871-4811-ade6-5581127ca3dc\") "
Oct 31 01:29:43.180360 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:43.180335    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ed93ba84-d871-4811-ade6-5581127ca3dc-workload-certs" pod "ed93ba84-d871-4811-ade6-5581127ca3dc" (UID: "ed93ba84-d871-4811-ade6-5581127ca3dc"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:29:43.181171 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:43.180563    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ed93ba84-d871-4811-ade6-5581127ca3dc-workload-socket" pod "ed93ba84-d871-4811-ade6-5581127ca3dc" (UID: "ed93ba84-d871-4811-ade6-5581127ca3dc"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:29:43.181171 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:43.180632    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ed93ba84-d871-4811-ade6-5581127ca3dc-istio-data" pod "ed93ba84-d871-4811-ade6-5581127ca3dc" (UID: "ed93ba84-d871-4811-ade6-5581127ca3dc"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:29:43.181171 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:43.180697    4284 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ed93ba84-d871-4811-ade6-5581127ca3dc-workload-socket\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:29:43.181171 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:43.180715    4284 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ed93ba84-d871-4811-ade6-5581127ca3dc-workload-certs\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:29:43.181171 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:43.180926    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ed93ba84-d871-4811-ade6-5581127ca3dc-istiod-ca-cert" pod "ed93ba84-d871-4811-ade6-5581127ca3dc" (UID: "ed93ba84-d871-4811-ade6-5581127ca3dc"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 01:29:43.181171 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:43.180953    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ed93ba84-d871-4811-ade6-5581127ca3dc-credential-socket" pod "ed93ba84-d871-4811-ade6-5581127ca3dc" (UID: "ed93ba84-d871-4811-ade6-5581127ca3dc"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:29:43.183190 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:43.183116    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ed93ba84-d871-4811-ade6-5581127ca3dc-istio-podinfo" pod "ed93ba84-d871-4811-ade6-5581127ca3dc" (UID: "ed93ba84-d871-4811-ade6-5581127ca3dc"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 01:29:43.183190 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:43.183154    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ed93ba84-d871-4811-ade6-5581127ca3dc-kube-api-access-cgll2" pod "ed93ba84-d871-4811-ade6-5581127ca3dc" (UID: "ed93ba84-d871-4811-ade6-5581127ca3dc"). InnerVolumeSpecName "kube-api-access-cgll2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:29:43.183954 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:43.183929    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ed93ba84-d871-4811-ade6-5581127ca3dc-istio-token" pod "ed93ba84-d871-4811-ade6-5581127ca3dc" (UID: "ed93ba84-d871-4811-ade6-5581127ca3dc"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:29:43.197418 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:43.196031    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ed93ba84-d871-4811-ade6-5581127ca3dc-istio-envoy" pod "ed93ba84-d871-4811-ade6-5581127ca3dc" (UID: "ed93ba84-d871-4811-ade6-5581127ca3dc"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:29:43.240237 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:43.232958    4284 patch_prober.go:29] interesting pod/e2e-extmod-a-c61da9-openshift-default-76f5864d45-7gh6h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.190:15021/healthz/ready\": dial tcp 10.134.0.190:15021: connect: connection refused" start-of-body=
Oct 31 01:29:43.240237 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:43.233017    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-c61da9-openshift-default-76f5864d45-7gh6h" podUID="164bb74b-a95d-4b77-97b6-f2ca2eb2cc09" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.190:15021/healthz/ready\": dial tcp 10.134.0.190:15021: connect: connection refused"
Oct 31 01:29:43.281896 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:43.281862    4284 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ed93ba84-d871-4811-ade6-5581127ca3dc-istio-envoy\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:29:43.281896 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:43.281897    4284 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cgll2\" (UniqueName: \"kubernetes.io/projected/ed93ba84-d871-4811-ade6-5581127ca3dc-kube-api-access-cgll2\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:29:43.282081 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:43.281908    4284 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ed93ba84-d871-4811-ade6-5581127ca3dc-istio-data\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:29:43.282081 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:43.281916    4284 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ed93ba84-d871-4811-ade6-5581127ca3dc-istio-podinfo\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:29:43.282081 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:43.281926    4284 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ed93ba84-d871-4811-ade6-5581127ca3dc-credential-socket\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:29:43.282081 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:43.281934    4284 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ed93ba84-d871-4811-ade6-5581127ca3dc-istio-token\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:29:43.282081 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:43.281942    4284 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ed93ba84-d871-4811-ade6-5581127ca3dc-istiod-ca-cert\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:29:43.311418 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:43.311382    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:29:43.481430 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:43.481354    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w4-8fb7f0-openshift-default-77b6794b69-bqwdf"]
Oct 31 01:29:43.481822 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:43.481793    4284 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-w4-8fb7f0-openshift-default-77b6794b69-bqwdf" podUID="8af02a59-8229-4932-a556-bfd6ea4e99e0" containerName="istio-proxy" containerID="cri-o://7cb5a2ecaaf5044a6ed8838a063816ef0b8555c12022651fbb86fe5ff0319e4e" gracePeriod=30
Oct 31 01:29:43.531997 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:43.531960    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:29:43.668867 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:43.668831    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-f9925a-openshift-default-6d99c6d9c-nfzmj" event={"ID":"ed93ba84-d871-4811-ade6-5581127ca3dc","Type":"ContainerDied","Data":"00c9f69c4c0600dd2f33044f13114e507d5fea62cc6e7c9a5f98ce401ab57ef5"}
Oct 31 01:29:43.669050 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:43.668887    4284 scope.go:122] "RemoveContainer" containerID="e229ba4a03c1cf9651c6d521874f8615a689cf88eeeaf9c4dc279abbd70d4673"
Oct 31 01:29:43.669050 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:43.668838    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-f9925a-openshift-default-6d99c6d9c-nfzmj"
Oct 31 01:29:43.836179 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:43.836096    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-f9925a-openshift-default-6d99c6d9c-nfzmj"]
Oct 31 01:29:43.842625 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:43.842599    4284 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-f9925a-openshift-default-6d99c6d9c-nfzmj"]
Oct 31 01:29:43.962665 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:43.962632    4284 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ed93ba84-d871-4811-ade6-5581127ca3dc" path="/var/lib/kubelet/pods/ed93ba84-d871-4811-ade6-5581127ca3dc/volumes"
Oct 31 01:29:44.232452 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:44.232414    4284 patch_prober.go:29] interesting pod/e2e-extmod-a-c61da9-openshift-default-76f5864d45-7gh6h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.190:15021/healthz/ready\": dial tcp 10.134.0.190:15021: connect: connection refused" start-of-body=
Oct 31 01:29:44.232839 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:44.232486    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-c61da9-openshift-default-76f5864d45-7gh6h" podUID="164bb74b-a95d-4b77-97b6-f2ca2eb2cc09" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.190:15021/healthz/ready\": dial tcp 10.134.0.190:15021: connect: connection refused"
Oct 31 01:29:44.461323 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:44.461293    4284 patch_prober.go:29] interesting pod/e2e-shared-a-w4-8fb7f0-openshift-default-77b6794b69-bqwdf container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 31 01:29:44.461486 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:44.461370    4284 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-w4-8fb7f0-openshift-default-77b6794b69-bqwdf" podUID="8af02a59-8229-4932-a556-bfd6ea4e99e0" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 31 01:29:45.232319 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:45.232231    4284 patch_prober.go:29] interesting pod/e2e-extmod-a-c61da9-openshift-default-76f5864d45-7gh6h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.190:15021/healthz/ready\": dial tcp 10.134.0.190:15021: connect: connection refused" start-of-body=
Oct 31 01:29:45.232319 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:45.232290    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-c61da9-openshift-default-76f5864d45-7gh6h" podUID="164bb74b-a95d-4b77-97b6-f2ca2eb2cc09" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.190:15021/healthz/ready\": dial tcp 10.134.0.190:15021: connect: connection refused"
Oct 31 01:29:46.109239 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:46.108777    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:29:46.235695 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:46.235657    4284 patch_prober.go:29] interesting pod/e2e-extmod-a-c61da9-openshift-default-76f5864d45-7gh6h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.190:15021/healthz/ready\": dial tcp 10.134.0.190:15021: connect: connection refused" start-of-body=
Oct 31 01:29:46.236311 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:46.236280    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-c61da9-openshift-default-76f5864d45-7gh6h" podUID="164bb74b-a95d-4b77-97b6-f2ca2eb2cc09" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.190:15021/healthz/ready\": dial tcp 10.134.0.190:15021: connect: connection refused"
Oct 31 01:29:46.299540 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:46.299371    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:29:46.486744 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:46.486708    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:29:47.015341 ip-10-0-132-173 kubenswrapper[4284]: E1031 01:29:47.015308    4284 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fd8sp" containerName="extractor"
Oct 31 01:29:47.175571 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:47.175529    4284 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-b63c24-75cd4d9679-cpvtw" podUID="fea709fd-6209-4b40-95f6-ea5104c1cb09" containerName="payload-pre-processing" containerID="cri-o://8bfc236166f01984405d4ec5670d53d05a5e6080fce6955bb83541e25c862230" gracePeriod=25
Oct 31 01:29:47.236552 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:47.236521    4284 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-extmod-a-c61da9-openshift-default-76f5864d45-7gh6h"
Oct 31 01:29:47.237053 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:47.236793    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-extmod-a-c61da9-openshift-default-76f5864d45-7gh6h"
Oct 31 01:29:47.237437 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:47.237416    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-extmod-a-c61da9-openshift-default-76f5864d45-7gh6h"
Oct 31 01:29:48.677534 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:48.677507    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w4-8fb7f0-openshift-default-77b6794b69-bqwdf"
Oct 31 01:29:48.697804 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:48.697775    4284 generic.go:356] "Generic (PLEG): container finished" podID="8af02a59-8229-4932-a556-bfd6ea4e99e0" containerID="7cb5a2ecaaf5044a6ed8838a063816ef0b8555c12022651fbb86fe5ff0319e4e" exitCode=0
Oct 31 01:29:48.697968 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:48.697839    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w4-8fb7f0-openshift-default-77b6794b69-bqwdf"
Oct 31 01:29:48.697968 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:48.697871    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w4-8fb7f0-openshift-default-77b6794b69-bqwdf" event={"ID":"8af02a59-8229-4932-a556-bfd6ea4e99e0","Type":"ContainerDied","Data":"7cb5a2ecaaf5044a6ed8838a063816ef0b8555c12022651fbb86fe5ff0319e4e"}
Oct 31 01:29:48.697968 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:48.697927    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w4-8fb7f0-openshift-default-77b6794b69-bqwdf" event={"ID":"8af02a59-8229-4932-a556-bfd6ea4e99e0","Type":"ContainerDied","Data":"4c3c32900870b927cc1656503b09f5daac4ad5a7fc4f3c76226d5fc4526ceaae"}
Oct 31 01:29:48.697968 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:48.697951    4284 scope.go:122] "RemoveContainer" containerID="7cb5a2ecaaf5044a6ed8838a063816ef0b8555c12022651fbb86fe5ff0319e4e"
Oct 31 01:29:48.707430 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:48.707395    4284 scope.go:122] "RemoveContainer" containerID="f7aad86c2c59b80bdf1c3ebaeb3464cc25b23ac0e09f031cbae0f32e5a09549a"
Oct 31 01:29:48.716409 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:48.716388    4284 scope.go:122] "RemoveContainer" containerID="7cb5a2ecaaf5044a6ed8838a063816ef0b8555c12022651fbb86fe5ff0319e4e"
Oct 31 01:29:48.716703 ip-10-0-132-173 kubenswrapper[4284]: E1031 01:29:48.716683    4284 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7cb5a2ecaaf5044a6ed8838a063816ef0b8555c12022651fbb86fe5ff0319e4e\": container with ID starting with 7cb5a2ecaaf5044a6ed8838a063816ef0b8555c12022651fbb86fe5ff0319e4e not found: ID does not exist" containerID="7cb5a2ecaaf5044a6ed8838a063816ef0b8555c12022651fbb86fe5ff0319e4e"
Oct 31 01:29:48.716788 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:48.716716    4284 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7cb5a2ecaaf5044a6ed8838a063816ef0b8555c12022651fbb86fe5ff0319e4e"} err="failed to get container status \"7cb5a2ecaaf5044a6ed8838a063816ef0b8555c12022651fbb86fe5ff0319e4e\": rpc error: code = NotFound desc = could not find container \"7cb5a2ecaaf5044a6ed8838a063816ef0b8555c12022651fbb86fe5ff0319e4e\": container with ID starting with 7cb5a2ecaaf5044a6ed8838a063816ef0b8555c12022651fbb86fe5ff0319e4e not found: ID does not exist"
Oct 31 01:29:48.716788 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:48.716743    4284 scope.go:122] "RemoveContainer" containerID="f7aad86c2c59b80bdf1c3ebaeb3464cc25b23ac0e09f031cbae0f32e5a09549a"
Oct 31 01:29:48.716984 ip-10-0-132-173 kubenswrapper[4284]: E1031 01:29:48.716966    4284 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f7aad86c2c59b80bdf1c3ebaeb3464cc25b23ac0e09f031cbae0f32e5a09549a\": container with ID starting with f7aad86c2c59b80bdf1c3ebaeb3464cc25b23ac0e09f031cbae0f32e5a09549a not found: ID does not exist" containerID="f7aad86c2c59b80bdf1c3ebaeb3464cc25b23ac0e09f031cbae0f32e5a09549a"
Oct 31 01:29:48.717062 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:48.716991    4284 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f7aad86c2c59b80bdf1c3ebaeb3464cc25b23ac0e09f031cbae0f32e5a09549a"} err="failed to get container status \"f7aad86c2c59b80bdf1c3ebaeb3464cc25b23ac0e09f031cbae0f32e5a09549a\": rpc error: code = NotFound desc = could not find container \"f7aad86c2c59b80bdf1c3ebaeb3464cc25b23ac0e09f031cbae0f32e5a09549a\": container with ID starting with f7aad86c2c59b80bdf1c3ebaeb3464cc25b23ac0e09f031cbae0f32e5a09549a not found: ID does not exist"
Oct 31 01:29:48.778130 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:48.778050    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8af02a59-8229-4932-a556-bfd6ea4e99e0-kube-api-access-xx4qm\" (UniqueName: \"kubernetes.io/projected/8af02a59-8229-4932-a556-bfd6ea4e99e0-kube-api-access-xx4qm\") pod \"8af02a59-8229-4932-a556-bfd6ea4e99e0\" (UID: \"8af02a59-8229-4932-a556-bfd6ea4e99e0\") "
Oct 31 01:29:48.778337 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:48.778130    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8af02a59-8229-4932-a556-bfd6ea4e99e0-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8af02a59-8229-4932-a556-bfd6ea4e99e0-workload-certs\") pod \"8af02a59-8229-4932-a556-bfd6ea4e99e0\" (UID: \"8af02a59-8229-4932-a556-bfd6ea4e99e0\") "
Oct 31 01:29:48.778337 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:48.778263    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8af02a59-8229-4932-a556-bfd6ea4e99e0-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8af02a59-8229-4932-a556-bfd6ea4e99e0-istio-data\") pod \"8af02a59-8229-4932-a556-bfd6ea4e99e0\" (UID: \"8af02a59-8229-4932-a556-bfd6ea4e99e0\") "
Oct 31 01:29:48.778337 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:48.778301    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8af02a59-8229-4932-a556-bfd6ea4e99e0-istio-token\" (UniqueName: \"kubernetes.io/projected/8af02a59-8229-4932-a556-bfd6ea4e99e0-istio-token\") pod \"8af02a59-8229-4932-a556-bfd6ea4e99e0\" (UID: \"8af02a59-8229-4932-a556-bfd6ea4e99e0\") "
Oct 31 01:29:48.778545 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:48.778344    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8af02a59-8229-4932-a556-bfd6ea4e99e0-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8af02a59-8229-4932-a556-bfd6ea4e99e0-credential-socket\") pod \"8af02a59-8229-4932-a556-bfd6ea4e99e0\" (UID: \"8af02a59-8229-4932-a556-bfd6ea4e99e0\") "
Oct 31 01:29:48.778545 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:48.778383    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8af02a59-8229-4932-a556-bfd6ea4e99e0-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8af02a59-8229-4932-a556-bfd6ea4e99e0-istio-envoy\") pod \"8af02a59-8229-4932-a556-bfd6ea4e99e0\" (UID: \"8af02a59-8229-4932-a556-bfd6ea4e99e0\") "
Oct 31 01:29:48.778545 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:48.778440    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8af02a59-8229-4932-a556-bfd6ea4e99e0-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8af02a59-8229-4932-a556-bfd6ea4e99e0-workload-socket\") pod \"8af02a59-8229-4932-a556-bfd6ea4e99e0\" (UID: \"8af02a59-8229-4932-a556-bfd6ea4e99e0\") "
Oct 31 01:29:48.778545 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:48.778473    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/8af02a59-8229-4932-a556-bfd6ea4e99e0-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8af02a59-8229-4932-a556-bfd6ea4e99e0-istio-podinfo\") pod \"8af02a59-8229-4932-a556-bfd6ea4e99e0\" (UID: \"8af02a59-8229-4932-a556-bfd6ea4e99e0\") "
Oct 31 01:29:48.778545 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:48.778416    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8af02a59-8229-4932-a556-bfd6ea4e99e0-workload-certs" pod "8af02a59-8229-4932-a556-bfd6ea4e99e0" (UID: "8af02a59-8229-4932-a556-bfd6ea4e99e0"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:29:48.778545 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:48.778474    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8af02a59-8229-4932-a556-bfd6ea4e99e0-istio-data" pod "8af02a59-8229-4932-a556-bfd6ea4e99e0" (UID: "8af02a59-8229-4932-a556-bfd6ea4e99e0"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:29:48.778545 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:48.778510    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/8af02a59-8229-4932-a556-bfd6ea4e99e0-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8af02a59-8229-4932-a556-bfd6ea4e99e0-istiod-ca-cert\") pod \"8af02a59-8229-4932-a556-bfd6ea4e99e0\" (UID: \"8af02a59-8229-4932-a556-bfd6ea4e99e0\") "
Oct 31 01:29:48.778953 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:48.778813    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8af02a59-8229-4932-a556-bfd6ea4e99e0-workload-socket" pod "8af02a59-8229-4932-a556-bfd6ea4e99e0" (UID: "8af02a59-8229-4932-a556-bfd6ea4e99e0"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:29:48.778953 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:48.778823    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8af02a59-8229-4932-a556-bfd6ea4e99e0-istiod-ca-cert" pod "8af02a59-8229-4932-a556-bfd6ea4e99e0" (UID: "8af02a59-8229-4932-a556-bfd6ea4e99e0"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 01:29:48.778953 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:48.778874    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8af02a59-8229-4932-a556-bfd6ea4e99e0-credential-socket" pod "8af02a59-8229-4932-a556-bfd6ea4e99e0" (UID: "8af02a59-8229-4932-a556-bfd6ea4e99e0"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:29:48.779438 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:48.779414    4284 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8af02a59-8229-4932-a556-bfd6ea4e99e0-workload-certs\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:29:48.779543 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:48.779443    4284 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8af02a59-8229-4932-a556-bfd6ea4e99e0-istio-data\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:29:48.779543 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:48.779459    4284 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8af02a59-8229-4932-a556-bfd6ea4e99e0-credential-socket\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:29:48.779543 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:48.779473    4284 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8af02a59-8229-4932-a556-bfd6ea4e99e0-workload-socket\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:29:48.779543 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:48.779487    4284 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8af02a59-8229-4932-a556-bfd6ea4e99e0-istiod-ca-cert\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:29:48.780707 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:48.780677    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8af02a59-8229-4932-a556-bfd6ea4e99e0-istio-envoy" pod "8af02a59-8229-4932-a556-bfd6ea4e99e0" (UID: "8af02a59-8229-4932-a556-bfd6ea4e99e0"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:29:48.780800 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:48.780753    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8af02a59-8229-4932-a556-bfd6ea4e99e0-kube-api-access-xx4qm" pod "8af02a59-8229-4932-a556-bfd6ea4e99e0" (UID: "8af02a59-8229-4932-a556-bfd6ea4e99e0"). InnerVolumeSpecName "kube-api-access-xx4qm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:29:48.780800 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:48.780773    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8af02a59-8229-4932-a556-bfd6ea4e99e0-istio-token" pod "8af02a59-8229-4932-a556-bfd6ea4e99e0" (UID: "8af02a59-8229-4932-a556-bfd6ea4e99e0"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:29:48.781314 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:48.781286    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/8af02a59-8229-4932-a556-bfd6ea4e99e0-istio-podinfo" pod "8af02a59-8229-4932-a556-bfd6ea4e99e0" (UID: "8af02a59-8229-4932-a556-bfd6ea4e99e0"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 01:29:48.880480 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:48.880445    4284 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xx4qm\" (UniqueName: \"kubernetes.io/projected/8af02a59-8229-4932-a556-bfd6ea4e99e0-kube-api-access-xx4qm\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:29:48.880480 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:48.880477    4284 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8af02a59-8229-4932-a556-bfd6ea4e99e0-istio-token\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:29:48.880480 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:48.880488    4284 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8af02a59-8229-4932-a556-bfd6ea4e99e0-istio-envoy\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:29:48.880730 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:48.880497    4284 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8af02a59-8229-4932-a556-bfd6ea4e99e0-istio-podinfo\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:29:49.114919 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:49.114827    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w4-8fb7f0-openshift-default-77b6794b69-bqwdf"]
Oct 31 01:29:49.120626 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:49.120599    4284 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-w4-8fb7f0-openshift-default-77b6794b69-bqwdf"]
Oct 31 01:29:49.962811 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:49.962779    4284 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="8af02a59-8229-4932-a556-bfd6ea4e99e0" path="/var/lib/kubelet/pods/8af02a59-8229-4932-a556-bfd6ea4e99e0/volumes"
Oct 31 01:29:49.985745 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:49.985691    4284 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w1-b63c24/premium-w1-b63c24-kserve-549f786d9-k7g8j" podUID="b6846d68-7b58-4637-a47f-cd55bbf79b98" containerName="main" containerID="cri-o://d441c0d2e5722e306c81c1f810ff4601f5dd44d407ba999997d31bdc7d65cd3a" gracePeriod=45
Oct 31 01:29:50.017707 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:50.017677    4284 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-b63c24/premium-w1-b63c24-kserve-549f786d9-k7g8j" podUID="b6846d68-7b58-4637-a47f-cd55bbf79b98" containerName="main" probeResult="failure" output="Get \"https://10.134.0.59:8000/ready\": dial tcp 10.134.0.59:8000: connect: connection refused"
Oct 31 01:29:50.247617 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:50.247591    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-b63c24/premium-w1-b63c24-kserve-549f786d9-k7g8j"
Oct 31 01:29:50.287974 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:50.287934    4284 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-b63c24-75cd4d9679-cpvtw" podUID="fea709fd-6209-4b40-95f6-ea5104c1cb09" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 31 01:29:50.394855 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:50.394828    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b6846d68-7b58-4637-a47f-cd55bbf79b98-kube-api-access-qgb87\" (UniqueName: \"kubernetes.io/projected/b6846d68-7b58-4637-a47f-cd55bbf79b98-kube-api-access-qgb87\") pod \"b6846d68-7b58-4637-a47f-cd55bbf79b98\" (UID: \"b6846d68-7b58-4637-a47f-cd55bbf79b98\") "
Oct 31 01:29:50.395029 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:50.394879    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b6846d68-7b58-4637-a47f-cd55bbf79b98-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b6846d68-7b58-4637-a47f-cd55bbf79b98-tmp-dir\") pod \"b6846d68-7b58-4637-a47f-cd55bbf79b98\" (UID: \"b6846d68-7b58-4637-a47f-cd55bbf79b98\") "
Oct 31 01:29:50.395029 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:50.394910    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b6846d68-7b58-4637-a47f-cd55bbf79b98-home\" (UniqueName: \"kubernetes.io/empty-dir/b6846d68-7b58-4637-a47f-cd55bbf79b98-home\") pod \"b6846d68-7b58-4637-a47f-cd55bbf79b98\" (UID: \"b6846d68-7b58-4637-a47f-cd55bbf79b98\") "
Oct 31 01:29:50.395029 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:50.394933    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b6846d68-7b58-4637-a47f-cd55bbf79b98-dshm\" (UniqueName: \"kubernetes.io/empty-dir/b6846d68-7b58-4637-a47f-cd55bbf79b98-dshm\") pod \"b6846d68-7b58-4637-a47f-cd55bbf79b98\" (UID: \"b6846d68-7b58-4637-a47f-cd55bbf79b98\") "
Oct 31 01:29:50.395158 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:50.395066    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b6846d68-7b58-4637-a47f-cd55bbf79b98-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b6846d68-7b58-4637-a47f-cd55bbf79b98-model-cache\") pod \"b6846d68-7b58-4637-a47f-cd55bbf79b98\" (UID: \"b6846d68-7b58-4637-a47f-cd55bbf79b98\") "
Oct 31 01:29:50.395158 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:50.395146    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/b6846d68-7b58-4637-a47f-cd55bbf79b98-tls-certs\" (UniqueName: \"kubernetes.io/secret/b6846d68-7b58-4637-a47f-cd55bbf79b98-tls-certs\") pod \"b6846d68-7b58-4637-a47f-cd55bbf79b98\" (UID: \"b6846d68-7b58-4637-a47f-cd55bbf79b98\") "
Oct 31 01:29:50.395452 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:50.395388    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b6846d68-7b58-4637-a47f-cd55bbf79b98-home" pod "b6846d68-7b58-4637-a47f-cd55bbf79b98" (UID: "b6846d68-7b58-4637-a47f-cd55bbf79b98"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:29:50.395452 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:50.395425    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b6846d68-7b58-4637-a47f-cd55bbf79b98-model-cache" pod "b6846d68-7b58-4637-a47f-cd55bbf79b98" (UID: "b6846d68-7b58-4637-a47f-cd55bbf79b98"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:29:50.395653 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:50.395489    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b6846d68-7b58-4637-a47f-cd55bbf79b98-tmp-dir" pod "b6846d68-7b58-4637-a47f-cd55bbf79b98" (UID: "b6846d68-7b58-4637-a47f-cd55bbf79b98"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:29:50.396405 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:50.396382    4284 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b6846d68-7b58-4637-a47f-cd55bbf79b98-tmp-dir\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:29:50.396532 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:50.396518    4284 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b6846d68-7b58-4637-a47f-cd55bbf79b98-home\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:29:50.396673 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:50.396648    4284 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b6846d68-7b58-4637-a47f-cd55bbf79b98-model-cache\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:29:50.400683 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:50.400656    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b6846d68-7b58-4637-a47f-cd55bbf79b98-dshm" pod "b6846d68-7b58-4637-a47f-cd55bbf79b98" (UID: "b6846d68-7b58-4637-a47f-cd55bbf79b98"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:29:50.400683 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:50.400672    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b6846d68-7b58-4637-a47f-cd55bbf79b98-kube-api-access-qgb87" pod "b6846d68-7b58-4637-a47f-cd55bbf79b98" (UID: "b6846d68-7b58-4637-a47f-cd55bbf79b98"). InnerVolumeSpecName "kube-api-access-qgb87". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:29:50.400887 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:50.400714    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b6846d68-7b58-4637-a47f-cd55bbf79b98-tls-certs" pod "b6846d68-7b58-4637-a47f-cd55bbf79b98" (UID: "b6846d68-7b58-4637-a47f-cd55bbf79b98"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 01:29:50.498134 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:50.498046    4284 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qgb87\" (UniqueName: \"kubernetes.io/projected/b6846d68-7b58-4637-a47f-cd55bbf79b98-kube-api-access-qgb87\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:29:50.498134 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:50.498077    4284 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b6846d68-7b58-4637-a47f-cd55bbf79b98-dshm\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:29:50.498134 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:50.498086    4284 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b6846d68-7b58-4637-a47f-cd55bbf79b98-tls-certs\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:29:50.713107 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:50.712878    4284 generic.go:356] "Generic (PLEG): container finished" podID="b6846d68-7b58-4637-a47f-cd55bbf79b98" containerID="d441c0d2e5722e306c81c1f810ff4601f5dd44d407ba999997d31bdc7d65cd3a" exitCode=0
Oct 31 01:29:50.713107 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:50.712956    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-b63c24/premium-w1-b63c24-kserve-549f786d9-k7g8j"
Oct 31 01:29:50.713107 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:50.713083    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-b63c24/premium-w1-b63c24-kserve-549f786d9-k7g8j" event={"ID":"b6846d68-7b58-4637-a47f-cd55bbf79b98","Type":"ContainerDied","Data":"d441c0d2e5722e306c81c1f810ff4601f5dd44d407ba999997d31bdc7d65cd3a"}
Oct 31 01:29:50.713666 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:50.713122    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-b63c24/premium-w1-b63c24-kserve-549f786d9-k7g8j" event={"ID":"b6846d68-7b58-4637-a47f-cd55bbf79b98","Type":"ContainerDied","Data":"1125046877e9650a0960bbc9c501d59767fd291a929eb8a566e671226cbe443a"}
Oct 31 01:29:50.713666 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:50.713158    4284 scope.go:122] "RemoveContainer" containerID="d441c0d2e5722e306c81c1f810ff4601f5dd44d407ba999997d31bdc7d65cd3a"
Oct 31 01:29:50.724245 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:50.724197    4284 scope.go:122] "RemoveContainer" containerID="d441c0d2e5722e306c81c1f810ff4601f5dd44d407ba999997d31bdc7d65cd3a"
Oct 31 01:29:50.724639 ip-10-0-132-173 kubenswrapper[4284]: E1031 01:29:50.724619    4284 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d441c0d2e5722e306c81c1f810ff4601f5dd44d407ba999997d31bdc7d65cd3a\": container with ID starting with d441c0d2e5722e306c81c1f810ff4601f5dd44d407ba999997d31bdc7d65cd3a not found: ID does not exist" containerID="d441c0d2e5722e306c81c1f810ff4601f5dd44d407ba999997d31bdc7d65cd3a"
Oct 31 01:29:50.724710 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:50.724646    4284 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d441c0d2e5722e306c81c1f810ff4601f5dd44d407ba999997d31bdc7d65cd3a"} err="failed to get container status \"d441c0d2e5722e306c81c1f810ff4601f5dd44d407ba999997d31bdc7d65cd3a\": rpc error: code = NotFound desc = could not find container \"d441c0d2e5722e306c81c1f810ff4601f5dd44d407ba999997d31bdc7d65cd3a\": container with ID starting with d441c0d2e5722e306c81c1f810ff4601f5dd44d407ba999997d31bdc7d65cd3a not found: ID does not exist"
Oct 31 01:29:50.871157 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:50.871077    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w1-b63c24/premium-w1-b63c24-kserve-549f786d9-k7g8j"]
Oct 31 01:29:50.876946 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:50.876919    4284 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w1-b63c24/premium-w1-b63c24-kserve-549f786d9-k7g8j"]
Oct 31 01:29:51.312342 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:51.312303    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:29:51.497573 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:51.497535    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:29:51.962478 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:51.962447    4284 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b6846d68-7b58-4637-a47f-cd55bbf79b98" path="/var/lib/kubelet/pods/b6846d68-7b58-4637-a47f-cd55bbf79b98/volumes"
Oct 31 01:29:56.327795 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:56.327755    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:29:57.136169 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:57.136124    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:29:57.291443 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:29:57.291407    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:30:00.243538 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:00.243499    4284 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-ef2572-29890170-827hl"]
Oct 31 01:30:00.244067 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:00.244031    4284 cpu_manager.go:414] "containerMap: removing container" podUID="8af02a59-8229-4932-a556-bfd6ea4e99e0" containerName="istio-proxy"
Oct 31 01:30:00.244067 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:00.244045    4284 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8af02a59-8229-4932-a556-bfd6ea4e99e0" containerName="istio-proxy"
Oct 31 01:30:00.244067 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:00.244065    4284 cpu_manager.go:414] "containerMap: removing container" podUID="8af02a59-8229-4932-a556-bfd6ea4e99e0" containerName="istio-proxy"
Oct 31 01:30:00.244187 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:00.244070    4284 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8af02a59-8229-4932-a556-bfd6ea4e99e0" containerName="istio-proxy"
Oct 31 01:30:00.244187 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:00.244092    4284 cpu_manager.go:414] "containerMap: removing container" podUID="ed93ba84-d871-4811-ade6-5581127ca3dc" containerName="istio-proxy"
Oct 31 01:30:00.244187 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:00.244097    4284 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ed93ba84-d871-4811-ade6-5581127ca3dc" containerName="istio-proxy"
Oct 31 01:30:00.244187 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:00.244107    4284 cpu_manager.go:414] "containerMap: removing container" podUID="ed93ba84-d871-4811-ade6-5581127ca3dc" containerName="istio-proxy"
Oct 31 01:30:00.244187 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:00.244112    4284 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ed93ba84-d871-4811-ade6-5581127ca3dc" containerName="istio-proxy"
Oct 31 01:30:00.244187 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:00.244122    4284 cpu_manager.go:414] "containerMap: removing container" podUID="b6846d68-7b58-4637-a47f-cd55bbf79b98" containerName="main"
Oct 31 01:30:00.244187 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:00.244126    4284 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b6846d68-7b58-4637-a47f-cd55bbf79b98" containerName="main"
Oct 31 01:30:00.244187 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:00.244186    4284 memory_manager.go:365] "RemoveStaleState removing state" podUID="b6846d68-7b58-4637-a47f-cd55bbf79b98" containerName="main"
Oct 31 01:30:00.244456 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:00.244195    4284 memory_manager.go:365] "RemoveStaleState removing state" podUID="8af02a59-8229-4932-a556-bfd6ea4e99e0" containerName="istio-proxy"
Oct 31 01:30:00.244456 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:00.244202    4284 memory_manager.go:365] "RemoveStaleState removing state" podUID="8af02a59-8229-4932-a556-bfd6ea4e99e0" containerName="istio-proxy"
Oct 31 01:30:00.244456 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:00.244234    4284 memory_manager.go:365] "RemoveStaleState removing state" podUID="ed93ba84-d871-4811-ade6-5581127ca3dc" containerName="istio-proxy"
Oct 31 01:30:00.244456 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:00.244241    4284 memory_manager.go:365] "RemoveStaleState removing state" podUID="ed93ba84-d871-4811-ade6-5581127ca3dc" containerName="istio-proxy"
Oct 31 01:30:00.248558 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:00.248540    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-ef2572-29890170-827hl"
Oct 31 01:30:00.288826 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:00.288783    4284 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-b63c24-75cd4d9679-cpvtw" podUID="fea709fd-6209-4b40-95f6-ea5104c1cb09" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 31 01:30:00.322831 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:00.322783    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-khgfw\" (UniqueName: \"kubernetes.io/projected/c3fc41e7-426c-4bdb-b5ef-55dce305cfae-kube-api-access-khgfw\") pod \"maas-api-key-cleanup-e2e-worker-w4-ef2572-29890170-827hl\" (UID: \"c3fc41e7-426c-4bdb-b5ef-55dce305cfae\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-ef2572-29890170-827hl"
Oct 31 01:30:00.323032 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:00.322949    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c3fc41e7-426c-4bdb-b5ef-55dce305cfae-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w4-ef2572-29890170-827hl\" (UID: \"c3fc41e7-426c-4bdb-b5ef-55dce305cfae\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-ef2572-29890170-827hl"
Oct 31 01:30:00.342376 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:00.342348    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-ef2572-29890170-827hl"]
Oct 31 01:30:00.385445 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:00.385414    4284 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-8pzqp\""
Oct 31 01:30:00.424444 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:00.424410    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-khgfw\" (UniqueName: \"kubernetes.io/projected/c3fc41e7-426c-4bdb-b5ef-55dce305cfae-kube-api-access-khgfw\") pod \"maas-api-key-cleanup-e2e-worker-w4-ef2572-29890170-827hl\" (UID: \"c3fc41e7-426c-4bdb-b5ef-55dce305cfae\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-ef2572-29890170-827hl"
Oct 31 01:30:00.424652 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:00.424481    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c3fc41e7-426c-4bdb-b5ef-55dce305cfae-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w4-ef2572-29890170-827hl\" (UID: \"c3fc41e7-426c-4bdb-b5ef-55dce305cfae\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-ef2572-29890170-827hl"
Oct 31 01:30:00.425042 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:00.425022    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c3fc41e7-426c-4bdb-b5ef-55dce305cfae-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w4-ef2572-29890170-827hl\" (UID: \"c3fc41e7-426c-4bdb-b5ef-55dce305cfae\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-ef2572-29890170-827hl"
Oct 31 01:30:00.503028 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:00.502951    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-khgfw\" (UniqueName: \"kubernetes.io/projected/c3fc41e7-426c-4bdb-b5ef-55dce305cfae-kube-api-access-khgfw\") pod \"maas-api-key-cleanup-e2e-worker-w4-ef2572-29890170-827hl\" (UID: \"c3fc41e7-426c-4bdb-b5ef-55dce305cfae\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-ef2572-29890170-827hl"
Oct 31 01:30:00.562801 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:00.562770    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-ef2572-29890170-827hl"
Oct 31 01:30:00.929801 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:00.929774    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-ef2572-29890170-827hl"]
Oct 31 01:30:00.931385 ip-10-0-132-173 kubenswrapper[4284]: W1031 01:30:00.931354    4284 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc3fc41e7_426c_4bdb_b5ef_55dce305cfae.slice/crio-c103be51702bab5a14c020e947f08809245830bdec9688a3e7c43fcc8c6d2e62 WatchSource:0}: Error finding container c103be51702bab5a14c020e947f08809245830bdec9688a3e7c43fcc8c6d2e62: Status 404 returned error can't find the container with id c103be51702bab5a14c020e947f08809245830bdec9688a3e7c43fcc8c6d2e62
Oct 31 01:30:01.218263 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:01.218198    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:30:01.773824 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:01.773776    4284 generic.go:356] "Generic (PLEG): container finished" podID="c3fc41e7-426c-4bdb-b5ef-55dce305cfae" containerID="1893f9c0dfc2c21909ee3b29e56bc0d76f9aa1bd39fcd75806aabd1ddbe0514b" exitCode=60
Oct 31 01:30:01.774349 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:01.773851    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-ef2572-29890170-827hl" event={"ID":"c3fc41e7-426c-4bdb-b5ef-55dce305cfae","Type":"ContainerDied","Data":"1893f9c0dfc2c21909ee3b29e56bc0d76f9aa1bd39fcd75806aabd1ddbe0514b"}
Oct 31 01:30:01.774349 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:01.773911    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-ef2572-29890170-827hl" event={"ID":"c3fc41e7-426c-4bdb-b5ef-55dce305cfae","Type":"ContainerStarted","Data":"c103be51702bab5a14c020e947f08809245830bdec9688a3e7c43fcc8c6d2e62"}
Oct 31 01:30:01.774349 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:01.774239    4284 scope.go:122] "RemoveContainer" containerID="1893f9c0dfc2c21909ee3b29e56bc0d76f9aa1bd39fcd75806aabd1ddbe0514b"
Oct 31 01:30:02.781499 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:02.781464    4284 generic.go:356] "Generic (PLEG): container finished" podID="c3fc41e7-426c-4bdb-b5ef-55dce305cfae" containerID="d1642a067a9e3b648f56d7bcad0f8f1d34168f42b9ea306ac0e8f743238d89bf" exitCode=60
Oct 31 01:30:02.781936 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:02.781558    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-ef2572-29890170-827hl" event={"ID":"c3fc41e7-426c-4bdb-b5ef-55dce305cfae","Type":"ContainerDied","Data":"d1642a067a9e3b648f56d7bcad0f8f1d34168f42b9ea306ac0e8f743238d89bf"}
Oct 31 01:30:02.781936 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:02.781630    4284 scope.go:122] "RemoveContainer" containerID="1893f9c0dfc2c21909ee3b29e56bc0d76f9aa1bd39fcd75806aabd1ddbe0514b"
Oct 31 01:30:02.781936 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:02.781908    4284 scope.go:122] "RemoveContainer" containerID="d1642a067a9e3b648f56d7bcad0f8f1d34168f42b9ea306ac0e8f743238d89bf"
Oct 31 01:30:02.782207 ip-10-0-132-173 kubenswrapper[4284]: E1031 01:30:02.782187    4284 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w4-ef2572-29890170-827hl_odh-ai-gateway-infra(c3fc41e7-426c-4bdb-b5ef-55dce305cfae)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-ef2572-29890170-827hl" podUID="c3fc41e7-426c-4bdb-b5ef-55dce305cfae"
Oct 31 01:30:03.789154 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:03.789125    4284 scope.go:122] "RemoveContainer" containerID="d1642a067a9e3b648f56d7bcad0f8f1d34168f42b9ea306ac0e8f743238d89bf"
Oct 31 01:30:03.790671 ip-10-0-132-173 kubenswrapper[4284]: E1031 01:30:03.789419    4284 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w4-ef2572-29890170-827hl_odh-ai-gateway-infra(c3fc41e7-426c-4bdb-b5ef-55dce305cfae)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-ef2572-29890170-827hl" podUID="c3fc41e7-426c-4bdb-b5ef-55dce305cfae"
Oct 31 01:30:05.611458 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:05.611416    4284 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-4b97d7b7-lq6kx"]
Oct 31 01:30:05.614347 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:05.614330    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-4b97d7b7-lq6kx"
Oct 31 01:30:05.670529 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:05.670485    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-4b97d7b7-lq6kx"]
Oct 31 01:30:05.682771 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:05.682740    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/e0d645bd-be9d-43da-b745-40ff32e3a51a-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-4b97d7b7-lq6kx\" (UID: \"e0d645bd-be9d-43da-b745-40ff32e3a51a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-4b97d7b7-lq6kx"
Oct 31 01:30:05.682942 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:05.682787    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vx9vt\" (UniqueName: \"kubernetes.io/projected/e0d645bd-be9d-43da-b745-40ff32e3a51a-kube-api-access-vx9vt\") pod \"maas-api-revoke-sub-4b97d7b7-lq6kx\" (UID: \"e0d645bd-be9d-43da-b745-40ff32e3a51a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-4b97d7b7-lq6kx"
Oct 31 01:30:05.682942 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:05.682909    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/e0d645bd-be9d-43da-b745-40ff32e3a51a-maas-api-xks-ca\") pod \"maas-api-revoke-sub-4b97d7b7-lq6kx\" (UID: \"e0d645bd-be9d-43da-b745-40ff32e3a51a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-4b97d7b7-lq6kx"
Oct 31 01:30:05.713634 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:05.713600    4284 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 31 01:30:05.783606 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:05.783569    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/e0d645bd-be9d-43da-b745-40ff32e3a51a-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-4b97d7b7-lq6kx\" (UID: \"e0d645bd-be9d-43da-b745-40ff32e3a51a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-4b97d7b7-lq6kx"
Oct 31 01:30:05.783838 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:05.783622    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vx9vt\" (UniqueName: \"kubernetes.io/projected/e0d645bd-be9d-43da-b745-40ff32e3a51a-kube-api-access-vx9vt\") pod \"maas-api-revoke-sub-4b97d7b7-lq6kx\" (UID: \"e0d645bd-be9d-43da-b745-40ff32e3a51a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-4b97d7b7-lq6kx"
Oct 31 01:30:05.783838 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:05.783675    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/e0d645bd-be9d-43da-b745-40ff32e3a51a-maas-api-xks-ca\") pod \"maas-api-revoke-sub-4b97d7b7-lq6kx\" (UID: \"e0d645bd-be9d-43da-b745-40ff32e3a51a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-4b97d7b7-lq6kx"
Oct 31 01:30:05.784343 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:05.784316    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/e0d645bd-be9d-43da-b745-40ff32e3a51a-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-4b97d7b7-lq6kx\" (UID: \"e0d645bd-be9d-43da-b745-40ff32e3a51a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-4b97d7b7-lq6kx"
Oct 31 01:30:05.786145 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:05.786121    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/e0d645bd-be9d-43da-b745-40ff32e3a51a-maas-api-xks-ca\") pod \"maas-api-revoke-sub-4b97d7b7-lq6kx\" (UID: \"e0d645bd-be9d-43da-b745-40ff32e3a51a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-4b97d7b7-lq6kx"
Oct 31 01:30:05.842824 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:05.842787    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vx9vt\" (UniqueName: \"kubernetes.io/projected/e0d645bd-be9d-43da-b745-40ff32e3a51a-kube-api-access-vx9vt\") pod \"maas-api-revoke-sub-4b97d7b7-lq6kx\" (UID: \"e0d645bd-be9d-43da-b745-40ff32e3a51a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-4b97d7b7-lq6kx"
Oct 31 01:30:05.926670 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:05.926634    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-4b97d7b7-lq6kx"
Oct 31 01:30:06.295685 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:06.295648    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-4b97d7b7-lq6kx"]
Oct 31 01:30:06.299242 ip-10-0-132-173 kubenswrapper[4284]: W1031 01:30:06.297655    4284 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode0d645bd_be9d_43da_b745_40ff32e3a51a.slice/crio-9cb5ecded2fa248a573d10de3611c59e97b2cf077dbb77582394479f47d65c0e WatchSource:0}: Error finding container 9cb5ecded2fa248a573d10de3611c59e97b2cf077dbb77582394479f47d65c0e: Status 404 returned error can't find the container with id 9cb5ecded2fa248a573d10de3611c59e97b2cf077dbb77582394479f47d65c0e
Oct 31 01:30:06.400153 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:06.400120    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:30:06.806819 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:06.806784    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-4b97d7b7-lq6kx" event={"ID":"e0d645bd-be9d-43da-b745-40ff32e3a51a","Type":"ContainerStarted","Data":"08b1038e44b10354540b2b2137bcfbdd97622eb9937088196ae2c395e8b7b2ce"}
Oct 31 01:30:06.807240 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:06.806840    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-4b97d7b7-lq6kx" event={"ID":"e0d645bd-be9d-43da-b745-40ff32e3a51a","Type":"ContainerStarted","Data":"9cb5ecded2fa248a573d10de3611c59e97b2cf077dbb77582394479f47d65c0e"}
Oct 31 01:30:06.921394 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:06.921338    4284 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-4b97d7b7-lq6kx" podStartSLOduration=1.921322424 podStartE2EDuration="1.921322424s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 01:30:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 01:30:06.919376027 +0000 UTC m=+3721.064382955" watchObservedRunningTime="2026-10-31 01:30:06.921322424 +0000 UTC m=+3721.066329340"
Oct 31 01:30:07.451965 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:07.451940    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-b63c24-75cd4d9679-cpvtw"
Oct 31 01:30:07.602872 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:07.602790    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/fea709fd-6209-4b40-95f6-ea5104c1cb09-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fea709fd-6209-4b40-95f6-ea5104c1cb09-plugins-config-volume\") pod \"fea709fd-6209-4b40-95f6-ea5104c1cb09\" (UID: \"fea709fd-6209-4b40-95f6-ea5104c1cb09\") "
Oct 31 01:30:07.603049 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:07.603030    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fea709fd-6209-4b40-95f6-ea5104c1cb09-kube-api-access-bn2dl\" (UniqueName: \"kubernetes.io/projected/fea709fd-6209-4b40-95f6-ea5104c1cb09-kube-api-access-bn2dl\") pod \"fea709fd-6209-4b40-95f6-ea5104c1cb09\" (UID: \"fea709fd-6209-4b40-95f6-ea5104c1cb09\") "
Oct 31 01:30:07.603242 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:07.603203    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fea709fd-6209-4b40-95f6-ea5104c1cb09-plugins-config-volume" pod "fea709fd-6209-4b40-95f6-ea5104c1cb09" (UID: "fea709fd-6209-4b40-95f6-ea5104c1cb09"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 01:30:07.603395 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:07.603382    4284 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fea709fd-6209-4b40-95f6-ea5104c1cb09-plugins-config-volume\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:30:07.605384 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:07.605362    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fea709fd-6209-4b40-95f6-ea5104c1cb09-kube-api-access-bn2dl" pod "fea709fd-6209-4b40-95f6-ea5104c1cb09" (UID: "fea709fd-6209-4b40-95f6-ea5104c1cb09"). InnerVolumeSpecName "kube-api-access-bn2dl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:30:07.704630 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:07.704592    4284 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bn2dl\" (UniqueName: \"kubernetes.io/projected/fea709fd-6209-4b40-95f6-ea5104c1cb09-kube-api-access-bn2dl\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:30:07.813667 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:07.813634    4284 generic.go:356] "Generic (PLEG): container finished" podID="e0d645bd-be9d-43da-b745-40ff32e3a51a" containerID="08b1038e44b10354540b2b2137bcfbdd97622eb9937088196ae2c395e8b7b2ce" exitCode=0
Oct 31 01:30:07.814137 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:07.813728    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-4b97d7b7-lq6kx" event={"ID":"e0d645bd-be9d-43da-b745-40ff32e3a51a","Type":"ContainerDied","Data":"08b1038e44b10354540b2b2137bcfbdd97622eb9937088196ae2c395e8b7b2ce"}
Oct 31 01:30:07.815204 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:07.815177    4284 generic.go:356] "Generic (PLEG): container finished" podID="fea709fd-6209-4b40-95f6-ea5104c1cb09" containerID="8bfc236166f01984405d4ec5670d53d05a5e6080fce6955bb83541e25c862230" exitCode=0
Oct 31 01:30:07.815342 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:07.815228    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-b63c24-75cd4d9679-cpvtw" event={"ID":"fea709fd-6209-4b40-95f6-ea5104c1cb09","Type":"ContainerDied","Data":"8bfc236166f01984405d4ec5670d53d05a5e6080fce6955bb83541e25c862230"}
Oct 31 01:30:07.815342 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:07.815262    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-b63c24-75cd4d9679-cpvtw" event={"ID":"fea709fd-6209-4b40-95f6-ea5104c1cb09","Type":"ContainerDied","Data":"053a606aa32adfd9b6746aa1115497d0709dd6ead01e25768c78f9189d0271dc"}
Oct 31 01:30:07.815342 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:07.815286    4284 scope.go:122] "RemoveContainer" containerID="8bfc236166f01984405d4ec5670d53d05a5e6080fce6955bb83541e25c862230"
Oct 31 01:30:07.815342 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:07.815230    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-b63c24-75cd4d9679-cpvtw"
Oct 31 01:30:07.824505 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:07.824486    4284 scope.go:122] "RemoveContainer" containerID="8bfc236166f01984405d4ec5670d53d05a5e6080fce6955bb83541e25c862230"
Oct 31 01:30:07.824768 ip-10-0-132-173 kubenswrapper[4284]: E1031 01:30:07.824749    4284 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8bfc236166f01984405d4ec5670d53d05a5e6080fce6955bb83541e25c862230\": container with ID starting with 8bfc236166f01984405d4ec5670d53d05a5e6080fce6955bb83541e25c862230 not found: ID does not exist" containerID="8bfc236166f01984405d4ec5670d53d05a5e6080fce6955bb83541e25c862230"
Oct 31 01:30:07.824811 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:07.824775    4284 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8bfc236166f01984405d4ec5670d53d05a5e6080fce6955bb83541e25c862230"} err="failed to get container status \"8bfc236166f01984405d4ec5670d53d05a5e6080fce6955bb83541e25c862230\": rpc error: code = NotFound desc = could not find container \"8bfc236166f01984405d4ec5670d53d05a5e6080fce6955bb83541e25c862230\": container with ID starting with 8bfc236166f01984405d4ec5670d53d05a5e6080fce6955bb83541e25c862230 not found: ID does not exist"
Oct 31 01:30:08.032501 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:08.032463    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w1-b63c24-75cd4d9679-cpvtw"]
Oct 31 01:30:08.036711 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:08.036681    4284 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w1-b63c24-75cd4d9679-cpvtw"]
Oct 31 01:30:08.962734 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:08.962707    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-4b97d7b7-lq6kx"
Oct 31 01:30:09.120072 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:09.119990    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e0d645bd-be9d-43da-b745-40ff32e3a51a-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/e0d645bd-be9d-43da-b745-40ff32e3a51a-maas-api-openshift-ca\") pod \"e0d645bd-be9d-43da-b745-40ff32e3a51a\" (UID: \"e0d645bd-be9d-43da-b745-40ff32e3a51a\") "
Oct 31 01:30:09.120072 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:09.120033    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/e0d645bd-be9d-43da-b745-40ff32e3a51a-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/e0d645bd-be9d-43da-b745-40ff32e3a51a-maas-api-xks-ca\") pod \"e0d645bd-be9d-43da-b745-40ff32e3a51a\" (UID: \"e0d645bd-be9d-43da-b745-40ff32e3a51a\") "
Oct 31 01:30:09.120351 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:09.120130    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e0d645bd-be9d-43da-b745-40ff32e3a51a-kube-api-access-vx9vt\" (UniqueName: \"kubernetes.io/projected/e0d645bd-be9d-43da-b745-40ff32e3a51a-kube-api-access-vx9vt\") pod \"e0d645bd-be9d-43da-b745-40ff32e3a51a\" (UID: \"e0d645bd-be9d-43da-b745-40ff32e3a51a\") "
Oct 31 01:30:09.120467 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:09.120442    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e0d645bd-be9d-43da-b745-40ff32e3a51a-maas-api-openshift-ca" pod "e0d645bd-be9d-43da-b745-40ff32e3a51a" (UID: "e0d645bd-be9d-43da-b745-40ff32e3a51a"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 01:30:09.120576 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:09.120560    4284 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/e0d645bd-be9d-43da-b745-40ff32e3a51a-maas-api-openshift-ca\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:30:09.122356 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:09.122333    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e0d645bd-be9d-43da-b745-40ff32e3a51a-maas-api-xks-ca" pod "e0d645bd-be9d-43da-b745-40ff32e3a51a" (UID: "e0d645bd-be9d-43da-b745-40ff32e3a51a"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 01:30:09.122446 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:09.122374    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e0d645bd-be9d-43da-b745-40ff32e3a51a-kube-api-access-vx9vt" pod "e0d645bd-be9d-43da-b745-40ff32e3a51a" (UID: "e0d645bd-be9d-43da-b745-40ff32e3a51a"). InnerVolumeSpecName "kube-api-access-vx9vt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:30:09.221497 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:09.221456    4284 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vx9vt\" (UniqueName: \"kubernetes.io/projected/e0d645bd-be9d-43da-b745-40ff32e3a51a-kube-api-access-vx9vt\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:30:09.221497 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:09.221490    4284 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/e0d645bd-be9d-43da-b745-40ff32e3a51a-maas-api-xks-ca\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:30:09.703974 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:09.703940    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:30:09.828910 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:09.828872    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-4b97d7b7-lq6kx"
Oct 31 01:30:09.828910 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:09.828900    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-4b97d7b7-lq6kx" event={"ID":"e0d645bd-be9d-43da-b745-40ff32e3a51a","Type":"ContainerDied","Data":"9cb5ecded2fa248a573d10de3611c59e97b2cf077dbb77582394479f47d65c0e"}
Oct 31 01:30:09.829142 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:09.828968    4284 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9cb5ecded2fa248a573d10de3611c59e97b2cf077dbb77582394479f47d65c0e"
Oct 31 01:30:09.963091 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:09.963005    4284 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="fea709fd-6209-4b40-95f6-ea5104c1cb09" path="/var/lib/kubelet/pods/fea709fd-6209-4b40-95f6-ea5104c1cb09/volumes"
Oct 31 01:30:11.192092 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:11.192061    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:30:11.736953 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:11.736905    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:30:11.883644 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:11.883609    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:30:13.038294 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:13.038263    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:30:15.959352 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:15.959323    4284 scope.go:122] "RemoveContainer" containerID="d1642a067a9e3b648f56d7bcad0f8f1d34168f42b9ea306ac0e8f743238d89bf"
Oct 31 01:30:15.971268 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:15.971241    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:30:16.126005 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:16.125961    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:30:16.272613 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:16.272525    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:30:16.875861 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:16.875825    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-ef2572-29890170-827hl" event={"ID":"c3fc41e7-426c-4bdb-b5ef-55dce305cfae","Type":"ContainerStarted","Data":"b4639997379993c1a7523c71a450a53e66e0604365affa54b06c967dda2ea25b"}
Oct 31 01:30:16.988624 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:16.988578    4284 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-ef2572-29890170-827hl" podStartSLOduration=16.988563994 podStartE2EDuration="16.988563994s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 01:30:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 01:30:16.986625374 +0000 UTC m=+3731.131632290" watchObservedRunningTime="2026-10-31 01:30:16.988563994 +0000 UTC m=+3731.133570910"
Oct 31 01:30:17.013381 ip-10-0-132-173 kubenswrapper[4284]: E1031 01:30:17.013347    4284 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fd8sp" containerName="extractor"
Oct 31 01:30:17.883012 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:17.882982    4284 generic.go:356] "Generic (PLEG): container finished" podID="c3fc41e7-426c-4bdb-b5ef-55dce305cfae" containerID="b4639997379993c1a7523c71a450a53e66e0604365affa54b06c967dda2ea25b" exitCode=60
Oct 31 01:30:17.883204 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:17.883134    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-ef2572-29890170-827hl" event={"ID":"c3fc41e7-426c-4bdb-b5ef-55dce305cfae","Type":"ContainerDied","Data":"b4639997379993c1a7523c71a450a53e66e0604365affa54b06c967dda2ea25b"}
Oct 31 01:30:17.883204 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:17.883177    4284 scope.go:122] "RemoveContainer" containerID="d1642a067a9e3b648f56d7bcad0f8f1d34168f42b9ea306ac0e8f743238d89bf"
Oct 31 01:30:17.883602 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:17.883578    4284 scope.go:122] "RemoveContainer" containerID="b4639997379993c1a7523c71a450a53e66e0604365affa54b06c967dda2ea25b"
Oct 31 01:30:17.883876 ip-10-0-132-173 kubenswrapper[4284]: E1031 01:30:17.883854    4284 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w4-ef2572-29890170-827hl_odh-ai-gateway-infra(c3fc41e7-426c-4bdb-b5ef-55dce305cfae)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-ef2572-29890170-827hl" podUID="c3fc41e7-426c-4bdb-b5ef-55dce305cfae"
Oct 31 01:30:18.029843 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:18.029811    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-ef2572-29890170-827hl"]
Oct 31 01:30:19.057589 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:19.057554    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-ef2572-29890170-827hl"
Oct 31 01:30:19.133550 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:19.133521    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c3fc41e7-426c-4bdb-b5ef-55dce305cfae-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c3fc41e7-426c-4bdb-b5ef-55dce305cfae-maas-api-service-ca\") pod \"c3fc41e7-426c-4bdb-b5ef-55dce305cfae\" (UID: \"c3fc41e7-426c-4bdb-b5ef-55dce305cfae\") "
Oct 31 01:30:19.133739 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:19.133656    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c3fc41e7-426c-4bdb-b5ef-55dce305cfae-kube-api-access-khgfw\" (UniqueName: \"kubernetes.io/projected/c3fc41e7-426c-4bdb-b5ef-55dce305cfae-kube-api-access-khgfw\") pod \"c3fc41e7-426c-4bdb-b5ef-55dce305cfae\" (UID: \"c3fc41e7-426c-4bdb-b5ef-55dce305cfae\") "
Oct 31 01:30:19.133894 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:19.133868    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c3fc41e7-426c-4bdb-b5ef-55dce305cfae-maas-api-service-ca" pod "c3fc41e7-426c-4bdb-b5ef-55dce305cfae" (UID: "c3fc41e7-426c-4bdb-b5ef-55dce305cfae"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 01:30:19.134080 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:19.134063    4284 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c3fc41e7-426c-4bdb-b5ef-55dce305cfae-maas-api-service-ca\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:30:19.136250 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:19.136225    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c3fc41e7-426c-4bdb-b5ef-55dce305cfae-kube-api-access-khgfw" pod "c3fc41e7-426c-4bdb-b5ef-55dce305cfae" (UID: "c3fc41e7-426c-4bdb-b5ef-55dce305cfae"). InnerVolumeSpecName "kube-api-access-khgfw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:30:19.234912 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:19.234836    4284 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-khgfw\" (UniqueName: \"kubernetes.io/projected/c3fc41e7-426c-4bdb-b5ef-55dce305cfae-kube-api-access-khgfw\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:30:19.915800 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:19.915772    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-ef2572-29890170-827hl"
Oct 31 01:30:19.915985 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:19.915764    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-ef2572-29890170-827hl" event={"ID":"c3fc41e7-426c-4bdb-b5ef-55dce305cfae","Type":"ContainerDied","Data":"c103be51702bab5a14c020e947f08809245830bdec9688a3e7c43fcc8c6d2e62"}
Oct 31 01:30:19.915985 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:19.915906    4284 scope.go:122] "RemoveContainer" containerID="b4639997379993c1a7523c71a450a53e66e0604365affa54b06c967dda2ea25b"
Oct 31 01:30:20.076850 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:20.076816    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-ef2572-29890170-827hl"]
Oct 31 01:30:20.140828 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:20.140785    4284 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-ef2572-29890170-827hl"]
Oct 31 01:30:20.170939 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:20.170862    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:30:20.332826 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:20.332784    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:30:21.189406 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:21.189363    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:30:21.860720 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:21.860684    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:30:21.962819 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:21.962774    4284 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c3fc41e7-426c-4bdb-b5ef-55dce305cfae" path="/var/lib/kubelet/pods/c3fc41e7-426c-4bdb-b5ef-55dce305cfae/volumes"
Oct 31 01:30:22.046943 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:22.046892    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:30:22.271938 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:22.271902    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:30:25.432501 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:25.432460    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:30:26.960939 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:26.960896    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:30:27.200576 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:27.200537    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:30:27.350743 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:27.350662    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:30:27.980748 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:27.980714    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:30:28.483838 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:28.483804    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:30:32.090689 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:32.090656    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:30:34.924789 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:34.924757    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:30:35.078149 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:35.078105    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:30:35.304844 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:35.304762    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:30:35.310429 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:35.310410    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w4-a72754-openshift-default-6dbf8c96d5-c4xcb"]
Oct 31 01:30:35.310617 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:35.310599    4284 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-w4-a72754-openshift-default-6dbf8c96d5-c4xcb" podUID="e6cf830a-f9d2-4d0c-90de-672a81d3a441" containerName="istio-proxy" containerID="cri-o://92c2545939d3489e8348b50f70f67c0efcde7fb11fe267c5b4f7d95142315d1a" gracePeriod=30
Oct 31 01:30:35.855301 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:35.855260    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:30:37.084886 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:37.084840    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:30:40.562070 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:40.562045    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w4-a72754-openshift-default-6dbf8c96d5-c4xcb"
Oct 31 01:30:40.677685 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:40.677658    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e6cf830a-f9d2-4d0c-90de-672a81d3a441-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e6cf830a-f9d2-4d0c-90de-672a81d3a441-credential-socket\") pod \"e6cf830a-f9d2-4d0c-90de-672a81d3a441\" (UID: \"e6cf830a-f9d2-4d0c-90de-672a81d3a441\") "
Oct 31 01:30:40.677866 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:40.677693    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e6cf830a-f9d2-4d0c-90de-672a81d3a441-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e6cf830a-f9d2-4d0c-90de-672a81d3a441-workload-certs\") pod \"e6cf830a-f9d2-4d0c-90de-672a81d3a441\" (UID: \"e6cf830a-f9d2-4d0c-90de-672a81d3a441\") "
Oct 31 01:30:40.677866 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:40.677717    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e6cf830a-f9d2-4d0c-90de-672a81d3a441-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e6cf830a-f9d2-4d0c-90de-672a81d3a441-workload-socket\") pod \"e6cf830a-f9d2-4d0c-90de-672a81d3a441\" (UID: \"e6cf830a-f9d2-4d0c-90de-672a81d3a441\") "
Oct 31 01:30:40.677866 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:40.677743    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e6cf830a-f9d2-4d0c-90de-672a81d3a441-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e6cf830a-f9d2-4d0c-90de-672a81d3a441-istio-envoy\") pod \"e6cf830a-f9d2-4d0c-90de-672a81d3a441\" (UID: \"e6cf830a-f9d2-4d0c-90de-672a81d3a441\") "
Oct 31 01:30:40.677866 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:40.677775    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e6cf830a-f9d2-4d0c-90de-672a81d3a441-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e6cf830a-f9d2-4d0c-90de-672a81d3a441-istio-data\") pod \"e6cf830a-f9d2-4d0c-90de-672a81d3a441\" (UID: \"e6cf830a-f9d2-4d0c-90de-672a81d3a441\") "
Oct 31 01:30:40.677866 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:40.677814    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/e6cf830a-f9d2-4d0c-90de-672a81d3a441-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e6cf830a-f9d2-4d0c-90de-672a81d3a441-istio-podinfo\") pod \"e6cf830a-f9d2-4d0c-90de-672a81d3a441\" (UID: \"e6cf830a-f9d2-4d0c-90de-672a81d3a441\") "
Oct 31 01:30:40.678146 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:40.677928    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e6cf830a-f9d2-4d0c-90de-672a81d3a441-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e6cf830a-f9d2-4d0c-90de-672a81d3a441-istiod-ca-cert\") pod \"e6cf830a-f9d2-4d0c-90de-672a81d3a441\" (UID: \"e6cf830a-f9d2-4d0c-90de-672a81d3a441\") "
Oct 31 01:30:40.678146 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:40.677982    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e6cf830a-f9d2-4d0c-90de-672a81d3a441-kube-api-access-svtpt\" (UniqueName: \"kubernetes.io/projected/e6cf830a-f9d2-4d0c-90de-672a81d3a441-kube-api-access-svtpt\") pod \"e6cf830a-f9d2-4d0c-90de-672a81d3a441\" (UID: \"e6cf830a-f9d2-4d0c-90de-672a81d3a441\") "
Oct 31 01:30:40.678146 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:40.678012    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e6cf830a-f9d2-4d0c-90de-672a81d3a441-istio-token\" (UniqueName: \"kubernetes.io/projected/e6cf830a-f9d2-4d0c-90de-672a81d3a441-istio-token\") pod \"e6cf830a-f9d2-4d0c-90de-672a81d3a441\" (UID: \"e6cf830a-f9d2-4d0c-90de-672a81d3a441\") "
Oct 31 01:30:40.678356 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:40.677986    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e6cf830a-f9d2-4d0c-90de-672a81d3a441-credential-socket" pod "e6cf830a-f9d2-4d0c-90de-672a81d3a441" (UID: "e6cf830a-f9d2-4d0c-90de-672a81d3a441"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:30:40.678356 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:40.678029    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e6cf830a-f9d2-4d0c-90de-672a81d3a441-workload-certs" pod "e6cf830a-f9d2-4d0c-90de-672a81d3a441" (UID: "e6cf830a-f9d2-4d0c-90de-672a81d3a441"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:30:40.678356 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:40.678065    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e6cf830a-f9d2-4d0c-90de-672a81d3a441-workload-socket" pod "e6cf830a-f9d2-4d0c-90de-672a81d3a441" (UID: "e6cf830a-f9d2-4d0c-90de-672a81d3a441"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:30:40.678356 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:40.678152    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e6cf830a-f9d2-4d0c-90de-672a81d3a441-istio-data" pod "e6cf830a-f9d2-4d0c-90de-672a81d3a441" (UID: "e6cf830a-f9d2-4d0c-90de-672a81d3a441"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:30:40.678560 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:40.678353    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e6cf830a-f9d2-4d0c-90de-672a81d3a441-istiod-ca-cert" pod "e6cf830a-f9d2-4d0c-90de-672a81d3a441" (UID: "e6cf830a-f9d2-4d0c-90de-672a81d3a441"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 01:30:40.678613 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:40.678577    4284 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e6cf830a-f9d2-4d0c-90de-672a81d3a441-istiod-ca-cert\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:30:40.678613 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:40.678600    4284 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e6cf830a-f9d2-4d0c-90de-672a81d3a441-credential-socket\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:30:40.678718 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:40.678615    4284 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e6cf830a-f9d2-4d0c-90de-672a81d3a441-workload-certs\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:30:40.678718 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:40.678630    4284 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e6cf830a-f9d2-4d0c-90de-672a81d3a441-workload-socket\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:30:40.678718 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:40.678645    4284 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e6cf830a-f9d2-4d0c-90de-672a81d3a441-istio-data\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:30:40.680723 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:40.680698    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e6cf830a-f9d2-4d0c-90de-672a81d3a441-istio-podinfo" pod "e6cf830a-f9d2-4d0c-90de-672a81d3a441" (UID: "e6cf830a-f9d2-4d0c-90de-672a81d3a441"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 01:30:40.680837 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:40.680719    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e6cf830a-f9d2-4d0c-90de-672a81d3a441-kube-api-access-svtpt" pod "e6cf830a-f9d2-4d0c-90de-672a81d3a441" (UID: "e6cf830a-f9d2-4d0c-90de-672a81d3a441"). InnerVolumeSpecName "kube-api-access-svtpt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:30:40.680837 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:40.680760    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e6cf830a-f9d2-4d0c-90de-672a81d3a441-istio-token" pod "e6cf830a-f9d2-4d0c-90de-672a81d3a441" (UID: "e6cf830a-f9d2-4d0c-90de-672a81d3a441"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:30:40.680918 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:40.680843    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e6cf830a-f9d2-4d0c-90de-672a81d3a441-istio-envoy" pod "e6cf830a-f9d2-4d0c-90de-672a81d3a441" (UID: "e6cf830a-f9d2-4d0c-90de-672a81d3a441"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:30:40.779654 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:40.779619    4284 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-svtpt\" (UniqueName: \"kubernetes.io/projected/e6cf830a-f9d2-4d0c-90de-672a81d3a441-kube-api-access-svtpt\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:30:40.779654 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:40.779650    4284 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e6cf830a-f9d2-4d0c-90de-672a81d3a441-istio-token\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:30:40.779654 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:40.779660    4284 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e6cf830a-f9d2-4d0c-90de-672a81d3a441-istio-envoy\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:30:40.779906 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:40.779671    4284 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e6cf830a-f9d2-4d0c-90de-672a81d3a441-istio-podinfo\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:30:40.793922 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:40.793892    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-ef2572/distinct-2-w4-ef2572-kserve-547fccbd5-dfpfx"]
Oct 31 01:30:40.964206 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:40.964118    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:30:41.049493 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:41.049458    4284 generic.go:356] "Generic (PLEG): container finished" podID="e6cf830a-f9d2-4d0c-90de-672a81d3a441" containerID="92c2545939d3489e8348b50f70f67c0efcde7fb11fe267c5b4f7d95142315d1a" exitCode=0
Oct 31 01:30:41.049688 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:41.049514    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w4-a72754-openshift-default-6dbf8c96d5-c4xcb"
Oct 31 01:30:41.049688 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:41.049544    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w4-a72754-openshift-default-6dbf8c96d5-c4xcb" event={"ID":"e6cf830a-f9d2-4d0c-90de-672a81d3a441","Type":"ContainerDied","Data":"92c2545939d3489e8348b50f70f67c0efcde7fb11fe267c5b4f7d95142315d1a"}
Oct 31 01:30:41.049688 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:41.049602    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w4-a72754-openshift-default-6dbf8c96d5-c4xcb" event={"ID":"e6cf830a-f9d2-4d0c-90de-672a81d3a441","Type":"ContainerDied","Data":"bc61762bfb713cbd0c1c9e53988d1065519ef17087c591c15f850bc6ff85e1a8"}
Oct 31 01:30:41.049688 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:41.049625    4284 scope.go:122] "RemoveContainer" containerID="92c2545939d3489e8348b50f70f67c0efcde7fb11fe267c5b4f7d95142315d1a"
Oct 31 01:30:41.059441 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:41.059416    4284 scope.go:122] "RemoveContainer" containerID="f159ec40f54ddaa13f0ed4da6afe2d0b0843c937e7b42b9fb4f37a1b49fed326"
Oct 31 01:30:41.067944 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:41.067924    4284 scope.go:122] "RemoveContainer" containerID="92c2545939d3489e8348b50f70f67c0efcde7fb11fe267c5b4f7d95142315d1a"
Oct 31 01:30:41.068263 ip-10-0-132-173 kubenswrapper[4284]: E1031 01:30:41.068243    4284 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"92c2545939d3489e8348b50f70f67c0efcde7fb11fe267c5b4f7d95142315d1a\": container with ID starting with 92c2545939d3489e8348b50f70f67c0efcde7fb11fe267c5b4f7d95142315d1a not found: ID does not exist" containerID="92c2545939d3489e8348b50f70f67c0efcde7fb11fe267c5b4f7d95142315d1a"
Oct 31 01:30:41.068330 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:41.068271    4284 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"92c2545939d3489e8348b50f70f67c0efcde7fb11fe267c5b4f7d95142315d1a"} err="failed to get container status \"92c2545939d3489e8348b50f70f67c0efcde7fb11fe267c5b4f7d95142315d1a\": rpc error: code = NotFound desc = could not find container \"92c2545939d3489e8348b50f70f67c0efcde7fb11fe267c5b4f7d95142315d1a\": container with ID starting with 92c2545939d3489e8348b50f70f67c0efcde7fb11fe267c5b4f7d95142315d1a not found: ID does not exist"
Oct 31 01:30:41.068330 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:41.068288    4284 scope.go:122] "RemoveContainer" containerID="f159ec40f54ddaa13f0ed4da6afe2d0b0843c937e7b42b9fb4f37a1b49fed326"
Oct 31 01:30:41.068788 ip-10-0-132-173 kubenswrapper[4284]: E1031 01:30:41.068769    4284 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f159ec40f54ddaa13f0ed4da6afe2d0b0843c937e7b42b9fb4f37a1b49fed326\": container with ID starting with f159ec40f54ddaa13f0ed4da6afe2d0b0843c937e7b42b9fb4f37a1b49fed326 not found: ID does not exist" containerID="f159ec40f54ddaa13f0ed4da6afe2d0b0843c937e7b42b9fb4f37a1b49fed326"
Oct 31 01:30:41.068847 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:41.068797    4284 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f159ec40f54ddaa13f0ed4da6afe2d0b0843c937e7b42b9fb4f37a1b49fed326"} err="failed to get container status \"f159ec40f54ddaa13f0ed4da6afe2d0b0843c937e7b42b9fb4f37a1b49fed326\": rpc error: code = NotFound desc = could not find container \"f159ec40f54ddaa13f0ed4da6afe2d0b0843c937e7b42b9fb4f37a1b49fed326\": container with ID starting with f159ec40f54ddaa13f0ed4da6afe2d0b0843c937e7b42b9fb4f37a1b49fed326 not found: ID does not exist"
Oct 31 01:30:41.208823 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:41.208792    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w4-a72754-openshift-default-6dbf8c96d5-c4xcb"]
Oct 31 01:30:41.214909 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:41.214851    4284 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-w4-a72754-openshift-default-6dbf8c96d5-c4xcb"]
Oct 31 01:30:41.965129 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:41.965085    4284 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e6cf830a-f9d2-4d0c-90de-672a81d3a441" path="/var/lib/kubelet/pods/e6cf830a-f9d2-4d0c-90de-672a81d3a441/volumes"
Oct 31 01:30:42.196353 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:42.196314    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:30:46.382244 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:46.382199    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w2-29ece5-6489c5b684-f79l9"]
Oct 31 01:30:46.382778 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:46.382475    4284 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-29ece5-6489c5b684-f79l9" podUID="b56df4cb-7bf2-4e02-ab77-f33e078aba3c" containerName="maas-api" containerID="cri-o://8356bb6bc1994c667b1e13a295044df1029916b59c67ce92cd60fde76552456b" gracePeriod=30
Oct 31 01:30:46.469947 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:46.469898    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:30:46.748288 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:46.747958    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-29ece5-6489c5b684-f79l9"
Oct 31 01:30:46.759546 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:46.758545    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b56df4cb-7bf2-4e02-ab77-f33e078aba3c-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b56df4cb-7bf2-4e02-ab77-f33e078aba3c-openshift-service-ca\") pod \"b56df4cb-7bf2-4e02-ab77-f33e078aba3c\" (UID: \"b56df4cb-7bf2-4e02-ab77-f33e078aba3c\") "
Oct 31 01:30:46.759546 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:46.758615    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b56df4cb-7bf2-4e02-ab77-f33e078aba3c-kube-api-access-fxktz\" (UniqueName: \"kubernetes.io/projected/b56df4cb-7bf2-4e02-ab77-f33e078aba3c-kube-api-access-fxktz\") pod \"b56df4cb-7bf2-4e02-ab77-f33e078aba3c\" (UID: \"b56df4cb-7bf2-4e02-ab77-f33e078aba3c\") "
Oct 31 01:30:46.759546 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:46.758794    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/b56df4cb-7bf2-4e02-ab77-f33e078aba3c-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b56df4cb-7bf2-4e02-ab77-f33e078aba3c-maas-api-tls\") pod \"b56df4cb-7bf2-4e02-ab77-f33e078aba3c\" (UID: \"b56df4cb-7bf2-4e02-ab77-f33e078aba3c\") "
Oct 31 01:30:46.759546 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:46.758826    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/b56df4cb-7bf2-4e02-ab77-f33e078aba3c-metrics-certs\" (UniqueName: \"kubernetes.io/secret/b56df4cb-7bf2-4e02-ab77-f33e078aba3c-metrics-certs\") pod \"b56df4cb-7bf2-4e02-ab77-f33e078aba3c\" (UID: \"b56df4cb-7bf2-4e02-ab77-f33e078aba3c\") "
Oct 31 01:30:46.759546 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:46.759097    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b56df4cb-7bf2-4e02-ab77-f33e078aba3c-openshift-service-ca" pod "b56df4cb-7bf2-4e02-ab77-f33e078aba3c" (UID: "b56df4cb-7bf2-4e02-ab77-f33e078aba3c"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 01:30:46.759546 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:46.759273    4284 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b56df4cb-7bf2-4e02-ab77-f33e078aba3c-openshift-service-ca\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:30:46.765273 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:46.763823    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b56df4cb-7bf2-4e02-ab77-f33e078aba3c-maas-api-tls" pod "b56df4cb-7bf2-4e02-ab77-f33e078aba3c" (UID: "b56df4cb-7bf2-4e02-ab77-f33e078aba3c"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 01:30:46.769575 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:46.769548    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b56df4cb-7bf2-4e02-ab77-f33e078aba3c-metrics-certs" pod "b56df4cb-7bf2-4e02-ab77-f33e078aba3c" (UID: "b56df4cb-7bf2-4e02-ab77-f33e078aba3c"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 01:30:46.769870 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:46.769850    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b56df4cb-7bf2-4e02-ab77-f33e078aba3c-kube-api-access-fxktz" pod "b56df4cb-7bf2-4e02-ab77-f33e078aba3c" (UID: "b56df4cb-7bf2-4e02-ab77-f33e078aba3c"). InnerVolumeSpecName "kube-api-access-fxktz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:30:46.860401 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:46.860371    4284 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b56df4cb-7bf2-4e02-ab77-f33e078aba3c-maas-api-tls\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:30:46.860401 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:46.860398    4284 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b56df4cb-7bf2-4e02-ab77-f33e078aba3c-metrics-certs\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:30:46.860612 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:46.860409    4284 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fxktz\" (UniqueName: \"kubernetes.io/projected/b56df4cb-7bf2-4e02-ab77-f33e078aba3c-kube-api-access-fxktz\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:30:47.039531 ip-10-0-132-173 kubenswrapper[4284]: E1031 01:30:47.039431    4284 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fd8sp" containerName="extractor"
Oct 31 01:30:47.082229 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:47.082176    4284 generic.go:356] "Generic (PLEG): container finished" podID="b56df4cb-7bf2-4e02-ab77-f33e078aba3c" containerID="8356bb6bc1994c667b1e13a295044df1029916b59c67ce92cd60fde76552456b" exitCode=0
Oct 31 01:30:47.082407 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:47.082256    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-29ece5-6489c5b684-f79l9"
Oct 31 01:30:47.082407 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:47.082258    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-29ece5-6489c5b684-f79l9" event={"ID":"b56df4cb-7bf2-4e02-ab77-f33e078aba3c","Type":"ContainerDied","Data":"8356bb6bc1994c667b1e13a295044df1029916b59c67ce92cd60fde76552456b"}
Oct 31 01:30:47.082407 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:47.082376    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-29ece5-6489c5b684-f79l9" event={"ID":"b56df4cb-7bf2-4e02-ab77-f33e078aba3c","Type":"ContainerDied","Data":"66012d9e15a9a9eafa75da06ffe370cb071fc95e668434468eef7d45d1a0356f"}
Oct 31 01:30:47.082407 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:47.082405    4284 scope.go:122] "RemoveContainer" containerID="8356bb6bc1994c667b1e13a295044df1029916b59c67ce92cd60fde76552456b"
Oct 31 01:30:47.120038 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:47.120019    4284 scope.go:122] "RemoveContainer" containerID="8356bb6bc1994c667b1e13a295044df1029916b59c67ce92cd60fde76552456b"
Oct 31 01:30:47.120375 ip-10-0-132-173 kubenswrapper[4284]: E1031 01:30:47.120357    4284 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8356bb6bc1994c667b1e13a295044df1029916b59c67ce92cd60fde76552456b\": container with ID starting with 8356bb6bc1994c667b1e13a295044df1029916b59c67ce92cd60fde76552456b not found: ID does not exist" containerID="8356bb6bc1994c667b1e13a295044df1029916b59c67ce92cd60fde76552456b"
Oct 31 01:30:47.120439 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:47.120383    4284 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8356bb6bc1994c667b1e13a295044df1029916b59c67ce92cd60fde76552456b"} err="failed to get container status \"8356bb6bc1994c667b1e13a295044df1029916b59c67ce92cd60fde76552456b\": rpc error: code = NotFound desc = could not find container \"8356bb6bc1994c667b1e13a295044df1029916b59c67ce92cd60fde76552456b\": container with ID starting with 8356bb6bc1994c667b1e13a295044df1029916b59c67ce92cd60fde76552456b not found: ID does not exist"
Oct 31 01:30:47.253091 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:47.253059    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w2-29ece5-6489c5b684-f79l9"]
Oct 31 01:30:47.256829 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:47.256805    4284 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w2-29ece5-6489c5b684-f79l9"]
Oct 31 01:30:47.539530 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:47.539488    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w1-b63c24-8544f46fb-2qm2b"]
Oct 31 01:30:47.539950 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:47.539712    4284 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-b63c24-8544f46fb-2qm2b" podUID="b8c47f8b-d458-4669-9af4-95fd324dff37" containerName="maas-api" containerID="cri-o://f6cfd624fbfeb47c5f8efb01043161aaf58053495ffd354b49fac0ffdef9ceae" gracePeriod=30
Oct 31 01:30:47.640704 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:47.640670    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:30:47.863076 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:47.863049    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-b63c24-8544f46fb-2qm2b"
Oct 31 01:30:47.874523 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:47.874500    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/b8c47f8b-d458-4669-9af4-95fd324dff37-metrics-certs\" (UniqueName: \"kubernetes.io/secret/b8c47f8b-d458-4669-9af4-95fd324dff37-metrics-certs\") pod \"b8c47f8b-d458-4669-9af4-95fd324dff37\" (UID: \"b8c47f8b-d458-4669-9af4-95fd324dff37\") "
Oct 31 01:30:47.874663 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:47.874542    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b8c47f8b-d458-4669-9af4-95fd324dff37-kube-api-access-55bzz\" (UniqueName: \"kubernetes.io/projected/b8c47f8b-d458-4669-9af4-95fd324dff37-kube-api-access-55bzz\") pod \"b8c47f8b-d458-4669-9af4-95fd324dff37\" (UID: \"b8c47f8b-d458-4669-9af4-95fd324dff37\") "
Oct 31 01:30:47.874663 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:47.874641    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/b8c47f8b-d458-4669-9af4-95fd324dff37-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b8c47f8b-d458-4669-9af4-95fd324dff37-maas-api-tls\") pod \"b8c47f8b-d458-4669-9af4-95fd324dff37\" (UID: \"b8c47f8b-d458-4669-9af4-95fd324dff37\") "
Oct 31 01:30:47.874751 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:47.874666    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b8c47f8b-d458-4669-9af4-95fd324dff37-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b8c47f8b-d458-4669-9af4-95fd324dff37-openshift-service-ca\") pod \"b8c47f8b-d458-4669-9af4-95fd324dff37\" (UID: \"b8c47f8b-d458-4669-9af4-95fd324dff37\") "
Oct 31 01:30:47.875080 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:47.875056    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b8c47f8b-d458-4669-9af4-95fd324dff37-openshift-service-ca" pod "b8c47f8b-d458-4669-9af4-95fd324dff37" (UID: "b8c47f8b-d458-4669-9af4-95fd324dff37"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 01:30:47.876770 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:47.876741    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b8c47f8b-d458-4669-9af4-95fd324dff37-metrics-certs" pod "b8c47f8b-d458-4669-9af4-95fd324dff37" (UID: "b8c47f8b-d458-4669-9af4-95fd324dff37"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 01:30:47.876870 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:47.876845    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b8c47f8b-d458-4669-9af4-95fd324dff37-maas-api-tls" pod "b8c47f8b-d458-4669-9af4-95fd324dff37" (UID: "b8c47f8b-d458-4669-9af4-95fd324dff37"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 01:30:47.876992 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:47.876973    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b8c47f8b-d458-4669-9af4-95fd324dff37-kube-api-access-55bzz" pod "b8c47f8b-d458-4669-9af4-95fd324dff37" (UID: "b8c47f8b-d458-4669-9af4-95fd324dff37"). InnerVolumeSpecName "kube-api-access-55bzz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:30:47.964081 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:47.963910    4284 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b56df4cb-7bf2-4e02-ab77-f33e078aba3c" path="/var/lib/kubelet/pods/b56df4cb-7bf2-4e02-ab77-f33e078aba3c/volumes"
Oct 31 01:30:47.977144 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:47.977115    4284 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b8c47f8b-d458-4669-9af4-95fd324dff37-metrics-certs\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:30:47.977306 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:47.977169    4284 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-55bzz\" (UniqueName: \"kubernetes.io/projected/b8c47f8b-d458-4669-9af4-95fd324dff37-kube-api-access-55bzz\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:30:47.977306 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:47.977187    4284 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b8c47f8b-d458-4669-9af4-95fd324dff37-maas-api-tls\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:30:47.977306 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:47.977199    4284 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b8c47f8b-d458-4669-9af4-95fd324dff37-openshift-service-ca\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:30:48.000016 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:47.995670    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:30:48.114427 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:48.114350    4284 generic.go:356] "Generic (PLEG): container finished" podID="b8c47f8b-d458-4669-9af4-95fd324dff37" containerID="f6cfd624fbfeb47c5f8efb01043161aaf58053495ffd354b49fac0ffdef9ceae" exitCode=0
Oct 31 01:30:48.114584 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:48.114435    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-b63c24-8544f46fb-2qm2b" event={"ID":"b8c47f8b-d458-4669-9af4-95fd324dff37","Type":"ContainerDied","Data":"f6cfd624fbfeb47c5f8efb01043161aaf58053495ffd354b49fac0ffdef9ceae"}
Oct 31 01:30:48.114584 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:48.114474    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-b63c24-8544f46fb-2qm2b" event={"ID":"b8c47f8b-d458-4669-9af4-95fd324dff37","Type":"ContainerDied","Data":"1334f961521998acf799b953c7324001930c2f82f14678b234cf24371422d47a"}
Oct 31 01:30:48.114584 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:48.114498    4284 scope.go:122] "RemoveContainer" containerID="f6cfd624fbfeb47c5f8efb01043161aaf58053495ffd354b49fac0ffdef9ceae"
Oct 31 01:30:48.114702 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:48.114616    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-b63c24-8544f46fb-2qm2b"
Oct 31 01:30:48.144404 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:48.143791    4284 scope.go:122] "RemoveContainer" containerID="f6cfd624fbfeb47c5f8efb01043161aaf58053495ffd354b49fac0ffdef9ceae"
Oct 31 01:30:48.150768 ip-10-0-132-173 kubenswrapper[4284]: E1031 01:30:48.150721    4284 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f6cfd624fbfeb47c5f8efb01043161aaf58053495ffd354b49fac0ffdef9ceae\": container with ID starting with f6cfd624fbfeb47c5f8efb01043161aaf58053495ffd354b49fac0ffdef9ceae not found: ID does not exist" containerID="f6cfd624fbfeb47c5f8efb01043161aaf58053495ffd354b49fac0ffdef9ceae"
Oct 31 01:30:48.151024 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:48.150994    4284 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f6cfd624fbfeb47c5f8efb01043161aaf58053495ffd354b49fac0ffdef9ceae"} err="failed to get container status \"f6cfd624fbfeb47c5f8efb01043161aaf58053495ffd354b49fac0ffdef9ceae\": rpc error: code = NotFound desc = could not find container \"f6cfd624fbfeb47c5f8efb01043161aaf58053495ffd354b49fac0ffdef9ceae\": container with ID starting with f6cfd624fbfeb47c5f8efb01043161aaf58053495ffd354b49fac0ffdef9ceae not found: ID does not exist"
Oct 31 01:30:48.272749 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:48.272719    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w1-b63c24-8544f46fb-2qm2b"]
Oct 31 01:30:48.276000 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:48.275959    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:30:48.276802 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:48.276783    4284 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w1-b63c24-8544f46fb-2qm2b"]
Oct 31 01:30:48.423200 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:48.423173    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:30:48.938868 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:48.938832    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:30:49.173104 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:49.173073    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:30:49.452235 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:49.452177    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:30:49.608243 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:49.608122    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:30:49.963585 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:49.963556    4284 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b8c47f8b-d458-4669-9af4-95fd324dff37" path="/var/lib/kubelet/pods/b8c47f8b-d458-4669-9af4-95fd324dff37/volumes"
Oct 31 01:30:50.194206 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:50.194173    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:30:53.144359 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:53.144330    4284 generic.go:356] "Generic (PLEG): container finished" podID="eed2968a-68fc-44a9-beb1-669b189009de" containerID="b7868070546d71cabda47b5a4b65ad5a91663b9d0c698b49d9a5f99301ba3488" exitCode=0
Oct 31 01:30:53.144799 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:53.144406    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-b-6bd7c8-openshift-default-5b6745476f-7xkcx" event={"ID":"eed2968a-68fc-44a9-beb1-669b189009de","Type":"ContainerDied","Data":"b7868070546d71cabda47b5a4b65ad5a91663b9d0c698b49d9a5f99301ba3488"}
Oct 31 01:30:53.144799 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:53.144800    4284 scope.go:122] "RemoveContainer" containerID="b7868070546d71cabda47b5a4b65ad5a91663b9d0c698b49d9a5f99301ba3488"
Oct 31 01:30:53.146152 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:53.146127    4284 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 01:30:53.146238 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:53.146189    4284 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 01:30:53.146285 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:53.146238    4284 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 01:30:54.151907 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:54.151872    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-b-6bd7c8-openshift-default-5b6745476f-7xkcx" event={"ID":"eed2968a-68fc-44a9-beb1-669b189009de","Type":"ContainerStarted","Data":"f4ad1449fff35f1f32cc8c21ffae19f18afa7d5ddd74639c8945603da9210c18"}
Oct 31 01:30:54.237564 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:54.237514    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:30:54.354681 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:54.354647    4284 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-extmod-b-6bd7c8-openshift-default-5b6745476f-7xkcx"
Oct 31 01:30:54.355694 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:54.355674    4284 patch_prober.go:29] interesting pod/e2e-extmod-b-6bd7c8-openshift-default-5b6745476f-7xkcx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.189:15021/healthz/ready\": dial tcp 10.134.0.189:15021: connect: connection refused" start-of-body=
Oct 31 01:30:54.355756 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:54.355721    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-b-6bd7c8-openshift-default-5b6745476f-7xkcx" podUID="eed2968a-68fc-44a9-beb1-669b189009de" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.189:15021/healthz/ready\": dial tcp 10.134.0.189:15021: connect: connection refused"
Oct 31 01:30:54.416146 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:54.415849    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:30:54.640322 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:54.640285    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-extmod-b-6bd7c8-openshift-default-5b6745476f-7xkcx"
Oct 31 01:30:54.768460 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:54.768366    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:30:55.354561 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:55.354528    4284 patch_prober.go:29] interesting pod/e2e-extmod-b-6bd7c8-openshift-default-5b6745476f-7xkcx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.189:15021/healthz/ready\": dial tcp 10.134.0.189:15021: connect: connection refused" start-of-body=
Oct 31 01:30:55.355065 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:55.354597    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-b-6bd7c8-openshift-default-5b6745476f-7xkcx" podUID="eed2968a-68fc-44a9-beb1-669b189009de" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.189:15021/healthz/ready\": dial tcp 10.134.0.189:15021: connect: connection refused"
Oct 31 01:30:55.578706 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:55.578675    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:30:55.728105 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:55.728069    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:30:55.815039 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:55.815000    4284 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w4-ef2572/distinct-2-w4-ef2572-kserve-547fccbd5-dfpfx" podUID="cb311a47-264c-4b33-bc31-377b94b0c59b" containerName="main" containerID="cri-o://14a3d41fce71267b2a444815c61ddebf71404ef6978a96a322698da59788d0fb" gracePeriod=45
Oct 31 01:30:55.850607 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:55.850569    4284 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-ef2572/distinct-2-w4-ef2572-kserve-547fccbd5-dfpfx" podUID="cb311a47-264c-4b33-bc31-377b94b0c59b" containerName="main" probeResult="failure" output="Get \"https://10.134.0.73:8000/ready\": dial tcp 10.134.0.73:8000: connect: connection refused"
Oct 31 01:30:55.977977 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:55.977947    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-29ece5-openshift-default-6fcb86dbc-n2k5v"]
Oct 31 01:30:55.978230 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:55.978158    4284 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w2-29ece5-openshift-default-6fcb86dbc-n2k5v" podUID="5fd0d1e4-e8d2-43a1-ac72-1e16bb0cbaef" containerName="istio-proxy" containerID="cri-o://b09d4f9a585ccdaac072f6e7ad63aaa1349624667b785adb080e3d5788e8dc42" gracePeriod=30
Oct 31 01:30:56.015423 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:56.015227    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:30:56.164935 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:56.164907    4284 generic.go:356] "Generic (PLEG): container finished" podID="cb311a47-264c-4b33-bc31-377b94b0c59b" containerID="14a3d41fce71267b2a444815c61ddebf71404ef6978a96a322698da59788d0fb" exitCode=0
Oct 31 01:30:56.165107 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:56.164978    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-ef2572/distinct-2-w4-ef2572-kserve-547fccbd5-dfpfx" event={"ID":"cb311a47-264c-4b33-bc31-377b94b0c59b","Type":"ContainerDied","Data":"14a3d41fce71267b2a444815c61ddebf71404ef6978a96a322698da59788d0fb"}
Oct 31 01:30:56.165107 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:56.165025    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-ef2572/distinct-2-w4-ef2572-kserve-547fccbd5-dfpfx" event={"ID":"cb311a47-264c-4b33-bc31-377b94b0c59b","Type":"ContainerDied","Data":"7894e2d9db6c518058e322229d3e0fcaa4635fd510e2db3ec114a7225ed67bdb"}
Oct 31 01:30:56.165107 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:56.165042    4284 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7894e2d9db6c518058e322229d3e0fcaa4635fd510e2db3ec114a7225ed67bdb"
Oct 31 01:30:56.166375 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:56.166361    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-ef2572/distinct-2-w4-ef2572-kserve-547fccbd5-dfpfx"
Oct 31 01:30:56.242662 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:56.242585    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cb311a47-264c-4b33-bc31-377b94b0c59b-dshm\" (UniqueName: \"kubernetes.io/empty-dir/cb311a47-264c-4b33-bc31-377b94b0c59b-dshm\") pod \"cb311a47-264c-4b33-bc31-377b94b0c59b\" (UID: \"cb311a47-264c-4b33-bc31-377b94b0c59b\") "
Oct 31 01:30:56.242662 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:56.242652    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cb311a47-264c-4b33-bc31-377b94b0c59b-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cb311a47-264c-4b33-bc31-377b94b0c59b-model-cache\") pod \"cb311a47-264c-4b33-bc31-377b94b0c59b\" (UID: \"cb311a47-264c-4b33-bc31-377b94b0c59b\") "
Oct 31 01:30:56.242896 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:56.242675    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/cb311a47-264c-4b33-bc31-377b94b0c59b-tls-certs\" (UniqueName: \"kubernetes.io/secret/cb311a47-264c-4b33-bc31-377b94b0c59b-tls-certs\") pod \"cb311a47-264c-4b33-bc31-377b94b0c59b\" (UID: \"cb311a47-264c-4b33-bc31-377b94b0c59b\") "
Oct 31 01:30:56.242896 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:56.242760    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cb311a47-264c-4b33-bc31-377b94b0c59b-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cb311a47-264c-4b33-bc31-377b94b0c59b-tmp-dir\") pod \"cb311a47-264c-4b33-bc31-377b94b0c59b\" (UID: \"cb311a47-264c-4b33-bc31-377b94b0c59b\") "
Oct 31 01:30:56.242896 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:56.242783    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/cb311a47-264c-4b33-bc31-377b94b0c59b-kube-api-access-2qczm\" (UniqueName: \"kubernetes.io/projected/cb311a47-264c-4b33-bc31-377b94b0c59b-kube-api-access-2qczm\") pod \"cb311a47-264c-4b33-bc31-377b94b0c59b\" (UID: \"cb311a47-264c-4b33-bc31-377b94b0c59b\") "
Oct 31 01:30:56.242896 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:56.242855    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cb311a47-264c-4b33-bc31-377b94b0c59b-home\" (UniqueName: \"kubernetes.io/empty-dir/cb311a47-264c-4b33-bc31-377b94b0c59b-home\") pod \"cb311a47-264c-4b33-bc31-377b94b0c59b\" (UID: \"cb311a47-264c-4b33-bc31-377b94b0c59b\") "
Oct 31 01:30:56.243130 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:56.242913    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cb311a47-264c-4b33-bc31-377b94b0c59b-model-cache" pod "cb311a47-264c-4b33-bc31-377b94b0c59b" (UID: "cb311a47-264c-4b33-bc31-377b94b0c59b"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:30:56.243130 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:56.242972    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cb311a47-264c-4b33-bc31-377b94b0c59b-tmp-dir" pod "cb311a47-264c-4b33-bc31-377b94b0c59b" (UID: "cb311a47-264c-4b33-bc31-377b94b0c59b"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:30:56.243234 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:56.243139    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cb311a47-264c-4b33-bc31-377b94b0c59b-home" pod "cb311a47-264c-4b33-bc31-377b94b0c59b" (UID: "cb311a47-264c-4b33-bc31-377b94b0c59b"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:30:56.243332 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:56.243314    4284 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cb311a47-264c-4b33-bc31-377b94b0c59b-home\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:30:56.243372 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:56.243339    4284 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cb311a47-264c-4b33-bc31-377b94b0c59b-model-cache\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:30:56.243372 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:56.243355    4284 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cb311a47-264c-4b33-bc31-377b94b0c59b-tmp-dir\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:30:56.245033 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:56.245002    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cb311a47-264c-4b33-bc31-377b94b0c59b-dshm" pod "cb311a47-264c-4b33-bc31-377b94b0c59b" (UID: "cb311a47-264c-4b33-bc31-377b94b0c59b"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:30:56.245179 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:56.245050    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cb311a47-264c-4b33-bc31-377b94b0c59b-tls-certs" pod "cb311a47-264c-4b33-bc31-377b94b0c59b" (UID: "cb311a47-264c-4b33-bc31-377b94b0c59b"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 01:30:56.245338 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:56.245320    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cb311a47-264c-4b33-bc31-377b94b0c59b-kube-api-access-2qczm" pod "cb311a47-264c-4b33-bc31-377b94b0c59b" (UID: "cb311a47-264c-4b33-bc31-377b94b0c59b"). InnerVolumeSpecName "kube-api-access-2qczm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:30:56.348990 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:56.348959    4284 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cb311a47-264c-4b33-bc31-377b94b0c59b-dshm\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:30:56.348990 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:56.348991    4284 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cb311a47-264c-4b33-bc31-377b94b0c59b-tls-certs\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:30:56.349196 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:56.349006    4284 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2qczm\" (UniqueName: \"kubernetes.io/projected/cb311a47-264c-4b33-bc31-377b94b0c59b-kube-api-access-2qczm\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:30:56.353857 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:56.353826    4284 patch_prober.go:29] interesting pod/e2e-extmod-b-6bd7c8-openshift-default-5b6745476f-7xkcx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.189:15021/healthz/ready\": dial tcp 10.134.0.189:15021: connect: connection refused" start-of-body=
Oct 31 01:30:56.354001 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:56.353883    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-b-6bd7c8-openshift-default-5b6745476f-7xkcx" podUID="eed2968a-68fc-44a9-beb1-669b189009de" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.189:15021/healthz/ready\": dial tcp 10.134.0.189:15021: connect: connection refused"
Oct 31 01:30:57.131542 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:57.131504    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:30:57.169422 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:57.169391    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-ef2572/distinct-2-w4-ef2572-kserve-547fccbd5-dfpfx"
Oct 31 01:30:57.310089 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:57.310056    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:30:57.325832 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:57.325805    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-ef2572/distinct-2-w4-ef2572-kserve-547fccbd5-dfpfx"]
Oct 31 01:30:57.353727 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:57.353686    4284 patch_prober.go:29] interesting pod/e2e-extmod-b-6bd7c8-openshift-default-5b6745476f-7xkcx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.189:15021/healthz/ready\": dial tcp 10.134.0.189:15021: connect: connection refused" start-of-body=
Oct 31 01:30:57.353887 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:57.353741    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-b-6bd7c8-openshift-default-5b6745476f-7xkcx" podUID="eed2968a-68fc-44a9-beb1-669b189009de" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.189:15021/healthz/ready\": dial tcp 10.134.0.189:15021: connect: connection refused"
Oct 31 01:30:57.373175 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:57.373148    4284 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w4-ef2572/distinct-2-w4-ef2572-kserve-547fccbd5-dfpfx"]
Oct 31 01:30:57.480813 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:57.480780    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-b63c24-openshift-default-d8c64855f-6mgf7"]
Oct 31 01:30:57.481063 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:57.481037    4284 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w1-b63c24-openshift-default-d8c64855f-6mgf7" podUID="ccf1138e-2e92-4ab6-bfdd-9979f0b69893" containerName="istio-proxy" containerID="cri-o://c6896cb617d6c6bc4e67b08c0156fea421b127c0b5542bb89fb23b3d63f2643c" gracePeriod=30
Oct 31 01:30:57.521266 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:57.520384    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:30:57.961884 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:57.961849    4284 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="cb311a47-264c-4b33-bc31-377b94b0c59b" path="/var/lib/kubelet/pods/cb311a47-264c-4b33-bc31-377b94b0c59b/volumes"
Oct 31 01:30:58.354737 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:58.354651    4284 patch_prober.go:29] interesting pod/e2e-extmod-b-6bd7c8-openshift-default-5b6745476f-7xkcx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.189:15021/healthz/ready\": dial tcp 10.134.0.189:15021: connect: connection refused" start-of-body=
Oct 31 01:30:58.354737 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:58.354713    4284 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-b-6bd7c8-openshift-default-5b6745476f-7xkcx" podUID="eed2968a-68fc-44a9-beb1-669b189009de" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.189:15021/healthz/ready\": dial tcp 10.134.0.189:15021: connect: connection refused"
Oct 31 01:30:59.527639 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:59.527608    4284 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-extmod-b-6bd7c8-openshift-default-5b6745476f-7xkcx"
Oct 31 01:30:59.529189 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:59.529162    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-extmod-b-6bd7c8-openshift-default-5b6745476f-7xkcx"
Oct 31 01:30:59.882243 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:30:59.882133    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:31:01.193853 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:31:01.193821    4284 generic.go:356] "Generic (PLEG): container finished" podID="5fd0d1e4-e8d2-43a1-ac72-1e16bb0cbaef" containerID="b09d4f9a585ccdaac072f6e7ad63aaa1349624667b785adb080e3d5788e8dc42" exitCode=0
Oct 31 01:31:01.194283 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:31:01.193893    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-29ece5-openshift-default-6fcb86dbc-n2k5v" event={"ID":"5fd0d1e4-e8d2-43a1-ac72-1e16bb0cbaef","Type":"ContainerDied","Data":"b09d4f9a585ccdaac072f6e7ad63aaa1349624667b785adb080e3d5788e8dc42"}
Oct 31 01:31:01.232418 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:31:01.232392    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-29ece5-openshift-default-6fcb86dbc-n2k5v"
Oct 31 01:31:01.300202 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:31:01.300168    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5fd0d1e4-e8d2-43a1-ac72-1e16bb0cbaef-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5fd0d1e4-e8d2-43a1-ac72-1e16bb0cbaef-istio-envoy\") pod \"5fd0d1e4-e8d2-43a1-ac72-1e16bb0cbaef\" (UID: \"5fd0d1e4-e8d2-43a1-ac72-1e16bb0cbaef\") "
Oct 31 01:31:01.300403 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:31:01.300253    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5fd0d1e4-e8d2-43a1-ac72-1e16bb0cbaef-kube-api-access-cpr4j\" (UniqueName: \"kubernetes.io/projected/5fd0d1e4-e8d2-43a1-ac72-1e16bb0cbaef-kube-api-access-cpr4j\") pod \"5fd0d1e4-e8d2-43a1-ac72-1e16bb0cbaef\" (UID: \"5fd0d1e4-e8d2-43a1-ac72-1e16bb0cbaef\") "
Oct 31 01:31:01.300403 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:31:01.300277    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5fd0d1e4-e8d2-43a1-ac72-1e16bb0cbaef-istio-token\" (UniqueName: \"kubernetes.io/projected/5fd0d1e4-e8d2-43a1-ac72-1e16bb0cbaef-istio-token\") pod \"5fd0d1e4-e8d2-43a1-ac72-1e16bb0cbaef\" (UID: \"5fd0d1e4-e8d2-43a1-ac72-1e16bb0cbaef\") "
Oct 31 01:31:01.300403 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:31:01.300309    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5fd0d1e4-e8d2-43a1-ac72-1e16bb0cbaef-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5fd0d1e4-e8d2-43a1-ac72-1e16bb0cbaef-workload-socket\") pod \"5fd0d1e4-e8d2-43a1-ac72-1e16bb0cbaef\" (UID: \"5fd0d1e4-e8d2-43a1-ac72-1e16bb0cbaef\") "
Oct 31 01:31:01.300403 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:31:01.300344    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5fd0d1e4-e8d2-43a1-ac72-1e16bb0cbaef-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5fd0d1e4-e8d2-43a1-ac72-1e16bb0cbaef-workload-certs\") pod \"5fd0d1e4-e8d2-43a1-ac72-1e16bb0cbaef\" (UID: \"5fd0d1e4-e8d2-43a1-ac72-1e16bb0cbaef\") "
Oct 31 01:31:01.300403 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:31:01.300397    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/5fd0d1e4-e8d2-43a1-ac72-1e16bb0cbaef-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5fd0d1e4-e8d2-43a1-ac72-1e16bb0cbaef-istiod-ca-cert\") pod \"5fd0d1e4-e8d2-43a1-ac72-1e16bb0cbaef\" (UID: \"5fd0d1e4-e8d2-43a1-ac72-1e16bb0cbaef\") "
Oct 31 01:31:01.300669 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:31:01.300460    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/5fd0d1e4-e8d2-43a1-ac72-1e16bb0cbaef-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5fd0d1e4-e8d2-43a1-ac72-1e16bb0cbaef-istio-podinfo\") pod \"5fd0d1e4-e8d2-43a1-ac72-1e16bb0cbaef\" (UID: \"5fd0d1e4-e8d2-43a1-ac72-1e16bb0cbaef\") "
Oct 31 01:31:01.300669 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:31:01.300498    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5fd0d1e4-e8d2-43a1-ac72-1e16bb0cbaef-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5fd0d1e4-e8d2-43a1-ac72-1e16bb0cbaef-istio-data\") pod \"5fd0d1e4-e8d2-43a1-ac72-1e16bb0cbaef\" (UID: \"5fd0d1e4-e8d2-43a1-ac72-1e16bb0cbaef\") "
Oct 31 01:31:01.300669 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:31:01.300550    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5fd0d1e4-e8d2-43a1-ac72-1e16bb0cbaef-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5fd0d1e4-e8d2-43a1-ac72-1e16bb0cbaef-credential-socket\") pod \"5fd0d1e4-e8d2-43a1-ac72-1e16bb0cbaef\" (UID: \"5fd0d1e4-e8d2-43a1-ac72-1e16bb0cbaef\") "
Oct 31 01:31:01.300669 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:31:01.300593    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5fd0d1e4-e8d2-43a1-ac72-1e16bb0cbaef-workload-socket" pod "5fd0d1e4-e8d2-43a1-ac72-1e16bb0cbaef" (UID: "5fd0d1e4-e8d2-43a1-ac72-1e16bb0cbaef"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:31:01.300921 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:31:01.300781    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5fd0d1e4-e8d2-43a1-ac72-1e16bb0cbaef-istio-data" pod "5fd0d1e4-e8d2-43a1-ac72-1e16bb0cbaef" (UID: "5fd0d1e4-e8d2-43a1-ac72-1e16bb0cbaef"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:31:01.300988 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:31:01.300953    4284 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5fd0d1e4-e8d2-43a1-ac72-1e16bb0cbaef-workload-socket\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:31:01.300988 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:31:01.300973    4284 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5fd0d1e4-e8d2-43a1-ac72-1e16bb0cbaef-istio-data\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:31:01.301255 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:31:01.301164    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5fd0d1e4-e8d2-43a1-ac72-1e16bb0cbaef-credential-socket" pod "5fd0d1e4-e8d2-43a1-ac72-1e16bb0cbaef" (UID: "5fd0d1e4-e8d2-43a1-ac72-1e16bb0cbaef"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:31:01.301255 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:31:01.301189    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5fd0d1e4-e8d2-43a1-ac72-1e16bb0cbaef-workload-certs" pod "5fd0d1e4-e8d2-43a1-ac72-1e16bb0cbaef" (UID: "5fd0d1e4-e8d2-43a1-ac72-1e16bb0cbaef"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:31:01.301531 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:31:01.301510    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5fd0d1e4-e8d2-43a1-ac72-1e16bb0cbaef-istiod-ca-cert" pod "5fd0d1e4-e8d2-43a1-ac72-1e16bb0cbaef" (UID: "5fd0d1e4-e8d2-43a1-ac72-1e16bb0cbaef"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 01:31:01.302754 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:31:01.302724    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5fd0d1e4-e8d2-43a1-ac72-1e16bb0cbaef-kube-api-access-cpr4j" pod "5fd0d1e4-e8d2-43a1-ac72-1e16bb0cbaef" (UID: "5fd0d1e4-e8d2-43a1-ac72-1e16bb0cbaef"). InnerVolumeSpecName "kube-api-access-cpr4j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:31:01.303000 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:31:01.302978    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5fd0d1e4-e8d2-43a1-ac72-1e16bb0cbaef-istio-token" pod "5fd0d1e4-e8d2-43a1-ac72-1e16bb0cbaef" (UID: "5fd0d1e4-e8d2-43a1-ac72-1e16bb0cbaef"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:31:01.303095 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:31:01.303054    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5fd0d1e4-e8d2-43a1-ac72-1e16bb0cbaef-istio-envoy" pod "5fd0d1e4-e8d2-43a1-ac72-1e16bb0cbaef" (UID: "5fd0d1e4-e8d2-43a1-ac72-1e16bb0cbaef"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:31:01.303203 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:31:01.303180    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/5fd0d1e4-e8d2-43a1-ac72-1e16bb0cbaef-istio-podinfo" pod "5fd0d1e4-e8d2-43a1-ac72-1e16bb0cbaef" (UID: "5fd0d1e4-e8d2-43a1-ac72-1e16bb0cbaef"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 01:31:01.402485 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:31:01.402453    4284 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cpr4j\" (UniqueName: \"kubernetes.io/projected/5fd0d1e4-e8d2-43a1-ac72-1e16bb0cbaef-kube-api-access-cpr4j\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:31:01.402485 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:31:01.402484    4284 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5fd0d1e4-e8d2-43a1-ac72-1e16bb0cbaef-istio-token\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:31:01.402485 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:31:01.402493    4284 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5fd0d1e4-e8d2-43a1-ac72-1e16bb0cbaef-workload-certs\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:31:01.402775 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:31:01.402502    4284 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5fd0d1e4-e8d2-43a1-ac72-1e16bb0cbaef-istiod-ca-cert\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:31:01.402775 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:31:01.402512    4284 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5fd0d1e4-e8d2-43a1-ac72-1e16bb0cbaef-istio-podinfo\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:31:01.402775 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:31:01.402520    4284 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5fd0d1e4-e8d2-43a1-ac72-1e16bb0cbaef-credential-socket\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:31:01.402775 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:31:01.402530    4284 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5fd0d1e4-e8d2-43a1-ac72-1e16bb0cbaef-istio-envoy\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:31:02.201269 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:31:02.201238    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-29ece5-openshift-default-6fcb86dbc-n2k5v"
Oct 31 01:31:02.201693 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:31:02.201246    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-29ece5-openshift-default-6fcb86dbc-n2k5v" event={"ID":"5fd0d1e4-e8d2-43a1-ac72-1e16bb0cbaef","Type":"ContainerDied","Data":"7ad40f2794f42b49c0c46dcd930e27e4ab3b856016bbd0194f89c08483dcb83f"}
Oct 31 01:31:02.201693 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:31:02.201379    4284 scope.go:122] "RemoveContainer" containerID="b09d4f9a585ccdaac072f6e7ad63aaa1349624667b785adb080e3d5788e8dc42"
Oct 31 01:31:02.406936 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:31:02.406901    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-29ece5-openshift-default-6fcb86dbc-n2k5v"]
Oct 31 01:31:02.410806 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:31:02.410781    4284 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w2-29ece5-openshift-default-6fcb86dbc-n2k5v"]
Oct 31 01:31:02.735822 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:31:02.735794    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-b63c24-openshift-default-d8c64855f-6mgf7"
Oct 31 01:31:02.816943 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:31:02.816909    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ccf1138e-2e92-4ab6-bfdd-9979f0b69893-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ccf1138e-2e92-4ab6-bfdd-9979f0b69893-workload-socket\") pod \"ccf1138e-2e92-4ab6-bfdd-9979f0b69893\" (UID: \"ccf1138e-2e92-4ab6-bfdd-9979f0b69893\") "
Oct 31 01:31:02.817179 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:31:02.816986    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ccf1138e-2e92-4ab6-bfdd-9979f0b69893-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ccf1138e-2e92-4ab6-bfdd-9979f0b69893-istio-data\") pod \"ccf1138e-2e92-4ab6-bfdd-9979f0b69893\" (UID: \"ccf1138e-2e92-4ab6-bfdd-9979f0b69893\") "
Oct 31 01:31:02.817179 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:31:02.817032    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ccf1138e-2e92-4ab6-bfdd-9979f0b69893-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ccf1138e-2e92-4ab6-bfdd-9979f0b69893-istiod-ca-cert\") pod \"ccf1138e-2e92-4ab6-bfdd-9979f0b69893\" (UID: \"ccf1138e-2e92-4ab6-bfdd-9979f0b69893\") "
Oct 31 01:31:02.817179 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:31:02.817094    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ccf1138e-2e92-4ab6-bfdd-9979f0b69893-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ccf1138e-2e92-4ab6-bfdd-9979f0b69893-istio-envoy\") pod \"ccf1138e-2e92-4ab6-bfdd-9979f0b69893\" (UID: \"ccf1138e-2e92-4ab6-bfdd-9979f0b69893\") "
Oct 31 01:31:02.817179 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:31:02.817134    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ccf1138e-2e92-4ab6-bfdd-9979f0b69893-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ccf1138e-2e92-4ab6-bfdd-9979f0b69893-credential-socket\") pod \"ccf1138e-2e92-4ab6-bfdd-9979f0b69893\" (UID: \"ccf1138e-2e92-4ab6-bfdd-9979f0b69893\") "
Oct 31 01:31:02.817494 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:31:02.817203    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ccf1138e-2e92-4ab6-bfdd-9979f0b69893-kube-api-access-47zsm\" (UniqueName: \"kubernetes.io/projected/ccf1138e-2e92-4ab6-bfdd-9979f0b69893-kube-api-access-47zsm\") pod \"ccf1138e-2e92-4ab6-bfdd-9979f0b69893\" (UID: \"ccf1138e-2e92-4ab6-bfdd-9979f0b69893\") "
Oct 31 01:31:02.817494 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:31:02.817271    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ccf1138e-2e92-4ab6-bfdd-9979f0b69893-istio-token\" (UniqueName: \"kubernetes.io/projected/ccf1138e-2e92-4ab6-bfdd-9979f0b69893-istio-token\") pod \"ccf1138e-2e92-4ab6-bfdd-9979f0b69893\" (UID: \"ccf1138e-2e92-4ab6-bfdd-9979f0b69893\") "
Oct 31 01:31:02.817494 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:31:02.817416    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ccf1138e-2e92-4ab6-bfdd-9979f0b69893-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ccf1138e-2e92-4ab6-bfdd-9979f0b69893-workload-certs\") pod \"ccf1138e-2e92-4ab6-bfdd-9979f0b69893\" (UID: \"ccf1138e-2e92-4ab6-bfdd-9979f0b69893\") "
Oct 31 01:31:02.817494 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:31:02.817459    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/ccf1138e-2e92-4ab6-bfdd-9979f0b69893-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ccf1138e-2e92-4ab6-bfdd-9979f0b69893-istio-podinfo\") pod \"ccf1138e-2e92-4ab6-bfdd-9979f0b69893\" (UID: \"ccf1138e-2e92-4ab6-bfdd-9979f0b69893\") "
Oct 31 01:31:02.817745 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:31:02.817729    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ccf1138e-2e92-4ab6-bfdd-9979f0b69893-istiod-ca-cert" pod "ccf1138e-2e92-4ab6-bfdd-9979f0b69893" (UID: "ccf1138e-2e92-4ab6-bfdd-9979f0b69893"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 01:31:02.817820 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:31:02.817786    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ccf1138e-2e92-4ab6-bfdd-9979f0b69893-workload-socket" pod "ccf1138e-2e92-4ab6-bfdd-9979f0b69893" (UID: "ccf1138e-2e92-4ab6-bfdd-9979f0b69893"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:31:02.817883 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:31:02.817853    4284 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ccf1138e-2e92-4ab6-bfdd-9979f0b69893-istiod-ca-cert\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:31:02.817883 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:31:02.817859    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ccf1138e-2e92-4ab6-bfdd-9979f0b69893-istio-data" pod "ccf1138e-2e92-4ab6-bfdd-9979f0b69893" (UID: "ccf1138e-2e92-4ab6-bfdd-9979f0b69893"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:31:02.817993 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:31:02.817913    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ccf1138e-2e92-4ab6-bfdd-9979f0b69893-credential-socket" pod "ccf1138e-2e92-4ab6-bfdd-9979f0b69893" (UID: "ccf1138e-2e92-4ab6-bfdd-9979f0b69893"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:31:02.817993 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:31:02.817947    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ccf1138e-2e92-4ab6-bfdd-9979f0b69893-workload-certs" pod "ccf1138e-2e92-4ab6-bfdd-9979f0b69893" (UID: "ccf1138e-2e92-4ab6-bfdd-9979f0b69893"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:31:02.820130 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:31:02.820101    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ccf1138e-2e92-4ab6-bfdd-9979f0b69893-kube-api-access-47zsm" pod "ccf1138e-2e92-4ab6-bfdd-9979f0b69893" (UID: "ccf1138e-2e92-4ab6-bfdd-9979f0b69893"). InnerVolumeSpecName "kube-api-access-47zsm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:31:02.820130 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:31:02.820110    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ccf1138e-2e92-4ab6-bfdd-9979f0b69893-istio-envoy" pod "ccf1138e-2e92-4ab6-bfdd-9979f0b69893" (UID: "ccf1138e-2e92-4ab6-bfdd-9979f0b69893"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:31:02.820410 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:31:02.820387    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ccf1138e-2e92-4ab6-bfdd-9979f0b69893-istio-podinfo" pod "ccf1138e-2e92-4ab6-bfdd-9979f0b69893" (UID: "ccf1138e-2e92-4ab6-bfdd-9979f0b69893"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 01:31:02.820495 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:31:02.820440    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ccf1138e-2e92-4ab6-bfdd-9979f0b69893-istio-token" pod "ccf1138e-2e92-4ab6-bfdd-9979f0b69893" (UID: "ccf1138e-2e92-4ab6-bfdd-9979f0b69893"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:31:02.918445 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:31:02.918413    4284 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ccf1138e-2e92-4ab6-bfdd-9979f0b69893-workload-certs\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:31:02.918445 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:31:02.918442    4284 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ccf1138e-2e92-4ab6-bfdd-9979f0b69893-istio-podinfo\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:31:02.918445 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:31:02.918451    4284 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ccf1138e-2e92-4ab6-bfdd-9979f0b69893-workload-socket\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:31:02.918772 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:31:02.918460    4284 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ccf1138e-2e92-4ab6-bfdd-9979f0b69893-istio-data\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:31:02.918772 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:31:02.918471    4284 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ccf1138e-2e92-4ab6-bfdd-9979f0b69893-istio-envoy\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:31:02.918772 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:31:02.918479    4284 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ccf1138e-2e92-4ab6-bfdd-9979f0b69893-credential-socket\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:31:02.918772 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:31:02.918488    4284 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-47zsm\" (UniqueName: \"kubernetes.io/projected/ccf1138e-2e92-4ab6-bfdd-9979f0b69893-kube-api-access-47zsm\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:31:02.918772 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:31:02.918497    4284 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ccf1138e-2e92-4ab6-bfdd-9979f0b69893-istio-token\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:31:03.208595 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:31:03.208562    4284 generic.go:356] "Generic (PLEG): container finished" podID="ccf1138e-2e92-4ab6-bfdd-9979f0b69893" containerID="c6896cb617d6c6bc4e67b08c0156fea421b127c0b5542bb89fb23b3d63f2643c" exitCode=0
Oct 31 01:31:03.209022 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:31:03.208608    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-b63c24-openshift-default-d8c64855f-6mgf7"
Oct 31 01:31:03.209022 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:31:03.208646    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-b63c24-openshift-default-d8c64855f-6mgf7" event={"ID":"ccf1138e-2e92-4ab6-bfdd-9979f0b69893","Type":"ContainerDied","Data":"c6896cb617d6c6bc4e67b08c0156fea421b127c0b5542bb89fb23b3d63f2643c"}
Oct 31 01:31:03.209022 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:31:03.208702    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-b63c24-openshift-default-d8c64855f-6mgf7" event={"ID":"ccf1138e-2e92-4ab6-bfdd-9979f0b69893","Type":"ContainerDied","Data":"c2aef73553e57cfe4e5f206d1c110cbd918a746ac604fac8c08d49fba6c6f819"}
Oct 31 01:31:03.209022 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:31:03.208727    4284 scope.go:122] "RemoveContainer" containerID="c6896cb617d6c6bc4e67b08c0156fea421b127c0b5542bb89fb23b3d63f2643c"
Oct 31 01:31:03.218674 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:31:03.218655    4284 scope.go:122] "RemoveContainer" containerID="352c1902de2c418962324852d5b808029015706578d88f6a05bb07b1fe519795"
Oct 31 01:31:03.227592 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:31:03.227563    4284 scope.go:122] "RemoveContainer" containerID="c6896cb617d6c6bc4e67b08c0156fea421b127c0b5542bb89fb23b3d63f2643c"
Oct 31 01:31:03.227948 ip-10-0-132-173 kubenswrapper[4284]: E1031 01:31:03.227926    4284 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c6896cb617d6c6bc4e67b08c0156fea421b127c0b5542bb89fb23b3d63f2643c\": container with ID starting with c6896cb617d6c6bc4e67b08c0156fea421b127c0b5542bb89fb23b3d63f2643c not found: ID does not exist" containerID="c6896cb617d6c6bc4e67b08c0156fea421b127c0b5542bb89fb23b3d63f2643c"
Oct 31 01:31:03.228014 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:31:03.227957    4284 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c6896cb617d6c6bc4e67b08c0156fea421b127c0b5542bb89fb23b3d63f2643c"} err="failed to get container status \"c6896cb617d6c6bc4e67b08c0156fea421b127c0b5542bb89fb23b3d63f2643c\": rpc error: code = NotFound desc = could not find container \"c6896cb617d6c6bc4e67b08c0156fea421b127c0b5542bb89fb23b3d63f2643c\": container with ID starting with c6896cb617d6c6bc4e67b08c0156fea421b127c0b5542bb89fb23b3d63f2643c not found: ID does not exist"
Oct 31 01:31:03.228014 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:31:03.227976    4284 scope.go:122] "RemoveContainer" containerID="352c1902de2c418962324852d5b808029015706578d88f6a05bb07b1fe519795"
Oct 31 01:31:03.228270 ip-10-0-132-173 kubenswrapper[4284]: E1031 01:31:03.228252    4284 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"352c1902de2c418962324852d5b808029015706578d88f6a05bb07b1fe519795\": container with ID starting with 352c1902de2c418962324852d5b808029015706578d88f6a05bb07b1fe519795 not found: ID does not exist" containerID="352c1902de2c418962324852d5b808029015706578d88f6a05bb07b1fe519795"
Oct 31 01:31:03.228326 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:31:03.228278    4284 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"352c1902de2c418962324852d5b808029015706578d88f6a05bb07b1fe519795"} err="failed to get container status \"352c1902de2c418962324852d5b808029015706578d88f6a05bb07b1fe519795\": rpc error: code = NotFound desc = could not find container \"352c1902de2c418962324852d5b808029015706578d88f6a05bb07b1fe519795\": container with ID starting with 352c1902de2c418962324852d5b808029015706578d88f6a05bb07b1fe519795 not found: ID does not exist"
Oct 31 01:31:03.366389 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:31:03.366355    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-b63c24-openshift-default-d8c64855f-6mgf7"]
Oct 31 01:31:03.414034 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:31:03.413997    4284 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w1-b63c24-openshift-default-d8c64855f-6mgf7"]
Oct 31 01:31:03.560404 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:31:03.560314    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:31:03.962366 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:31:03.962325    4284 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5fd0d1e4-e8d2-43a1-ac72-1e16bb0cbaef" path="/var/lib/kubelet/pods/5fd0d1e4-e8d2-43a1-ac72-1e16bb0cbaef/volumes"
Oct 31 01:31:03.962850 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:31:03.962830    4284 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ccf1138e-2e92-4ab6-bfdd-9979f0b69893" path="/var/lib/kubelet/pods/ccf1138e-2e92-4ab6-bfdd-9979f0b69893/volumes"
Oct 31 01:31:04.796454 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:31:04.796417    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:31:04.989914 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:31:04.989869    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:31:05.134505 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:31:05.134408    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:31:05.302224 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:31:05.302188    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:31:05.647861 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:31:05.647826    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:31:05.830630 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:31:05.830595    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:31:05.974411 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:31:05.974357    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:31:06.771644 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:31:06.771607    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:31:06.920271 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:31:06.920241    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:31:07.356763 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:31:07.356727    4284 scope.go:122] "RemoveContainer" containerID="14a3d41fce71267b2a444815c61ddebf71404ef6978a96a322698da59788d0fb"
Oct 31 01:31:09.585805 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:31:09.585768    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:31:10.043948 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:31:10.043910    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:31:13.049447 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:31:13.049411    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:31:13.200508 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:31:13.200471    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:31:13.964866 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:31:13.964830    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:31:14.815482 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:31:14.815442    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:31:17.013334 ip-10-0-132-173 kubenswrapper[4284]: E1031 01:31:17.013290    4284 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fd8sp" containerName="extractor"
Oct 31 01:31:17.181410 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:31:17.181374    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:31:17.339789 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:31:17.339701    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:31:18.216886 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:31:18.216850    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:31:18.428384 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:31:18.428352    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:31:18.577968 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:31:18.577879    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:31:19.697720 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:31:19.697685    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:31:19.906499 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:31:19.906451    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:31:21.499270 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:31:21.499203    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:31:21.706655 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:31:21.706607    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:31:21.887138 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:31:21.887054    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:31:24.690828 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:31:24.690797    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:31:27.081847 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:31:27.081804    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:31:27.231520 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:31:27.231468    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:31:27.992109 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:31:27.992066    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:31:28.187644 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:31:28.187603    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:31:28.964370 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:31:28.964339    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:31:30.638489 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:31:30.638448    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:31:30.864951 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:31:30.864910    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:31:31.083231 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:31:31.083177    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:31:31.629696 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:31:31.629654    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:31:36.128506 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:31:36.128467    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:31:37.630394 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:31:37.630357    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:31:37.825735 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:31:37.825695    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:31:37.974584 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:31:37.974546    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:31:38.248909 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:31:38.248828    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:31:40.458276 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:31:40.458240    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:31:40.672740 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:31:40.672703    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:31:40.994151 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:31:40.994109    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:31:45.197154 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:31:45.197111    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:31:46.736778 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:31:46.736686    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:31:46.942982 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:31:46.942952    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:31:47.013504 ip-10-0-132-173 kubenswrapper[4284]: E1031 01:31:47.013421    4284 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fd8sp" containerName="extractor"
Oct 31 01:31:47.125398 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:31:47.125360    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:31:47.726423 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:31:47.726389    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:31:52.190996 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:31:52.190961    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:31:54.153827 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:31:54.153785    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:31:54.342643 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:31:54.342609    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:31:54.560636 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:31:54.560541    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-extmod-a-c61da9-openshift-default-76f5864d45-7gh6h"]
Oct 31 01:31:54.560861 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:31:54.560829    4284 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-extmod-a-c61da9-openshift-default-76f5864d45-7gh6h" podUID="164bb74b-a95d-4b77-97b6-f2ca2eb2cc09" containerName="istio-proxy" containerID="cri-o://c26f97643d9e04d41f19ba84e12bf4a369dd9a0b62d12c705f68f2decc1301d1" gracePeriod=30
Oct 31 01:31:54.639531 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:31:54.639476    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:31:57.185979 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:31:57.185943    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:31:59.817661 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:31:59.817633    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-a-c61da9-openshift-default-76f5864d45-7gh6h"
Oct 31 01:32:00.008705 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:32:00.008668    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/164bb74b-a95d-4b77-97b6-f2ca2eb2cc09-istio-token\" (UniqueName: \"kubernetes.io/projected/164bb74b-a95d-4b77-97b6-f2ca2eb2cc09-istio-token\") pod \"164bb74b-a95d-4b77-97b6-f2ca2eb2cc09\" (UID: \"164bb74b-a95d-4b77-97b6-f2ca2eb2cc09\") "
Oct 31 01:32:00.008904 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:32:00.008754    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/164bb74b-a95d-4b77-97b6-f2ca2eb2cc09-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/164bb74b-a95d-4b77-97b6-f2ca2eb2cc09-workload-certs\") pod \"164bb74b-a95d-4b77-97b6-f2ca2eb2cc09\" (UID: \"164bb74b-a95d-4b77-97b6-f2ca2eb2cc09\") "
Oct 31 01:32:00.008904 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:32:00.008811    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/164bb74b-a95d-4b77-97b6-f2ca2eb2cc09-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/164bb74b-a95d-4b77-97b6-f2ca2eb2cc09-istio-envoy\") pod \"164bb74b-a95d-4b77-97b6-f2ca2eb2cc09\" (UID: \"164bb74b-a95d-4b77-97b6-f2ca2eb2cc09\") "
Oct 31 01:32:00.008904 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:32:00.008872    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/164bb74b-a95d-4b77-97b6-f2ca2eb2cc09-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/164bb74b-a95d-4b77-97b6-f2ca2eb2cc09-credential-socket\") pod \"164bb74b-a95d-4b77-97b6-f2ca2eb2cc09\" (UID: \"164bb74b-a95d-4b77-97b6-f2ca2eb2cc09\") "
Oct 31 01:32:00.009087 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:32:00.008906    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/164bb74b-a95d-4b77-97b6-f2ca2eb2cc09-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/164bb74b-a95d-4b77-97b6-f2ca2eb2cc09-istiod-ca-cert\") pod \"164bb74b-a95d-4b77-97b6-f2ca2eb2cc09\" (UID: \"164bb74b-a95d-4b77-97b6-f2ca2eb2cc09\") "
Oct 31 01:32:00.009087 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:32:00.008946    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/164bb74b-a95d-4b77-97b6-f2ca2eb2cc09-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/164bb74b-a95d-4b77-97b6-f2ca2eb2cc09-istio-podinfo\") pod \"164bb74b-a95d-4b77-97b6-f2ca2eb2cc09\" (UID: \"164bb74b-a95d-4b77-97b6-f2ca2eb2cc09\") "
Oct 31 01:32:00.009087 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:32:00.008974    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/164bb74b-a95d-4b77-97b6-f2ca2eb2cc09-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/164bb74b-a95d-4b77-97b6-f2ca2eb2cc09-workload-socket\") pod \"164bb74b-a95d-4b77-97b6-f2ca2eb2cc09\" (UID: \"164bb74b-a95d-4b77-97b6-f2ca2eb2cc09\") "
Oct 31 01:32:00.009630 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:32:00.009598    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/164bb74b-a95d-4b77-97b6-f2ca2eb2cc09-istiod-ca-cert" pod "164bb74b-a95d-4b77-97b6-f2ca2eb2cc09" (UID: "164bb74b-a95d-4b77-97b6-f2ca2eb2cc09"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 01:32:00.009761 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:32:00.009685    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/164bb74b-a95d-4b77-97b6-f2ca2eb2cc09-credential-socket" pod "164bb74b-a95d-4b77-97b6-f2ca2eb2cc09" (UID: "164bb74b-a95d-4b77-97b6-f2ca2eb2cc09"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:32:00.009761 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:32:00.009730    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/164bb74b-a95d-4b77-97b6-f2ca2eb2cc09-workload-certs" pod "164bb74b-a95d-4b77-97b6-f2ca2eb2cc09" (UID: "164bb74b-a95d-4b77-97b6-f2ca2eb2cc09"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:32:00.010051 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:32:00.009769    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/164bb74b-a95d-4b77-97b6-f2ca2eb2cc09-workload-socket" pod "164bb74b-a95d-4b77-97b6-f2ca2eb2cc09" (UID: "164bb74b-a95d-4b77-97b6-f2ca2eb2cc09"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:32:00.010051 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:32:00.009864    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/164bb74b-a95d-4b77-97b6-f2ca2eb2cc09-kube-api-access-cdk2p\" (UniqueName: \"kubernetes.io/projected/164bb74b-a95d-4b77-97b6-f2ca2eb2cc09-kube-api-access-cdk2p\") pod \"164bb74b-a95d-4b77-97b6-f2ca2eb2cc09\" (UID: \"164bb74b-a95d-4b77-97b6-f2ca2eb2cc09\") "
Oct 31 01:32:00.010051 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:32:00.009933    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/164bb74b-a95d-4b77-97b6-f2ca2eb2cc09-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/164bb74b-a95d-4b77-97b6-f2ca2eb2cc09-istio-data\") pod \"164bb74b-a95d-4b77-97b6-f2ca2eb2cc09\" (UID: \"164bb74b-a95d-4b77-97b6-f2ca2eb2cc09\") "
Oct 31 01:32:00.019230 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:32:00.011125    4284 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/164bb74b-a95d-4b77-97b6-f2ca2eb2cc09-workload-certs\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:32:00.019230 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:32:00.011159    4284 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/164bb74b-a95d-4b77-97b6-f2ca2eb2cc09-credential-socket\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:32:00.019230 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:32:00.011180    4284 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/164bb74b-a95d-4b77-97b6-f2ca2eb2cc09-istiod-ca-cert\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:32:00.019230 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:32:00.011195    4284 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/164bb74b-a95d-4b77-97b6-f2ca2eb2cc09-workload-socket\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:32:00.019230 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:32:00.011444    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/164bb74b-a95d-4b77-97b6-f2ca2eb2cc09-istio-data" pod "164bb74b-a95d-4b77-97b6-f2ca2eb2cc09" (UID: "164bb74b-a95d-4b77-97b6-f2ca2eb2cc09"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:32:00.024498 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:32:00.024460    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/164bb74b-a95d-4b77-97b6-f2ca2eb2cc09-istio-podinfo" pod "164bb74b-a95d-4b77-97b6-f2ca2eb2cc09" (UID: "164bb74b-a95d-4b77-97b6-f2ca2eb2cc09"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 01:32:00.024639 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:32:00.024535    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/164bb74b-a95d-4b77-97b6-f2ca2eb2cc09-istio-envoy" pod "164bb74b-a95d-4b77-97b6-f2ca2eb2cc09" (UID: "164bb74b-a95d-4b77-97b6-f2ca2eb2cc09"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:32:00.026320 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:32:00.026288    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/164bb74b-a95d-4b77-97b6-f2ca2eb2cc09-kube-api-access-cdk2p" pod "164bb74b-a95d-4b77-97b6-f2ca2eb2cc09" (UID: "164bb74b-a95d-4b77-97b6-f2ca2eb2cc09"). InnerVolumeSpecName "kube-api-access-cdk2p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:32:00.026320 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:32:00.026297    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/164bb74b-a95d-4b77-97b6-f2ca2eb2cc09-istio-token" pod "164bb74b-a95d-4b77-97b6-f2ca2eb2cc09" (UID: "164bb74b-a95d-4b77-97b6-f2ca2eb2cc09"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:32:00.111957 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:32:00.111920    4284 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/164bb74b-a95d-4b77-97b6-f2ca2eb2cc09-istio-envoy\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:32:00.111957 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:32:00.111955    4284 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/164bb74b-a95d-4b77-97b6-f2ca2eb2cc09-istio-podinfo\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:32:00.111957 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:32:00.111972    4284 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cdk2p\" (UniqueName: \"kubernetes.io/projected/164bb74b-a95d-4b77-97b6-f2ca2eb2cc09-kube-api-access-cdk2p\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:32:00.112311 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:32:00.111987    4284 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/164bb74b-a95d-4b77-97b6-f2ca2eb2cc09-istio-data\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:32:00.112311 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:32:00.112001    4284 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/164bb74b-a95d-4b77-97b6-f2ca2eb2cc09-istio-token\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:32:00.503792 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:32:00.503760    4284 generic.go:356] "Generic (PLEG): container finished" podID="164bb74b-a95d-4b77-97b6-f2ca2eb2cc09" containerID="c26f97643d9e04d41f19ba84e12bf4a369dd9a0b62d12c705f68f2decc1301d1" exitCode=0
Oct 31 01:32:00.503992 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:32:00.503825    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-a-c61da9-openshift-default-76f5864d45-7gh6h"
Oct 31 01:32:00.503992 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:32:00.503850    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-c61da9-openshift-default-76f5864d45-7gh6h" event={"ID":"164bb74b-a95d-4b77-97b6-f2ca2eb2cc09","Type":"ContainerDied","Data":"c26f97643d9e04d41f19ba84e12bf4a369dd9a0b62d12c705f68f2decc1301d1"}
Oct 31 01:32:00.503992 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:32:00.503927    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-c61da9-openshift-default-76f5864d45-7gh6h" event={"ID":"164bb74b-a95d-4b77-97b6-f2ca2eb2cc09","Type":"ContainerDied","Data":"6c1d6853cdc348aafddac529f46c3d140849d16717917bd83a5229dacbdea95b"}
Oct 31 01:32:00.503992 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:32:00.503946    4284 scope.go:122] "RemoveContainer" containerID="c26f97643d9e04d41f19ba84e12bf4a369dd9a0b62d12c705f68f2decc1301d1"
Oct 31 01:32:00.513335 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:32:00.513318    4284 scope.go:122] "RemoveContainer" containerID="c26f97643d9e04d41f19ba84e12bf4a369dd9a0b62d12c705f68f2decc1301d1"
Oct 31 01:32:00.513593 ip-10-0-132-173 kubenswrapper[4284]: E1031 01:32:00.513572    4284 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c26f97643d9e04d41f19ba84e12bf4a369dd9a0b62d12c705f68f2decc1301d1\": container with ID starting with c26f97643d9e04d41f19ba84e12bf4a369dd9a0b62d12c705f68f2decc1301d1 not found: ID does not exist" containerID="c26f97643d9e04d41f19ba84e12bf4a369dd9a0b62d12c705f68f2decc1301d1"
Oct 31 01:32:00.513658 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:32:00.513601    4284 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c26f97643d9e04d41f19ba84e12bf4a369dd9a0b62d12c705f68f2decc1301d1"} err="failed to get container status \"c26f97643d9e04d41f19ba84e12bf4a369dd9a0b62d12c705f68f2decc1301d1\": rpc error: code = NotFound desc = could not find container \"c26f97643d9e04d41f19ba84e12bf4a369dd9a0b62d12c705f68f2decc1301d1\": container with ID starting with c26f97643d9e04d41f19ba84e12bf4a369dd9a0b62d12c705f68f2decc1301d1 not found: ID does not exist"
Oct 31 01:32:00.662774 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:32:00.662742    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-extmod-a-c61da9-openshift-default-76f5864d45-7gh6h"]
Oct 31 01:32:00.710384 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:32:00.710342    4284 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-extmod-a-c61da9-openshift-default-76f5864d45-7gh6h"]
Oct 31 01:32:01.962900 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:32:01.962866    4284 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="164bb74b-a95d-4b77-97b6-f2ca2eb2cc09" path="/var/lib/kubelet/pods/164bb74b-a95d-4b77-97b6-f2ca2eb2cc09/volumes"
Oct 31 01:32:02.212311 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:32:02.212277    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:32:07.093203 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:32:07.093163    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:32:07.500761 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:32:07.500719    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:32:07.690125 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:32:07.690033    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:32:10.379145 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:32:10.379078    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:32:11.893898 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:32:11.893866    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:32:16.153402 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:32:16.153364    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:32:17.013265 ip-10-0-132-173 kubenswrapper[4284]: E1031 01:32:17.013204    4284 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fd8sp" containerName="extractor"
Oct 31 01:32:17.675969 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:32:17.675931    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:32:17.898406 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:32:17.898373    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:32:18.046489 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:32:18.046412    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:32:18.665001 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:32:18.664965    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:32:22.969709 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:32:22.969670    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:32:25.284201 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:32:25.284168    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:32:25.440526 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:32:25.440489    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:32:25.828850 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:32:25.828816    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:32:25.836477 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:32:25.836446    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-extmod-b-6bd7c8-openshift-default-5b6745476f-7xkcx"]
Oct 31 01:32:25.836681 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:32:25.836659    4284 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-extmod-b-6bd7c8-openshift-default-5b6745476f-7xkcx" podUID="eed2968a-68fc-44a9-beb1-669b189009de" containerName="istio-proxy" containerID="cri-o://f4ad1449fff35f1f32cc8c21ffae19f18afa7d5ddd74639c8945603da9210c18" gracePeriod=30
Oct 31 01:32:28.061494 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:32:28.061433    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:32:29.530171 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:32:29.530136    4284 patch_prober.go:29] interesting pod/e2e-extmod-b-6bd7c8-openshift-default-5b6745476f-7xkcx container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 31 01:32:29.530577 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:32:29.530226    4284 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-extmod-b-6bd7c8-openshift-default-5b6745476f-7xkcx" podUID="eed2968a-68fc-44a9-beb1-669b189009de" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 31 01:32:31.083434 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:32:31.083406    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-b-6bd7c8-openshift-default-5b6745476f-7xkcx"
Oct 31 01:32:31.214939 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:32:31.214912    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/eed2968a-68fc-44a9-beb1-669b189009de-kube-api-access-674s4\" (UniqueName: \"kubernetes.io/projected/eed2968a-68fc-44a9-beb1-669b189009de-kube-api-access-674s4\") pod \"eed2968a-68fc-44a9-beb1-669b189009de\" (UID: \"eed2968a-68fc-44a9-beb1-669b189009de\") "
Oct 31 01:32:31.215115 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:32:31.214960    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/eed2968a-68fc-44a9-beb1-669b189009de-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eed2968a-68fc-44a9-beb1-669b189009de-workload-certs\") pod \"eed2968a-68fc-44a9-beb1-669b189009de\" (UID: \"eed2968a-68fc-44a9-beb1-669b189009de\") "
Oct 31 01:32:31.215115 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:32:31.215003    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/eed2968a-68fc-44a9-beb1-669b189009de-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eed2968a-68fc-44a9-beb1-669b189009de-workload-socket\") pod \"eed2968a-68fc-44a9-beb1-669b189009de\" (UID: \"eed2968a-68fc-44a9-beb1-669b189009de\") "
Oct 31 01:32:31.215115 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:32:31.215040    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/eed2968a-68fc-44a9-beb1-669b189009de-istio-token\" (UniqueName: \"kubernetes.io/projected/eed2968a-68fc-44a9-beb1-669b189009de-istio-token\") pod \"eed2968a-68fc-44a9-beb1-669b189009de\" (UID: \"eed2968a-68fc-44a9-beb1-669b189009de\") "
Oct 31 01:32:31.215115 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:32:31.215076    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/eed2968a-68fc-44a9-beb1-669b189009de-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eed2968a-68fc-44a9-beb1-669b189009de-istio-podinfo\") pod \"eed2968a-68fc-44a9-beb1-669b189009de\" (UID: \"eed2968a-68fc-44a9-beb1-669b189009de\") "
Oct 31 01:32:31.215360 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:32:31.215163    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/eed2968a-68fc-44a9-beb1-669b189009de-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eed2968a-68fc-44a9-beb1-669b189009de-credential-socket\") pod \"eed2968a-68fc-44a9-beb1-669b189009de\" (UID: \"eed2968a-68fc-44a9-beb1-669b189009de\") "
Oct 31 01:32:31.215360 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:32:31.215271    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/eed2968a-68fc-44a9-beb1-669b189009de-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eed2968a-68fc-44a9-beb1-669b189009de-istiod-ca-cert\") pod \"eed2968a-68fc-44a9-beb1-669b189009de\" (UID: \"eed2968a-68fc-44a9-beb1-669b189009de\") "
Oct 31 01:32:31.215482 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:32:31.215373    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/eed2968a-68fc-44a9-beb1-669b189009de-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eed2968a-68fc-44a9-beb1-669b189009de-istio-data\") pod \"eed2968a-68fc-44a9-beb1-669b189009de\" (UID: \"eed2968a-68fc-44a9-beb1-669b189009de\") "
Oct 31 01:32:31.215482 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:32:31.215373    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eed2968a-68fc-44a9-beb1-669b189009de-workload-socket" pod "eed2968a-68fc-44a9-beb1-669b189009de" (UID: "eed2968a-68fc-44a9-beb1-669b189009de"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:32:31.215482 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:32:31.215386    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eed2968a-68fc-44a9-beb1-669b189009de-workload-certs" pod "eed2968a-68fc-44a9-beb1-669b189009de" (UID: "eed2968a-68fc-44a9-beb1-669b189009de"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:32:31.215482 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:32:31.215420    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/eed2968a-68fc-44a9-beb1-669b189009de-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eed2968a-68fc-44a9-beb1-669b189009de-istio-envoy\") pod \"eed2968a-68fc-44a9-beb1-669b189009de\" (UID: \"eed2968a-68fc-44a9-beb1-669b189009de\") "
Oct 31 01:32:31.215720 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:32:31.215582    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eed2968a-68fc-44a9-beb1-669b189009de-credential-socket" pod "eed2968a-68fc-44a9-beb1-669b189009de" (UID: "eed2968a-68fc-44a9-beb1-669b189009de"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:32:31.215720 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:32:31.215680    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eed2968a-68fc-44a9-beb1-669b189009de-istio-data" pod "eed2968a-68fc-44a9-beb1-669b189009de" (UID: "eed2968a-68fc-44a9-beb1-669b189009de"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:32:31.216061 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:32:31.215925    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/eed2968a-68fc-44a9-beb1-669b189009de-istiod-ca-cert" pod "eed2968a-68fc-44a9-beb1-669b189009de" (UID: "eed2968a-68fc-44a9-beb1-669b189009de"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 01:32:31.216061 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:32:31.215990    4284 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eed2968a-68fc-44a9-beb1-669b189009de-workload-certs\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:32:31.216061 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:32:31.216015    4284 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eed2968a-68fc-44a9-beb1-669b189009de-workload-socket\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:32:31.216061 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:32:31.216030    4284 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eed2968a-68fc-44a9-beb1-669b189009de-credential-socket\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:32:31.216061 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:32:31.216045    4284 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eed2968a-68fc-44a9-beb1-669b189009de-istio-data\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:32:31.217452 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:32:31.217412    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eed2968a-68fc-44a9-beb1-669b189009de-kube-api-access-674s4" pod "eed2968a-68fc-44a9-beb1-669b189009de" (UID: "eed2968a-68fc-44a9-beb1-669b189009de"). InnerVolumeSpecName "kube-api-access-674s4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:32:31.217452 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:32:31.217418    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eed2968a-68fc-44a9-beb1-669b189009de-istio-token" pod "eed2968a-68fc-44a9-beb1-669b189009de" (UID: "eed2968a-68fc-44a9-beb1-669b189009de"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:32:31.217606 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:32:31.217492    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/eed2968a-68fc-44a9-beb1-669b189009de-istio-podinfo" pod "eed2968a-68fc-44a9-beb1-669b189009de" (UID: "eed2968a-68fc-44a9-beb1-669b189009de"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 01:32:31.217687 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:32:31.217673    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eed2968a-68fc-44a9-beb1-669b189009de-istio-envoy" pod "eed2968a-68fc-44a9-beb1-669b189009de" (UID: "eed2968a-68fc-44a9-beb1-669b189009de"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:32:31.317181 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:32:31.317149    4284 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-674s4\" (UniqueName: \"kubernetes.io/projected/eed2968a-68fc-44a9-beb1-669b189009de-kube-api-access-674s4\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:32:31.317181 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:32:31.317176    4284 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eed2968a-68fc-44a9-beb1-669b189009de-istio-token\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:32:31.317181 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:32:31.317186    4284 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eed2968a-68fc-44a9-beb1-669b189009de-istio-podinfo\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:32:31.317441 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:32:31.317195    4284 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eed2968a-68fc-44a9-beb1-669b189009de-istiod-ca-cert\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:32:31.317441 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:32:31.317203    4284 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eed2968a-68fc-44a9-beb1-669b189009de-istio-envoy\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:32:31.661499 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:32:31.661465    4284 generic.go:356] "Generic (PLEG): container finished" podID="eed2968a-68fc-44a9-beb1-669b189009de" containerID="f4ad1449fff35f1f32cc8c21ffae19f18afa7d5ddd74639c8945603da9210c18" exitCode=0
Oct 31 01:32:31.661697 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:32:31.661523    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-b-6bd7c8-openshift-default-5b6745476f-7xkcx" event={"ID":"eed2968a-68fc-44a9-beb1-669b189009de","Type":"ContainerDied","Data":"f4ad1449fff35f1f32cc8c21ffae19f18afa7d5ddd74639c8945603da9210c18"}
Oct 31 01:32:31.661697 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:32:31.661525    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-b-6bd7c8-openshift-default-5b6745476f-7xkcx"
Oct 31 01:32:31.661697 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:32:31.661566    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-b-6bd7c8-openshift-default-5b6745476f-7xkcx" event={"ID":"eed2968a-68fc-44a9-beb1-669b189009de","Type":"ContainerDied","Data":"683a0b6ca2d78692032147204521d1f63a780caff4e25859ce6e67101ed8686c"}
Oct 31 01:32:31.661697 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:32:31.661587    4284 scope.go:122] "RemoveContainer" containerID="f4ad1449fff35f1f32cc8c21ffae19f18afa7d5ddd74639c8945603da9210c18"
Oct 31 01:32:31.670954 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:32:31.670936    4284 scope.go:122] "RemoveContainer" containerID="b7868070546d71cabda47b5a4b65ad5a91663b9d0c698b49d9a5f99301ba3488"
Oct 31 01:32:31.678973 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:32:31.678958    4284 scope.go:122] "RemoveContainer" containerID="f4ad1449fff35f1f32cc8c21ffae19f18afa7d5ddd74639c8945603da9210c18"
Oct 31 01:32:31.679243 ip-10-0-132-173 kubenswrapper[4284]: E1031 01:32:31.679226    4284 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f4ad1449fff35f1f32cc8c21ffae19f18afa7d5ddd74639c8945603da9210c18\": container with ID starting with f4ad1449fff35f1f32cc8c21ffae19f18afa7d5ddd74639c8945603da9210c18 not found: ID does not exist" containerID="f4ad1449fff35f1f32cc8c21ffae19f18afa7d5ddd74639c8945603da9210c18"
Oct 31 01:32:31.679301 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:32:31.679251    4284 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f4ad1449fff35f1f32cc8c21ffae19f18afa7d5ddd74639c8945603da9210c18"} err="failed to get container status \"f4ad1449fff35f1f32cc8c21ffae19f18afa7d5ddd74639c8945603da9210c18\": rpc error: code = NotFound desc = could not find container \"f4ad1449fff35f1f32cc8c21ffae19f18afa7d5ddd74639c8945603da9210c18\": container with ID starting with f4ad1449fff35f1f32cc8c21ffae19f18afa7d5ddd74639c8945603da9210c18 not found: ID does not exist"
Oct 31 01:32:31.679301 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:32:31.679268    4284 scope.go:122] "RemoveContainer" containerID="b7868070546d71cabda47b5a4b65ad5a91663b9d0c698b49d9a5f99301ba3488"
Oct 31 01:32:31.679533 ip-10-0-132-173 kubenswrapper[4284]: E1031 01:32:31.679516    4284 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b7868070546d71cabda47b5a4b65ad5a91663b9d0c698b49d9a5f99301ba3488\": container with ID starting with b7868070546d71cabda47b5a4b65ad5a91663b9d0c698b49d9a5f99301ba3488 not found: ID does not exist" containerID="b7868070546d71cabda47b5a4b65ad5a91663b9d0c698b49d9a5f99301ba3488"
Oct 31 01:32:31.679585 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:32:31.679537    4284 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b7868070546d71cabda47b5a4b65ad5a91663b9d0c698b49d9a5f99301ba3488"} err="failed to get container status \"b7868070546d71cabda47b5a4b65ad5a91663b9d0c698b49d9a5f99301ba3488\": rpc error: code = NotFound desc = could not find container \"b7868070546d71cabda47b5a4b65ad5a91663b9d0c698b49d9a5f99301ba3488\": container with ID starting with b7868070546d71cabda47b5a4b65ad5a91663b9d0c698b49d9a5f99301ba3488 not found: ID does not exist"
Oct 31 01:32:31.820857 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:32:31.820817    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-extmod-b-6bd7c8-openshift-default-5b6745476f-7xkcx"]
Oct 31 01:32:31.826946 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:32:31.826925    4284 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-extmod-b-6bd7c8-openshift-default-5b6745476f-7xkcx"]
Oct 31 01:32:31.963066 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:32:31.962977    4284 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="eed2968a-68fc-44a9-beb1-669b189009de" path="/var/lib/kubelet/pods/eed2968a-68fc-44a9-beb1-669b189009de/volumes"
Oct 31 01:32:33.102862 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:32:33.102826    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:32:36.073434 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:32:36.073397    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:32:36.221899 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:32:36.221853    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:32:37.232404 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:32:37.232373    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-cb9a07-5877444fcd-d6mg2"]
Oct 31 01:32:37.232850 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:32:37.232570    4284 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-cb9a07-5877444fcd-d6mg2" podUID="b8db1668-6ee8-4d2e-b531-73300b46d926" containerName="maas-api" containerID="cri-o://b7e492fcafcb503afb146c68f7604163908ebd5efa0e350f2d56094b4169bc26" gracePeriod=30
Oct 31 01:32:37.407490 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:32:37.407464    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:32:37.492839 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:32:37.492813    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-cb9a07-5877444fcd-d6mg2"
Oct 31 01:32:37.578874 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:32:37.578844    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b8db1668-6ee8-4d2e-b531-73300b46d926-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b8db1668-6ee8-4d2e-b531-73300b46d926-openshift-service-ca\") pod \"b8db1668-6ee8-4d2e-b531-73300b46d926\" (UID: \"b8db1668-6ee8-4d2e-b531-73300b46d926\") "
Oct 31 01:32:37.578874 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:32:37.578878    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b8db1668-6ee8-4d2e-b531-73300b46d926-kube-api-access-4dqpp\" (UniqueName: \"kubernetes.io/projected/b8db1668-6ee8-4d2e-b531-73300b46d926-kube-api-access-4dqpp\") pod \"b8db1668-6ee8-4d2e-b531-73300b46d926\" (UID: \"b8db1668-6ee8-4d2e-b531-73300b46d926\") "
Oct 31 01:32:37.579127 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:32:37.578962    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/b8db1668-6ee8-4d2e-b531-73300b46d926-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b8db1668-6ee8-4d2e-b531-73300b46d926-maas-api-tls\") pod \"b8db1668-6ee8-4d2e-b531-73300b46d926\" (UID: \"b8db1668-6ee8-4d2e-b531-73300b46d926\") "
Oct 31 01:32:37.579127 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:32:37.579008    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/b8db1668-6ee8-4d2e-b531-73300b46d926-metrics-certs\" (UniqueName: \"kubernetes.io/secret/b8db1668-6ee8-4d2e-b531-73300b46d926-metrics-certs\") pod \"b8db1668-6ee8-4d2e-b531-73300b46d926\" (UID: \"b8db1668-6ee8-4d2e-b531-73300b46d926\") "
Oct 31 01:32:37.579317 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:32:37.579285    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b8db1668-6ee8-4d2e-b531-73300b46d926-openshift-service-ca" pod "b8db1668-6ee8-4d2e-b531-73300b46d926" (UID: "b8db1668-6ee8-4d2e-b531-73300b46d926"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 01:32:37.581247 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:32:37.581198    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b8db1668-6ee8-4d2e-b531-73300b46d926-metrics-certs" pod "b8db1668-6ee8-4d2e-b531-73300b46d926" (UID: "b8db1668-6ee8-4d2e-b531-73300b46d926"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 01:32:37.581371 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:32:37.581295    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b8db1668-6ee8-4d2e-b531-73300b46d926-maas-api-tls" pod "b8db1668-6ee8-4d2e-b531-73300b46d926" (UID: "b8db1668-6ee8-4d2e-b531-73300b46d926"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 01:32:37.581371 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:32:37.581312    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b8db1668-6ee8-4d2e-b531-73300b46d926-kube-api-access-4dqpp" pod "b8db1668-6ee8-4d2e-b531-73300b46d926" (UID: "b8db1668-6ee8-4d2e-b531-73300b46d926"). InnerVolumeSpecName "kube-api-access-4dqpp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:32:37.680644 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:32:37.680605    4284 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b8db1668-6ee8-4d2e-b531-73300b46d926-openshift-service-ca\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:32:37.680644 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:32:37.680635    4284 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4dqpp\" (UniqueName: \"kubernetes.io/projected/b8db1668-6ee8-4d2e-b531-73300b46d926-kube-api-access-4dqpp\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:32:37.680644 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:32:37.680646    4284 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b8db1668-6ee8-4d2e-b531-73300b46d926-maas-api-tls\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:32:37.680644 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:32:37.680656    4284 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b8db1668-6ee8-4d2e-b531-73300b46d926-metrics-certs\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:32:37.690808 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:32:37.690774    4284 generic.go:356] "Generic (PLEG): container finished" podID="b8db1668-6ee8-4d2e-b531-73300b46d926" containerID="b7e492fcafcb503afb146c68f7604163908ebd5efa0e350f2d56094b4169bc26" exitCode=0
Oct 31 01:32:37.690969 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:32:37.690823    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-cb9a07-5877444fcd-d6mg2"
Oct 31 01:32:37.690969 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:32:37.690844    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-cb9a07-5877444fcd-d6mg2" event={"ID":"b8db1668-6ee8-4d2e-b531-73300b46d926","Type":"ContainerDied","Data":"b7e492fcafcb503afb146c68f7604163908ebd5efa0e350f2d56094b4169bc26"}
Oct 31 01:32:37.690969 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:32:37.690895    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-cb9a07-5877444fcd-d6mg2" event={"ID":"b8db1668-6ee8-4d2e-b531-73300b46d926","Type":"ContainerDied","Data":"3ccad606a8b78cf6c880ada2b0dea51d3178e3da0b0f492b88649a17c87a3ddb"}
Oct 31 01:32:37.690969 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:32:37.690926    4284 scope.go:122] "RemoveContainer" containerID="b7e492fcafcb503afb146c68f7604163908ebd5efa0e350f2d56094b4169bc26"
Oct 31 01:32:37.701058 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:32:37.701038    4284 scope.go:122] "RemoveContainer" containerID="b7e492fcafcb503afb146c68f7604163908ebd5efa0e350f2d56094b4169bc26"
Oct 31 01:32:37.701437 ip-10-0-132-173 kubenswrapper[4284]: E1031 01:32:37.701418    4284 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b7e492fcafcb503afb146c68f7604163908ebd5efa0e350f2d56094b4169bc26\": container with ID starting with b7e492fcafcb503afb146c68f7604163908ebd5efa0e350f2d56094b4169bc26 not found: ID does not exist" containerID="b7e492fcafcb503afb146c68f7604163908ebd5efa0e350f2d56094b4169bc26"
Oct 31 01:32:37.701495 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:32:37.701446    4284 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b7e492fcafcb503afb146c68f7604163908ebd5efa0e350f2d56094b4169bc26"} err="failed to get container status \"b7e492fcafcb503afb146c68f7604163908ebd5efa0e350f2d56094b4169bc26\": rpc error: code = NotFound desc = could not find container \"b7e492fcafcb503afb146c68f7604163908ebd5efa0e350f2d56094b4169bc26\": container with ID starting with b7e492fcafcb503afb146c68f7604163908ebd5efa0e350f2d56094b4169bc26 not found: ID does not exist"
Oct 31 01:32:37.705613 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:32:37.705587    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w0-cb9a07-d74f7465c-pcn8h"]
Oct 31 01:32:37.860632 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:32:37.860542    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:32:37.962460 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:32:37.962432    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-cb9a07-5877444fcd-d6mg2"]
Oct 31 01:32:37.965618 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:32:37.965594    4284 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-cb9a07-5877444fcd-d6mg2"]
Oct 31 01:32:38.060845 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:32:38.060811    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:32:39.087932 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:32:39.087895    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:32:39.961762 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:32:39.961726    4284 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b8db1668-6ee8-4d2e-b531-73300b46d926" path="/var/lib/kubelet/pods/b8db1668-6ee8-4d2e-b531-73300b46d926/volumes"
Oct 31 01:32:42.336477 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:32:42.336442    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:32:42.606905 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:32:42.606820    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:32:42.706058 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:32:42.705985    4284 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-a-w0-cb9a07-d74f7465c-pcn8h" podUID="293f3856-d42d-456a-afad-a0a2b0b60081" containerName="payload-processing" containerID="cri-o://9f7347989cb631dfc6fb8a5e54ab00796b3730cc15e9f67b70d160d25e1a620d" gracePeriod=25
Oct 31 01:32:43.310923 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:32:43.310892    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:32:43.963366 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:32:43.963327    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:32:46.319746 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:32:46.319709    4284 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-shared-a-w0-cb9a07-d74f7465c-pcn8h" podUID="293f3856-d42d-456a-afad-a0a2b0b60081" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 31 01:32:47.014144 ip-10-0-132-173 kubenswrapper[4284]: E1031 01:32:47.014110    4284 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fd8sp" containerName="extractor"
Oct 31 01:32:49.020023 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:32:49.019982    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:32:49.723426 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:32:49.723393    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:32:49.945699 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:32:49.945664    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:32:50.116986 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:32:50.116890    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:32:54.087921 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:32:54.087877    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:32:56.319775 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:32:56.319725    4284 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-shared-a-w0-cb9a07-d74f7465c-pcn8h" podUID="293f3856-d42d-456a-afad-a0a2b0b60081" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 31 01:32:58.891174 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:32:58.891142    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:33:02.895048 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:33:02.895007    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:33:02.981123 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:33:02.981099    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w0-cb9a07-d74f7465c-pcn8h"
Oct 31 01:33:03.127144 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:33:03.127107    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/293f3856-d42d-456a-afad-a0a2b0b60081-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/293f3856-d42d-456a-afad-a0a2b0b60081-plugins-config-volume\") pod \"293f3856-d42d-456a-afad-a0a2b0b60081\" (UID: \"293f3856-d42d-456a-afad-a0a2b0b60081\") "
Oct 31 01:33:03.127377 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:33:03.127199    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/293f3856-d42d-456a-afad-a0a2b0b60081-kube-api-access-vcsgw\" (UniqueName: \"kubernetes.io/projected/293f3856-d42d-456a-afad-a0a2b0b60081-kube-api-access-vcsgw\") pod \"293f3856-d42d-456a-afad-a0a2b0b60081\" (UID: \"293f3856-d42d-456a-afad-a0a2b0b60081\") "
Oct 31 01:33:03.127550 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:33:03.127523    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/293f3856-d42d-456a-afad-a0a2b0b60081-plugins-config-volume" pod "293f3856-d42d-456a-afad-a0a2b0b60081" (UID: "293f3856-d42d-456a-afad-a0a2b0b60081"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 01:33:03.131019 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:33:03.130983    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/293f3856-d42d-456a-afad-a0a2b0b60081-kube-api-access-vcsgw" pod "293f3856-d42d-456a-afad-a0a2b0b60081" (UID: "293f3856-d42d-456a-afad-a0a2b0b60081"). InnerVolumeSpecName "kube-api-access-vcsgw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:33:03.228466 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:33:03.228375    4284 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/293f3856-d42d-456a-afad-a0a2b0b60081-plugins-config-volume\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:33:03.228466 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:33:03.228407    4284 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vcsgw\" (UniqueName: \"kubernetes.io/projected/293f3856-d42d-456a-afad-a0a2b0b60081-kube-api-access-vcsgw\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:33:03.809635 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:33:03.809587    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:33:03.813745 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:33:03.813718    4284 generic.go:356] "Generic (PLEG): container finished" podID="293f3856-d42d-456a-afad-a0a2b0b60081" containerID="9f7347989cb631dfc6fb8a5e54ab00796b3730cc15e9f67b70d160d25e1a620d" exitCode=0
Oct 31 01:33:03.813916 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:33:03.813749    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w0-cb9a07-d74f7465c-pcn8h" event={"ID":"293f3856-d42d-456a-afad-a0a2b0b60081","Type":"ContainerDied","Data":"9f7347989cb631dfc6fb8a5e54ab00796b3730cc15e9f67b70d160d25e1a620d"}
Oct 31 01:33:03.813916 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:33:03.813779    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w0-cb9a07-d74f7465c-pcn8h"
Oct 31 01:33:03.813916 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:33:03.813820    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w0-cb9a07-d74f7465c-pcn8h" event={"ID":"293f3856-d42d-456a-afad-a0a2b0b60081","Type":"ContainerDied","Data":"63c4e3686dd159e7f6f71fc71b61ed6ad6a6638fd8b07817228f43e14a335f22"}
Oct 31 01:33:03.813916 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:33:03.813835    4284 scope.go:122] "RemoveContainer" containerID="9f7347989cb631dfc6fb8a5e54ab00796b3730cc15e9f67b70d160d25e1a620d"
Oct 31 01:33:03.823239 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:33:03.822953    4284 scope.go:122] "RemoveContainer" containerID="9f7347989cb631dfc6fb8a5e54ab00796b3730cc15e9f67b70d160d25e1a620d"
Oct 31 01:33:03.823336 ip-10-0-132-173 kubenswrapper[4284]: E1031 01:33:03.823288    4284 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9f7347989cb631dfc6fb8a5e54ab00796b3730cc15e9f67b70d160d25e1a620d\": container with ID starting with 9f7347989cb631dfc6fb8a5e54ab00796b3730cc15e9f67b70d160d25e1a620d not found: ID does not exist" containerID="9f7347989cb631dfc6fb8a5e54ab00796b3730cc15e9f67b70d160d25e1a620d"
Oct 31 01:33:03.823391 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:33:03.823324    4284 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9f7347989cb631dfc6fb8a5e54ab00796b3730cc15e9f67b70d160d25e1a620d"} err="failed to get container status \"9f7347989cb631dfc6fb8a5e54ab00796b3730cc15e9f67b70d160d25e1a620d\": rpc error: code = NotFound desc = could not find container \"9f7347989cb631dfc6fb8a5e54ab00796b3730cc15e9f67b70d160d25e1a620d\": container with ID starting with 9f7347989cb631dfc6fb8a5e54ab00796b3730cc15e9f67b70d160d25e1a620d not found: ID does not exist"
Oct 31 01:33:03.971827 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:33:03.971789    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w0-cb9a07-d74f7465c-pcn8h"]
Oct 31 01:33:04.019335 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:33:04.019305    4284 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w0-cb9a07-d74f7465c-pcn8h"]
Oct 31 01:33:05.961151 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:33:05.961119    4284 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="293f3856-d42d-456a-afad-a0a2b0b60081" path="/var/lib/kubelet/pods/293f3856-d42d-456a-afad-a0a2b0b60081/volumes"
Oct 31 01:33:08.695833 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:33:08.695794    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:33:10.695987 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:33:10.695954    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:33:12.782888 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:33:12.782851    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-7f8596-79fb4467bd-qxqqj"]
Oct 31 01:33:12.783358 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:33:12.783051    4284 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-7f8596-79fb4467bd-qxqqj" podUID="6dcf3d48-1abd-4417-8933-d3dabc39b565" containerName="maas-api" containerID="cri-o://93313ebf6cee996cf4194248cf36e7d77a47314ac706cb40e0a9ff2b7216f0e0" gracePeriod=30
Oct 31 01:33:12.888826 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:33:12.888785    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:33:13.035093 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:33:13.035034    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-7f8596-79fb4467bd-qxqqj"
Oct 31 01:33:13.120762 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:33:13.120730    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/6dcf3d48-1abd-4417-8933-d3dabc39b565-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6dcf3d48-1abd-4417-8933-d3dabc39b565-maas-api-tls\") pod \"6dcf3d48-1abd-4417-8933-d3dabc39b565\" (UID: \"6dcf3d48-1abd-4417-8933-d3dabc39b565\") "
Oct 31 01:33:13.120941 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:33:13.120793    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6dcf3d48-1abd-4417-8933-d3dabc39b565-kube-api-access-jrtk7\" (UniqueName: \"kubernetes.io/projected/6dcf3d48-1abd-4417-8933-d3dabc39b565-kube-api-access-jrtk7\") pod \"6dcf3d48-1abd-4417-8933-d3dabc39b565\" (UID: \"6dcf3d48-1abd-4417-8933-d3dabc39b565\") "
Oct 31 01:33:13.120941 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:33:13.120857    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/6dcf3d48-1abd-4417-8933-d3dabc39b565-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6dcf3d48-1abd-4417-8933-d3dabc39b565-openshift-service-ca\") pod \"6dcf3d48-1abd-4417-8933-d3dabc39b565\" (UID: \"6dcf3d48-1abd-4417-8933-d3dabc39b565\") "
Oct 31 01:33:13.120941 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:33:13.120899    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/6dcf3d48-1abd-4417-8933-d3dabc39b565-metrics-certs\" (UniqueName: \"kubernetes.io/secret/6dcf3d48-1abd-4417-8933-d3dabc39b565-metrics-certs\") pod \"6dcf3d48-1abd-4417-8933-d3dabc39b565\" (UID: \"6dcf3d48-1abd-4417-8933-d3dabc39b565\") "
Oct 31 01:33:13.121314 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:33:13.121286    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6dcf3d48-1abd-4417-8933-d3dabc39b565-openshift-service-ca" pod "6dcf3d48-1abd-4417-8933-d3dabc39b565" (UID: "6dcf3d48-1abd-4417-8933-d3dabc39b565"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 01:33:13.123038 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:33:13.123012    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6dcf3d48-1abd-4417-8933-d3dabc39b565-metrics-certs" pod "6dcf3d48-1abd-4417-8933-d3dabc39b565" (UID: "6dcf3d48-1abd-4417-8933-d3dabc39b565"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 01:33:13.123133 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:33:13.123035    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6dcf3d48-1abd-4417-8933-d3dabc39b565-kube-api-access-jrtk7" pod "6dcf3d48-1abd-4417-8933-d3dabc39b565" (UID: "6dcf3d48-1abd-4417-8933-d3dabc39b565"). InnerVolumeSpecName "kube-api-access-jrtk7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:33:13.123133 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:33:13.123066    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6dcf3d48-1abd-4417-8933-d3dabc39b565-maas-api-tls" pod "6dcf3d48-1abd-4417-8933-d3dabc39b565" (UID: "6dcf3d48-1abd-4417-8933-d3dabc39b565"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 01:33:13.222333 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:33:13.222291    4284 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jrtk7\" (UniqueName: \"kubernetes.io/projected/6dcf3d48-1abd-4417-8933-d3dabc39b565-kube-api-access-jrtk7\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:33:13.222333 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:33:13.222325    4284 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6dcf3d48-1abd-4417-8933-d3dabc39b565-openshift-service-ca\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:33:13.222333 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:33:13.222341    4284 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6dcf3d48-1abd-4417-8933-d3dabc39b565-metrics-certs\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:33:13.222586 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:33:13.222354    4284 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6dcf3d48-1abd-4417-8933-d3dabc39b565-maas-api-tls\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:33:13.861444 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:33:13.861409    4284 generic.go:356] "Generic (PLEG): container finished" podID="6dcf3d48-1abd-4417-8933-d3dabc39b565" containerID="93313ebf6cee996cf4194248cf36e7d77a47314ac706cb40e0a9ff2b7216f0e0" exitCode=0
Oct 31 01:33:13.861984 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:33:13.861462    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-7f8596-79fb4467bd-qxqqj"
Oct 31 01:33:13.861984 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:33:13.861485    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-7f8596-79fb4467bd-qxqqj" event={"ID":"6dcf3d48-1abd-4417-8933-d3dabc39b565","Type":"ContainerDied","Data":"93313ebf6cee996cf4194248cf36e7d77a47314ac706cb40e0a9ff2b7216f0e0"}
Oct 31 01:33:13.861984 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:33:13.861560    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-7f8596-79fb4467bd-qxqqj" event={"ID":"6dcf3d48-1abd-4417-8933-d3dabc39b565","Type":"ContainerDied","Data":"81da6da65e0ac9a067d1fc2a0d9a3689a32a57b23babed8789efe2f77d15c979"}
Oct 31 01:33:13.861984 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:33:13.861584    4284 scope.go:122] "RemoveContainer" containerID="93313ebf6cee996cf4194248cf36e7d77a47314ac706cb40e0a9ff2b7216f0e0"
Oct 31 01:33:13.871141 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:33:13.871126    4284 scope.go:122] "RemoveContainer" containerID="93313ebf6cee996cf4194248cf36e7d77a47314ac706cb40e0a9ff2b7216f0e0"
Oct 31 01:33:13.871420 ip-10-0-132-173 kubenswrapper[4284]: E1031 01:33:13.871403    4284 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"93313ebf6cee996cf4194248cf36e7d77a47314ac706cb40e0a9ff2b7216f0e0\": container with ID starting with 93313ebf6cee996cf4194248cf36e7d77a47314ac706cb40e0a9ff2b7216f0e0 not found: ID does not exist" containerID="93313ebf6cee996cf4194248cf36e7d77a47314ac706cb40e0a9ff2b7216f0e0"
Oct 31 01:33:13.871475 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:33:13.871429    4284 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"93313ebf6cee996cf4194248cf36e7d77a47314ac706cb40e0a9ff2b7216f0e0"} err="failed to get container status \"93313ebf6cee996cf4194248cf36e7d77a47314ac706cb40e0a9ff2b7216f0e0\": rpc error: code = NotFound desc = could not find container \"93313ebf6cee996cf4194248cf36e7d77a47314ac706cb40e0a9ff2b7216f0e0\": container with ID starting with 93313ebf6cee996cf4194248cf36e7d77a47314ac706cb40e0a9ff2b7216f0e0 not found: ID does not exist"
Oct 31 01:33:14.020755 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:33:14.020721    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-7f8596-79fb4467bd-qxqqj"]
Oct 31 01:33:14.025250 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:33:14.025206    4284 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-7f8596-79fb4467bd-qxqqj"]
Oct 31 01:33:14.384505 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:33:14.384474    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:33:14.639923 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:33:14.639835    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:33:14.786460 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:33:14.786422    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:33:15.406387 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:33:15.406351    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:33:15.728471 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:33:15.728373    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:33:15.962002 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:33:15.961962    4284 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6dcf3d48-1abd-4417-8933-d3dabc39b565" path="/var/lib/kubelet/pods/6dcf3d48-1abd-4417-8933-d3dabc39b565/volumes"
Oct 31 01:33:17.014063 ip-10-0-132-173 kubenswrapper[4284]: E1031 01:33:17.013983    4284 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fd8sp" containerName="extractor"
Oct 31 01:33:19.794087 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:33:19.794046    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:33:22.335713 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:33:22.335675    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:33:22.480652 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:33:22.480611    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:33:22.683082 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:33:22.683046    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:33:23.261018 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:33:23.260982    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:33:24.971995 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:33:24.971961    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:33:28.433233 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:33:28.433169    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:33:30.183736 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:33:30.183696    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:33:35.041227 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:33:35.041183    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:33:35.290539 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:33:35.290503    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-8bec73-74b4bf6479-9vmvq"]
Oct 31 01:33:35.400631 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:33:35.400552    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-8bec73-6b85599f55-5vzdh"]
Oct 31 01:33:35.667187 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:33:35.667140    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:33:35.868230 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:33:35.868180    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:33:39.373034 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:33:39.373000    4284 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-f88cd455-zwqnv"]
Oct 31 01:33:39.373566 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:33:39.373546    4284 cpu_manager.go:414] "containerMap: removing container" podUID="c3fc41e7-426c-4bdb-b5ef-55dce305cfae" containerName="cleanup"
Oct 31 01:33:39.373626 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:33:39.373569    4284 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c3fc41e7-426c-4bdb-b5ef-55dce305cfae" containerName="cleanup"
Oct 31 01:33:39.373626 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:33:39.373582    4284 cpu_manager.go:414] "containerMap: removing container" podUID="eed2968a-68fc-44a9-beb1-669b189009de" containerName="istio-proxy"
Oct 31 01:33:39.373626 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:33:39.373590    4284 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="eed2968a-68fc-44a9-beb1-669b189009de" containerName="istio-proxy"
Oct 31 01:33:39.373626 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:33:39.373599    4284 cpu_manager.go:414] "containerMap: removing container" podUID="b8db1668-6ee8-4d2e-b531-73300b46d926" containerName="maas-api"
Oct 31 01:33:39.373626 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:33:39.373606    4284 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b8db1668-6ee8-4d2e-b531-73300b46d926" containerName="maas-api"
Oct 31 01:33:39.373626 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:33:39.373620    4284 cpu_manager.go:414] "containerMap: removing container" podUID="b56df4cb-7bf2-4e02-ab77-f33e078aba3c" containerName="maas-api"
Oct 31 01:33:39.373626 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:33:39.373627    4284 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b56df4cb-7bf2-4e02-ab77-f33e078aba3c" containerName="maas-api"
Oct 31 01:33:39.373942 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:33:39.373639    4284 cpu_manager.go:414] "containerMap: removing container" podUID="cb311a47-264c-4b33-bc31-377b94b0c59b" containerName="main"
Oct 31 01:33:39.373942 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:33:39.373646    4284 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="cb311a47-264c-4b33-bc31-377b94b0c59b" containerName="main"
Oct 31 01:33:39.373942 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:33:39.373659    4284 cpu_manager.go:414] "containerMap: removing container" podUID="e6cf830a-f9d2-4d0c-90de-672a81d3a441" containerName="istio-proxy"
Oct 31 01:33:39.373942 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:33:39.373666    4284 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e6cf830a-f9d2-4d0c-90de-672a81d3a441" containerName="istio-proxy"
Oct 31 01:33:39.373942 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:33:39.373683    4284 cpu_manager.go:414] "containerMap: removing container" podUID="6dcf3d48-1abd-4417-8933-d3dabc39b565" containerName="maas-api"
Oct 31 01:33:39.373942 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:33:39.373690    4284 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6dcf3d48-1abd-4417-8933-d3dabc39b565" containerName="maas-api"
Oct 31 01:33:39.373942 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:33:39.373707    4284 cpu_manager.go:414] "containerMap: removing container" podUID="b8c47f8b-d458-4669-9af4-95fd324dff37" containerName="maas-api"
Oct 31 01:33:39.373942 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:33:39.373713    4284 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b8c47f8b-d458-4669-9af4-95fd324dff37" containerName="maas-api"
Oct 31 01:33:39.373942 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:33:39.373729    4284 cpu_manager.go:414] "containerMap: removing container" podUID="164bb74b-a95d-4b77-97b6-f2ca2eb2cc09" containerName="istio-proxy"
Oct 31 01:33:39.373942 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:33:39.373736    4284 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="164bb74b-a95d-4b77-97b6-f2ca2eb2cc09" containerName="istio-proxy"
Oct 31 01:33:39.373942 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:33:39.373749    4284 cpu_manager.go:414] "containerMap: removing container" podUID="ccf1138e-2e92-4ab6-bfdd-9979f0b69893" containerName="istio-proxy"
Oct 31 01:33:39.373942 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:33:39.373756    4284 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ccf1138e-2e92-4ab6-bfdd-9979f0b69893" containerName="istio-proxy"
Oct 31 01:33:39.373942 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:33:39.373770    4284 cpu_manager.go:414] "containerMap: removing container" podUID="ccf1138e-2e92-4ab6-bfdd-9979f0b69893" containerName="istio-proxy"
Oct 31 01:33:39.373942 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:33:39.373777    4284 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ccf1138e-2e92-4ab6-bfdd-9979f0b69893" containerName="istio-proxy"
Oct 31 01:33:39.373942 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:33:39.373787    4284 cpu_manager.go:414] "containerMap: removing container" podUID="fea709fd-6209-4b40-95f6-ea5104c1cb09" containerName="payload-pre-processing"
Oct 31 01:33:39.373942 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:33:39.373794    4284 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="fea709fd-6209-4b40-95f6-ea5104c1cb09" containerName="payload-pre-processing"
Oct 31 01:33:39.373942 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:33:39.373807    4284 cpu_manager.go:414] "containerMap: removing container" podUID="293f3856-d42d-456a-afad-a0a2b0b60081" containerName="payload-processing"
Oct 31 01:33:39.373942 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:33:39.373814    4284 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="293f3856-d42d-456a-afad-a0a2b0b60081" containerName="payload-processing"
Oct 31 01:33:39.373942 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:33:39.373824    4284 cpu_manager.go:414] "containerMap: removing container" podUID="e6cf830a-f9d2-4d0c-90de-672a81d3a441" containerName="istio-proxy"
Oct 31 01:33:39.373942 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:33:39.373830    4284 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e6cf830a-f9d2-4d0c-90de-672a81d3a441" containerName="istio-proxy"
Oct 31 01:33:39.373942 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:33:39.373843    4284 cpu_manager.go:414] "containerMap: removing container" podUID="e0d645bd-be9d-43da-b745-40ff32e3a51a" containerName="revoke-keys"
Oct 31 01:33:39.373942 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:33:39.373851    4284 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e0d645bd-be9d-43da-b745-40ff32e3a51a" containerName="revoke-keys"
Oct 31 01:33:39.373942 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:33:39.373863    4284 cpu_manager.go:414] "containerMap: removing container" podUID="5fd0d1e4-e8d2-43a1-ac72-1e16bb0cbaef" containerName="istio-proxy"
Oct 31 01:33:39.373942 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:33:39.373870    4284 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5fd0d1e4-e8d2-43a1-ac72-1e16bb0cbaef" containerName="istio-proxy"
Oct 31 01:33:39.373942 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:33:39.373885    4284 cpu_manager.go:414] "containerMap: removing container" podUID="c3fc41e7-426c-4bdb-b5ef-55dce305cfae" containerName="cleanup"
Oct 31 01:33:39.373942 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:33:39.373891    4284 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c3fc41e7-426c-4bdb-b5ef-55dce305cfae" containerName="cleanup"
Oct 31 01:33:39.374853 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:33:39.373976    4284 memory_manager.go:365] "RemoveStaleState removing state" podUID="b8c47f8b-d458-4669-9af4-95fd324dff37" containerName="maas-api"
Oct 31 01:33:39.374853 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:33:39.373986    4284 memory_manager.go:365] "RemoveStaleState removing state" podUID="eed2968a-68fc-44a9-beb1-669b189009de" containerName="istio-proxy"
Oct 31 01:33:39.374853 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:33:39.373992    4284 memory_manager.go:365] "RemoveStaleState removing state" podUID="164bb74b-a95d-4b77-97b6-f2ca2eb2cc09" containerName="istio-proxy"
Oct 31 01:33:39.374853 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:33:39.373998    4284 memory_manager.go:365] "RemoveStaleState removing state" podUID="ccf1138e-2e92-4ab6-bfdd-9979f0b69893" containerName="istio-proxy"
Oct 31 01:33:39.374853 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:33:39.374003    4284 memory_manager.go:365] "RemoveStaleState removing state" podUID="c3fc41e7-426c-4bdb-b5ef-55dce305cfae" containerName="cleanup"
Oct 31 01:33:39.374853 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:33:39.374009    4284 memory_manager.go:365] "RemoveStaleState removing state" podUID="fea709fd-6209-4b40-95f6-ea5104c1cb09" containerName="payload-pre-processing"
Oct 31 01:33:39.374853 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:33:39.374014    4284 memory_manager.go:365] "RemoveStaleState removing state" podUID="293f3856-d42d-456a-afad-a0a2b0b60081" containerName="payload-processing"
Oct 31 01:33:39.374853 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:33:39.374019    4284 memory_manager.go:365] "RemoveStaleState removing state" podUID="e6cf830a-f9d2-4d0c-90de-672a81d3a441" containerName="istio-proxy"
Oct 31 01:33:39.374853 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:33:39.374024    4284 memory_manager.go:365] "RemoveStaleState removing state" podUID="e0d645bd-be9d-43da-b745-40ff32e3a51a" containerName="revoke-keys"
Oct 31 01:33:39.374853 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:33:39.374030    4284 memory_manager.go:365] "RemoveStaleState removing state" podUID="5fd0d1e4-e8d2-43a1-ac72-1e16bb0cbaef" containerName="istio-proxy"
Oct 31 01:33:39.374853 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:33:39.374036    4284 memory_manager.go:365] "RemoveStaleState removing state" podUID="c3fc41e7-426c-4bdb-b5ef-55dce305cfae" containerName="cleanup"
Oct 31 01:33:39.374853 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:33:39.374041    4284 memory_manager.go:365] "RemoveStaleState removing state" podUID="b8db1668-6ee8-4d2e-b531-73300b46d926" containerName="maas-api"
Oct 31 01:33:39.374853 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:33:39.374046    4284 memory_manager.go:365] "RemoveStaleState removing state" podUID="b56df4cb-7bf2-4e02-ab77-f33e078aba3c" containerName="maas-api"
Oct 31 01:33:39.374853 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:33:39.374052    4284 memory_manager.go:365] "RemoveStaleState removing state" podUID="cb311a47-264c-4b33-bc31-377b94b0c59b" containerName="main"
Oct 31 01:33:39.374853 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:33:39.374058    4284 memory_manager.go:365] "RemoveStaleState removing state" podUID="6dcf3d48-1abd-4417-8933-d3dabc39b565" containerName="maas-api"
Oct 31 01:33:39.377206 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:33:39.377192    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f88cd455-zwqnv"
Oct 31 01:33:39.430779 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:33:39.430752    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-f88cd455-zwqnv"]
Oct 31 01:33:39.474734 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:33:39.474695    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kl8nv\" (UniqueName: \"kubernetes.io/projected/c756875d-8413-4d7d-ba88-c78cc982b3f1-kube-api-access-kl8nv\") pod \"maas-api-revoke-sub-f88cd455-zwqnv\" (UID: \"c756875d-8413-4d7d-ba88-c78cc982b3f1\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f88cd455-zwqnv"
Oct 31 01:33:39.474903 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:33:39.474861    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/c756875d-8413-4d7d-ba88-c78cc982b3f1-maas-api-xks-ca\") pod \"maas-api-revoke-sub-f88cd455-zwqnv\" (UID: \"c756875d-8413-4d7d-ba88-c78cc982b3f1\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f88cd455-zwqnv"
Oct 31 01:33:39.474968 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:33:39.474922    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/c756875d-8413-4d7d-ba88-c78cc982b3f1-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-f88cd455-zwqnv\" (UID: \"c756875d-8413-4d7d-ba88-c78cc982b3f1\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f88cd455-zwqnv"
Oct 31 01:33:39.475755 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:33:39.475735    4284 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 31 01:33:39.476006 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:33:39.475988    4284 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-8pzqp\""
Oct 31 01:33:39.575875 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:33:39.575834    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/c756875d-8413-4d7d-ba88-c78cc982b3f1-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-f88cd455-zwqnv\" (UID: \"c756875d-8413-4d7d-ba88-c78cc982b3f1\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f88cd455-zwqnv"
Oct 31 01:33:39.576087 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:33:39.575893    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kl8nv\" (UniqueName: \"kubernetes.io/projected/c756875d-8413-4d7d-ba88-c78cc982b3f1-kube-api-access-kl8nv\") pod \"maas-api-revoke-sub-f88cd455-zwqnv\" (UID: \"c756875d-8413-4d7d-ba88-c78cc982b3f1\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f88cd455-zwqnv"
Oct 31 01:33:39.576087 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:33:39.575997    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/c756875d-8413-4d7d-ba88-c78cc982b3f1-maas-api-xks-ca\") pod \"maas-api-revoke-sub-f88cd455-zwqnv\" (UID: \"c756875d-8413-4d7d-ba88-c78cc982b3f1\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f88cd455-zwqnv"
Oct 31 01:33:39.576520 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:33:39.576498    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/c756875d-8413-4d7d-ba88-c78cc982b3f1-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-f88cd455-zwqnv\" (UID: \"c756875d-8413-4d7d-ba88-c78cc982b3f1\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f88cd455-zwqnv"
Oct 31 01:33:39.578483 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:33:39.578462    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/c756875d-8413-4d7d-ba88-c78cc982b3f1-maas-api-xks-ca\") pod \"maas-api-revoke-sub-f88cd455-zwqnv\" (UID: \"c756875d-8413-4d7d-ba88-c78cc982b3f1\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f88cd455-zwqnv"
Oct 31 01:33:39.634809 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:33:39.634740    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kl8nv\" (UniqueName: \"kubernetes.io/projected/c756875d-8413-4d7d-ba88-c78cc982b3f1-kube-api-access-kl8nv\") pod \"maas-api-revoke-sub-f88cd455-zwqnv\" (UID: \"c756875d-8413-4d7d-ba88-c78cc982b3f1\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f88cd455-zwqnv"
Oct 31 01:33:39.689377 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:33:39.689335    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f88cd455-zwqnv"
Oct 31 01:33:40.053535 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:33:40.053508    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-f88cd455-zwqnv"]
Oct 31 01:33:40.054847 ip-10-0-132-173 kubenswrapper[4284]: W1031 01:33:40.054822    4284 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc756875d_8413_4d7d_ba88_c78cc982b3f1.slice/crio-a3df2d05375337c5961c84d2b6f2616693646865f3da0b67e285b71759473058 WatchSource:0}: Error finding container a3df2d05375337c5961c84d2b6f2616693646865f3da0b67e285b71759473058: Status 404 returned error can't find the container with id a3df2d05375337c5961c84d2b6f2616693646865f3da0b67e285b71759473058
Oct 31 01:33:40.291485 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:33:40.291450    4284 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-worker-w0-8bec73-74b4bf6479-9vmvq" podUID="e9f98efa-eb3e-4a3f-bf43-9b1603d0b725" containerName="payload-processing" containerID="cri-o://9f1eafe8bb29a0117f13f39e0a2be7f5c1b6f53647d41d58da4587867a669e37" gracePeriod=25
Oct 31 01:33:40.401551 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:33:40.401501    4284 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-8bec73-6b85599f55-5vzdh" podUID="fc15a078-6a56-4f96-b6a7-ff31044d75c6" containerName="payload-pre-processing" containerID="cri-o://24e9dee9b31c2674a821f1c7e1a29bc9c6eb97bef1e61399f5c5593833e423c2" gracePeriod=25
Oct 31 01:33:40.990878 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:33:40.990842    4284 generic.go:356] "Generic (PLEG): container finished" podID="c756875d-8413-4d7d-ba88-c78cc982b3f1" containerID="8f1d96e66ccd286b2ea5d6769fb83a5c1166623dfcd3e0b82bbb3f97c1d5d02c" exitCode=0
Oct 31 01:33:40.991056 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:33:40.990927    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f88cd455-zwqnv" event={"ID":"c756875d-8413-4d7d-ba88-c78cc982b3f1","Type":"ContainerDied","Data":"8f1d96e66ccd286b2ea5d6769fb83a5c1166623dfcd3e0b82bbb3f97c1d5d02c"}
Oct 31 01:33:40.991056 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:33:40.990978    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f88cd455-zwqnv" event={"ID":"c756875d-8413-4d7d-ba88-c78cc982b3f1","Type":"ContainerStarted","Data":"a3df2d05375337c5961c84d2b6f2616693646865f3da0b67e285b71759473058"}
Oct 31 01:33:42.126036 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:33:42.126003    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f88cd455-zwqnv"
Oct 31 01:33:42.194558 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:33:42.194526    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/c756875d-8413-4d7d-ba88-c78cc982b3f1-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/c756875d-8413-4d7d-ba88-c78cc982b3f1-maas-api-xks-ca\") pod \"c756875d-8413-4d7d-ba88-c78cc982b3f1\" (UID: \"c756875d-8413-4d7d-ba88-c78cc982b3f1\") "
Oct 31 01:33:42.194735 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:33:42.194576    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c756875d-8413-4d7d-ba88-c78cc982b3f1-kube-api-access-kl8nv\" (UniqueName: \"kubernetes.io/projected/c756875d-8413-4d7d-ba88-c78cc982b3f1-kube-api-access-kl8nv\") pod \"c756875d-8413-4d7d-ba88-c78cc982b3f1\" (UID: \"c756875d-8413-4d7d-ba88-c78cc982b3f1\") "
Oct 31 01:33:42.194735 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:33:42.194620    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c756875d-8413-4d7d-ba88-c78cc982b3f1-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/c756875d-8413-4d7d-ba88-c78cc982b3f1-maas-api-openshift-ca\") pod \"c756875d-8413-4d7d-ba88-c78cc982b3f1\" (UID: \"c756875d-8413-4d7d-ba88-c78cc982b3f1\") "
Oct 31 01:33:42.195004 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:33:42.194980    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c756875d-8413-4d7d-ba88-c78cc982b3f1-maas-api-openshift-ca" pod "c756875d-8413-4d7d-ba88-c78cc982b3f1" (UID: "c756875d-8413-4d7d-ba88-c78cc982b3f1"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 01:33:42.196952 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:33:42.196925    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c756875d-8413-4d7d-ba88-c78cc982b3f1-kube-api-access-kl8nv" pod "c756875d-8413-4d7d-ba88-c78cc982b3f1" (UID: "c756875d-8413-4d7d-ba88-c78cc982b3f1"). InnerVolumeSpecName "kube-api-access-kl8nv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:33:42.196952 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:33:42.196934    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c756875d-8413-4d7d-ba88-c78cc982b3f1-maas-api-xks-ca" pod "c756875d-8413-4d7d-ba88-c78cc982b3f1" (UID: "c756875d-8413-4d7d-ba88-c78cc982b3f1"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 01:33:42.295711 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:33:42.295635    4284 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/c756875d-8413-4d7d-ba88-c78cc982b3f1-maas-api-openshift-ca\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:33:42.295711 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:33:42.295668    4284 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/c756875d-8413-4d7d-ba88-c78cc982b3f1-maas-api-xks-ca\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:33:42.295711 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:33:42.295683    4284 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kl8nv\" (UniqueName: \"kubernetes.io/projected/c756875d-8413-4d7d-ba88-c78cc982b3f1-kube-api-access-kl8nv\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:33:43.001233 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:33:43.001186    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f88cd455-zwqnv" event={"ID":"c756875d-8413-4d7d-ba88-c78cc982b3f1","Type":"ContainerDied","Data":"a3df2d05375337c5961c84d2b6f2616693646865f3da0b67e285b71759473058"}
Oct 31 01:33:43.001413 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:33:43.001261    4284 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a3df2d05375337c5961c84d2b6f2616693646865f3da0b67e285b71759473058"
Oct 31 01:33:43.001413 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:33:43.001196    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f88cd455-zwqnv"
Oct 31 01:33:46.425031 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:33:46.424996    4284 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-w0-8bec73-74b4bf6479-9vmvq" podUID="e9f98efa-eb3e-4a3f-bf43-9b1603d0b725" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 31 01:33:46.425479 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:33:46.424999    4284 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-8bec73-6b85599f55-5vzdh" podUID="fc15a078-6a56-4f96-b6a7-ff31044d75c6" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 31 01:33:47.013088 ip-10-0-132-173 kubenswrapper[4284]: E1031 01:33:47.013047    4284 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fd8sp" containerName="extractor"
Oct 31 01:33:54.807597 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:33:54.807550    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:33:56.243145 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:33:56.243099    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:33:56.425160 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:33:56.425118    4284 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-8bec73-6b85599f55-5vzdh" podUID="fc15a078-6a56-4f96-b6a7-ff31044d75c6" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 31 01:33:56.425160 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:33:56.425132    4284 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-w0-8bec73-74b4bf6479-9vmvq" podUID="e9f98efa-eb3e-4a3f-bf43-9b1603d0b725" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 31 01:33:56.508525 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:33:56.508442    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:33:57.257721 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:33:57.257687    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:34:00.534745 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:34:00.534712    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w0-8bec73-74b4bf6479-9vmvq"
Oct 31 01:34:00.644961 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:34:00.644937    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-8bec73-6b85599f55-5vzdh"
Oct 31 01:34:00.667618 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:34:00.667590    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e9f98efa-eb3e-4a3f-bf43-9b1603d0b725-kube-api-access-p6969\" (UniqueName: \"kubernetes.io/projected/e9f98efa-eb3e-4a3f-bf43-9b1603d0b725-kube-api-access-p6969\") pod \"e9f98efa-eb3e-4a3f-bf43-9b1603d0b725\" (UID: \"e9f98efa-eb3e-4a3f-bf43-9b1603d0b725\") "
Oct 31 01:34:00.667805 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:34:00.667634    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e9f98efa-eb3e-4a3f-bf43-9b1603d0b725-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e9f98efa-eb3e-4a3f-bf43-9b1603d0b725-plugins-config-volume\") pod \"e9f98efa-eb3e-4a3f-bf43-9b1603d0b725\" (UID: \"e9f98efa-eb3e-4a3f-bf43-9b1603d0b725\") "
Oct 31 01:34:00.668052 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:34:00.668024    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e9f98efa-eb3e-4a3f-bf43-9b1603d0b725-plugins-config-volume" pod "e9f98efa-eb3e-4a3f-bf43-9b1603d0b725" (UID: "e9f98efa-eb3e-4a3f-bf43-9b1603d0b725"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 01:34:00.669913 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:34:00.669889    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e9f98efa-eb3e-4a3f-bf43-9b1603d0b725-kube-api-access-p6969" pod "e9f98efa-eb3e-4a3f-bf43-9b1603d0b725" (UID: "e9f98efa-eb3e-4a3f-bf43-9b1603d0b725"). InnerVolumeSpecName "kube-api-access-p6969". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:34:00.768743 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:34:00.768704    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fc15a078-6a56-4f96-b6a7-ff31044d75c6-kube-api-access-g8zkq\" (UniqueName: \"kubernetes.io/projected/fc15a078-6a56-4f96-b6a7-ff31044d75c6-kube-api-access-g8zkq\") pod \"fc15a078-6a56-4f96-b6a7-ff31044d75c6\" (UID: \"fc15a078-6a56-4f96-b6a7-ff31044d75c6\") "
Oct 31 01:34:00.768950 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:34:00.768773    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/fc15a078-6a56-4f96-b6a7-ff31044d75c6-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fc15a078-6a56-4f96-b6a7-ff31044d75c6-plugins-config-volume\") pod \"fc15a078-6a56-4f96-b6a7-ff31044d75c6\" (UID: \"fc15a078-6a56-4f96-b6a7-ff31044d75c6\") "
Oct 31 01:34:00.769118 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:34:00.769097    4284 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p6969\" (UniqueName: \"kubernetes.io/projected/e9f98efa-eb3e-4a3f-bf43-9b1603d0b725-kube-api-access-p6969\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:34:00.769193 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:34:00.769124    4284 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e9f98efa-eb3e-4a3f-bf43-9b1603d0b725-plugins-config-volume\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:34:00.769193 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:34:00.769165    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fc15a078-6a56-4f96-b6a7-ff31044d75c6-plugins-config-volume" pod "fc15a078-6a56-4f96-b6a7-ff31044d75c6" (UID: "fc15a078-6a56-4f96-b6a7-ff31044d75c6"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 01:34:00.771023 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:34:00.770998    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fc15a078-6a56-4f96-b6a7-ff31044d75c6-kube-api-access-g8zkq" pod "fc15a078-6a56-4f96-b6a7-ff31044d75c6" (UID: "fc15a078-6a56-4f96-b6a7-ff31044d75c6"). InnerVolumeSpecName "kube-api-access-g8zkq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:34:00.870347 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:34:00.870255    4284 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g8zkq\" (UniqueName: \"kubernetes.io/projected/fc15a078-6a56-4f96-b6a7-ff31044d75c6-kube-api-access-g8zkq\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:34:00.870347 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:34:00.870288    4284 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fc15a078-6a56-4f96-b6a7-ff31044d75c6-plugins-config-volume\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:34:01.084908 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:34:01.084877    4284 generic.go:356] "Generic (PLEG): container finished" podID="e9f98efa-eb3e-4a3f-bf43-9b1603d0b725" containerID="9f1eafe8bb29a0117f13f39e0a2be7f5c1b6f53647d41d58da4587867a669e37" exitCode=0
Oct 31 01:34:01.085097 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:34:01.084922    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w0-8bec73-74b4bf6479-9vmvq"
Oct 31 01:34:01.085097 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:34:01.084964    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w0-8bec73-74b4bf6479-9vmvq" event={"ID":"e9f98efa-eb3e-4a3f-bf43-9b1603d0b725","Type":"ContainerDied","Data":"9f1eafe8bb29a0117f13f39e0a2be7f5c1b6f53647d41d58da4587867a669e37"}
Oct 31 01:34:01.085097 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:34:01.085017    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w0-8bec73-74b4bf6479-9vmvq" event={"ID":"e9f98efa-eb3e-4a3f-bf43-9b1603d0b725","Type":"ContainerDied","Data":"bbb75327449d55bd07c4ad02fdc735e7e081e1ac5c50153d9795becd5082e8b5"}
Oct 31 01:34:01.085097 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:34:01.085044    4284 scope.go:122] "RemoveContainer" containerID="9f1eafe8bb29a0117f13f39e0a2be7f5c1b6f53647d41d58da4587867a669e37"
Oct 31 01:34:01.086286 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:34:01.086266    4284 generic.go:356] "Generic (PLEG): container finished" podID="fc15a078-6a56-4f96-b6a7-ff31044d75c6" containerID="24e9dee9b31c2674a821f1c7e1a29bc9c6eb97bef1e61399f5c5593833e423c2" exitCode=0
Oct 31 01:34:01.086383 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:34:01.086297    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-8bec73-6b85599f55-5vzdh" event={"ID":"fc15a078-6a56-4f96-b6a7-ff31044d75c6","Type":"ContainerDied","Data":"24e9dee9b31c2674a821f1c7e1a29bc9c6eb97bef1e61399f5c5593833e423c2"}
Oct 31 01:34:01.086383 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:34:01.086320    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-8bec73-6b85599f55-5vzdh"
Oct 31 01:34:01.086383 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:34:01.086350    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-8bec73-6b85599f55-5vzdh" event={"ID":"fc15a078-6a56-4f96-b6a7-ff31044d75c6","Type":"ContainerDied","Data":"b78cde91ee5e5ec00978c066aad31e22261264941388d854be1c8eff4f64f505"}
Oct 31 01:34:01.094763 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:34:01.094669    4284 scope.go:122] "RemoveContainer" containerID="9f1eafe8bb29a0117f13f39e0a2be7f5c1b6f53647d41d58da4587867a669e37"
Oct 31 01:34:01.095074 ip-10-0-132-173 kubenswrapper[4284]: E1031 01:34:01.095048    4284 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9f1eafe8bb29a0117f13f39e0a2be7f5c1b6f53647d41d58da4587867a669e37\": container with ID starting with 9f1eafe8bb29a0117f13f39e0a2be7f5c1b6f53647d41d58da4587867a669e37 not found: ID does not exist" containerID="9f1eafe8bb29a0117f13f39e0a2be7f5c1b6f53647d41d58da4587867a669e37"
Oct 31 01:34:01.095169 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:34:01.095082    4284 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9f1eafe8bb29a0117f13f39e0a2be7f5c1b6f53647d41d58da4587867a669e37"} err="failed to get container status \"9f1eafe8bb29a0117f13f39e0a2be7f5c1b6f53647d41d58da4587867a669e37\": rpc error: code = NotFound desc = could not find container \"9f1eafe8bb29a0117f13f39e0a2be7f5c1b6f53647d41d58da4587867a669e37\": container with ID starting with 9f1eafe8bb29a0117f13f39e0a2be7f5c1b6f53647d41d58da4587867a669e37 not found: ID does not exist"
Oct 31 01:34:01.095169 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:34:01.095107    4284 scope.go:122] "RemoveContainer" containerID="24e9dee9b31c2674a821f1c7e1a29bc9c6eb97bef1e61399f5c5593833e423c2"
Oct 31 01:34:01.103395 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:34:01.103379    4284 scope.go:122] "RemoveContainer" containerID="24e9dee9b31c2674a821f1c7e1a29bc9c6eb97bef1e61399f5c5593833e423c2"
Oct 31 01:34:01.103690 ip-10-0-132-173 kubenswrapper[4284]: E1031 01:34:01.103671    4284 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"24e9dee9b31c2674a821f1c7e1a29bc9c6eb97bef1e61399f5c5593833e423c2\": container with ID starting with 24e9dee9b31c2674a821f1c7e1a29bc9c6eb97bef1e61399f5c5593833e423c2 not found: ID does not exist" containerID="24e9dee9b31c2674a821f1c7e1a29bc9c6eb97bef1e61399f5c5593833e423c2"
Oct 31 01:34:01.103735 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:34:01.103699    4284 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"24e9dee9b31c2674a821f1c7e1a29bc9c6eb97bef1e61399f5c5593833e423c2"} err="failed to get container status \"24e9dee9b31c2674a821f1c7e1a29bc9c6eb97bef1e61399f5c5593833e423c2\": rpc error: code = NotFound desc = could not find container \"24e9dee9b31c2674a821f1c7e1a29bc9c6eb97bef1e61399f5c5593833e423c2\": container with ID starting with 24e9dee9b31c2674a821f1c7e1a29bc9c6eb97bef1e61399f5c5593833e423c2 not found: ID does not exist"
Oct 31 01:34:01.263787 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:34:01.263744    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-8bec73-74b4bf6479-9vmvq"]
Oct 31 01:34:01.282862 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:34:01.282826    4284 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-8bec73-74b4bf6479-9vmvq"]
Oct 31 01:34:01.388543 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:34:01.388503    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-8bec73-6b85599f55-5vzdh"]
Oct 31 01:34:01.391979 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:34:01.391951    4284 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-8bec73-6b85599f55-5vzdh"]
Oct 31 01:34:01.963016 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:34:01.962982    4284 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e9f98efa-eb3e-4a3f-bf43-9b1603d0b725" path="/var/lib/kubelet/pods/e9f98efa-eb3e-4a3f-bf43-9b1603d0b725/volumes"
Oct 31 01:34:01.963569 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:34:01.963369    4284 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="fc15a078-6a56-4f96-b6a7-ff31044d75c6" path="/var/lib/kubelet/pods/fc15a078-6a56-4f96-b6a7-ff31044d75c6/volumes"
Oct 31 01:34:03.469957 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:34:03.469923    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:34:03.706234 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:34:03.706170    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:34:03.853804 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:34:03.853717    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:34:04.100501 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:34:04.100471    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-nc722"]
Oct 31 01:34:04.100746 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:34:04.100663    4284 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-nc722" podUID="e999d104-c9ea-45a7-9930-7f829013c52e" containerName="istio-proxy" containerID="cri-o://4c17ea82450732e49ab6f3eafde1abf9c57f07d3fa62849d8311a5eaf9435ccf" gracePeriod=30
Oct 31 01:34:04.122773 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:34:04.122686    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:34:06.286857 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:34:06.286825    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:34:06.473827 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:34:06.473771    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:34:06.621435 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:34:06.621362    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:34:09.362059 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:34:09.362032    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-nc722"
Oct 31 01:34:09.448091 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:34:09.448059    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e999d104-c9ea-45a7-9930-7f829013c52e-istio-token\" (UniqueName: \"kubernetes.io/projected/e999d104-c9ea-45a7-9930-7f829013c52e-istio-token\") pod \"e999d104-c9ea-45a7-9930-7f829013c52e\" (UID: \"e999d104-c9ea-45a7-9930-7f829013c52e\") "
Oct 31 01:34:09.448321 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:34:09.448099    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/e999d104-c9ea-45a7-9930-7f829013c52e-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e999d104-c9ea-45a7-9930-7f829013c52e-istio-podinfo\") pod \"e999d104-c9ea-45a7-9930-7f829013c52e\" (UID: \"e999d104-c9ea-45a7-9930-7f829013c52e\") "
Oct 31 01:34:09.448321 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:34:09.448125    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e999d104-c9ea-45a7-9930-7f829013c52e-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e999d104-c9ea-45a7-9930-7f829013c52e-istiod-ca-cert\") pod \"e999d104-c9ea-45a7-9930-7f829013c52e\" (UID: \"e999d104-c9ea-45a7-9930-7f829013c52e\") "
Oct 31 01:34:09.448321 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:34:09.448168    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e999d104-c9ea-45a7-9930-7f829013c52e-kube-api-access-vl9hm\" (UniqueName: \"kubernetes.io/projected/e999d104-c9ea-45a7-9930-7f829013c52e-kube-api-access-vl9hm\") pod \"e999d104-c9ea-45a7-9930-7f829013c52e\" (UID: \"e999d104-c9ea-45a7-9930-7f829013c52e\") "
Oct 31 01:34:09.448321 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:34:09.448275    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e999d104-c9ea-45a7-9930-7f829013c52e-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e999d104-c9ea-45a7-9930-7f829013c52e-istio-data\") pod \"e999d104-c9ea-45a7-9930-7f829013c52e\" (UID: \"e999d104-c9ea-45a7-9930-7f829013c52e\") "
Oct 31 01:34:09.448321 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:34:09.448311    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e999d104-c9ea-45a7-9930-7f829013c52e-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e999d104-c9ea-45a7-9930-7f829013c52e-istio-envoy\") pod \"e999d104-c9ea-45a7-9930-7f829013c52e\" (UID: \"e999d104-c9ea-45a7-9930-7f829013c52e\") "
Oct 31 01:34:09.448628 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:34:09.448361    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e999d104-c9ea-45a7-9930-7f829013c52e-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e999d104-c9ea-45a7-9930-7f829013c52e-credential-socket\") pod \"e999d104-c9ea-45a7-9930-7f829013c52e\" (UID: \"e999d104-c9ea-45a7-9930-7f829013c52e\") "
Oct 31 01:34:09.448628 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:34:09.448412    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e999d104-c9ea-45a7-9930-7f829013c52e-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e999d104-c9ea-45a7-9930-7f829013c52e-workload-certs\") pod \"e999d104-c9ea-45a7-9930-7f829013c52e\" (UID: \"e999d104-c9ea-45a7-9930-7f829013c52e\") "
Oct 31 01:34:09.448628 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:34:09.448473    4284 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e999d104-c9ea-45a7-9930-7f829013c52e-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e999d104-c9ea-45a7-9930-7f829013c52e-workload-socket\") pod \"e999d104-c9ea-45a7-9930-7f829013c52e\" (UID: \"e999d104-c9ea-45a7-9930-7f829013c52e\") "
Oct 31 01:34:09.448628 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:34:09.448572    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e999d104-c9ea-45a7-9930-7f829013c52e-istio-data" pod "e999d104-c9ea-45a7-9930-7f829013c52e" (UID: "e999d104-c9ea-45a7-9930-7f829013c52e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:34:09.448628 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:34:09.448574    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e999d104-c9ea-45a7-9930-7f829013c52e-istiod-ca-cert" pod "e999d104-c9ea-45a7-9930-7f829013c52e" (UID: "e999d104-c9ea-45a7-9930-7f829013c52e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 01:34:09.448975 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:34:09.448942    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e999d104-c9ea-45a7-9930-7f829013c52e-workload-certs" pod "e999d104-c9ea-45a7-9930-7f829013c52e" (UID: "e999d104-c9ea-45a7-9930-7f829013c52e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:34:09.449060 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:34:09.449040    4284 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e999d104-c9ea-45a7-9930-7f829013c52e-istiod-ca-cert\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:34:09.449141 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:34:09.449067    4284 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e999d104-c9ea-45a7-9930-7f829013c52e-istio-data\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:34:09.449141 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:34:09.449122    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e999d104-c9ea-45a7-9930-7f829013c52e-credential-socket" pod "e999d104-c9ea-45a7-9930-7f829013c52e" (UID: "e999d104-c9ea-45a7-9930-7f829013c52e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:34:09.449355 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:34:09.449336    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e999d104-c9ea-45a7-9930-7f829013c52e-workload-socket" pod "e999d104-c9ea-45a7-9930-7f829013c52e" (UID: "e999d104-c9ea-45a7-9930-7f829013c52e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:34:09.450488 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:34:09.450459    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e999d104-c9ea-45a7-9930-7f829013c52e-istio-token" pod "e999d104-c9ea-45a7-9930-7f829013c52e" (UID: "e999d104-c9ea-45a7-9930-7f829013c52e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:34:09.450600 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:34:09.450516    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e999d104-c9ea-45a7-9930-7f829013c52e-kube-api-access-vl9hm" pod "e999d104-c9ea-45a7-9930-7f829013c52e" (UID: "e999d104-c9ea-45a7-9930-7f829013c52e"). InnerVolumeSpecName "kube-api-access-vl9hm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:34:09.450912 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:34:09.450896    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e999d104-c9ea-45a7-9930-7f829013c52e-istio-podinfo" pod "e999d104-c9ea-45a7-9930-7f829013c52e" (UID: "e999d104-c9ea-45a7-9930-7f829013c52e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 01:34:09.450980 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:34:09.450943    4284 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e999d104-c9ea-45a7-9930-7f829013c52e-istio-envoy" pod "e999d104-c9ea-45a7-9930-7f829013c52e" (UID: "e999d104-c9ea-45a7-9930-7f829013c52e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:34:09.550465 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:34:09.550435    4284 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e999d104-c9ea-45a7-9930-7f829013c52e-istio-envoy\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:34:09.550465 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:34:09.550463    4284 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e999d104-c9ea-45a7-9930-7f829013c52e-credential-socket\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:34:09.550465 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:34:09.550472    4284 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e999d104-c9ea-45a7-9930-7f829013c52e-workload-certs\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:34:09.550702 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:34:09.550481    4284 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e999d104-c9ea-45a7-9930-7f829013c52e-workload-socket\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:34:09.550702 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:34:09.550491    4284 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e999d104-c9ea-45a7-9930-7f829013c52e-istio-token\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:34:09.550702 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:34:09.550499    4284 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e999d104-c9ea-45a7-9930-7f829013c52e-istio-podinfo\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:34:09.550702 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:34:09.550507    4284 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vl9hm\" (UniqueName: \"kubernetes.io/projected/e999d104-c9ea-45a7-9930-7f829013c52e-kube-api-access-vl9hm\") on node \"ip-10-0-132-173.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 01:34:09.689100 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:34:09.689067    4284 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_ai-gateway-controller-d6d549b95-zmcsl_3b3b88b3-5e11-44e1-9e0c-084e86e20d96/manager/0.log"
Oct 31 01:34:10.133296 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:34:10.133260    4284 generic.go:356] "Generic (PLEG): container finished" podID="e999d104-c9ea-45a7-9930-7f829013c52e" containerID="4c17ea82450732e49ab6f3eafde1abf9c57f07d3fa62849d8311a5eaf9435ccf" exitCode=0
Oct 31 01:34:10.133505 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:34:10.133321    4284 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-nc722"
Oct 31 01:34:10.133505 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:34:10.133352    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-nc722" event={"ID":"e999d104-c9ea-45a7-9930-7f829013c52e","Type":"ContainerDied","Data":"4c17ea82450732e49ab6f3eafde1abf9c57f07d3fa62849d8311a5eaf9435ccf"}
Oct 31 01:34:10.133505 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:34:10.133416    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-nc722" event={"ID":"e999d104-c9ea-45a7-9930-7f829013c52e","Type":"ContainerDied","Data":"7f8dca014920dab7f7db7d6a317608194db2aa8f2ff3d46fa0e48adce5090040"}
Oct 31 01:34:10.133505 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:34:10.133443    4284 scope.go:122] "RemoveContainer" containerID="4c17ea82450732e49ab6f3eafde1abf9c57f07d3fa62849d8311a5eaf9435ccf"
Oct 31 01:34:10.142006 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:34:10.141978    4284 scope.go:122] "RemoveContainer" containerID="08276160cce04ccd7627275ee1008a8ae05b058341b2e2d9f29fa4049acfdb7d"
Oct 31 01:34:10.150171 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:34:10.150153    4284 scope.go:122] "RemoveContainer" containerID="4c17ea82450732e49ab6f3eafde1abf9c57f07d3fa62849d8311a5eaf9435ccf"
Oct 31 01:34:10.150469 ip-10-0-132-173 kubenswrapper[4284]: E1031 01:34:10.150448    4284 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4c17ea82450732e49ab6f3eafde1abf9c57f07d3fa62849d8311a5eaf9435ccf\": container with ID starting with 4c17ea82450732e49ab6f3eafde1abf9c57f07d3fa62849d8311a5eaf9435ccf not found: ID does not exist" containerID="4c17ea82450732e49ab6f3eafde1abf9c57f07d3fa62849d8311a5eaf9435ccf"
Oct 31 01:34:10.150550 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:34:10.150476    4284 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4c17ea82450732e49ab6f3eafde1abf9c57f07d3fa62849d8311a5eaf9435ccf"} err="failed to get container status \"4c17ea82450732e49ab6f3eafde1abf9c57f07d3fa62849d8311a5eaf9435ccf\": rpc error: code = NotFound desc = could not find container \"4c17ea82450732e49ab6f3eafde1abf9c57f07d3fa62849d8311a5eaf9435ccf\": container with ID starting with 4c17ea82450732e49ab6f3eafde1abf9c57f07d3fa62849d8311a5eaf9435ccf not found: ID does not exist"
Oct 31 01:34:10.150550 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:34:10.150495    4284 scope.go:122] "RemoveContainer" containerID="08276160cce04ccd7627275ee1008a8ae05b058341b2e2d9f29fa4049acfdb7d"
Oct 31 01:34:10.150763 ip-10-0-132-173 kubenswrapper[4284]: E1031 01:34:10.150746    4284 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"08276160cce04ccd7627275ee1008a8ae05b058341b2e2d9f29fa4049acfdb7d\": container with ID starting with 08276160cce04ccd7627275ee1008a8ae05b058341b2e2d9f29fa4049acfdb7d not found: ID does not exist" containerID="08276160cce04ccd7627275ee1008a8ae05b058341b2e2d9f29fa4049acfdb7d"
Oct 31 01:34:10.150815 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:34:10.150768    4284 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"08276160cce04ccd7627275ee1008a8ae05b058341b2e2d9f29fa4049acfdb7d"} err="failed to get container status \"08276160cce04ccd7627275ee1008a8ae05b058341b2e2d9f29fa4049acfdb7d\": rpc error: code = NotFound desc = could not find container \"08276160cce04ccd7627275ee1008a8ae05b058341b2e2d9f29fa4049acfdb7d\": container with ID starting with 08276160cce04ccd7627275ee1008a8ae05b058341b2e2d9f29fa4049acfdb7d not found: ID does not exist"
Oct 31 01:34:10.245444 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:34:10.245363    4284 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-module-controller-manager-7566cf5fc8-9l4r5_e452dfac-477e-46b3-8300-e90197abb7e9/manager/0.log"
Oct 31 01:34:10.340479 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:34:10.340436    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-nc722"]
Oct 31 01:34:10.388310 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:34:10.388273    4284 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-8bec73-openshift-default-5bb9f8ccd5-nc722"]
Oct 31 01:34:11.025029 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:34:11.024995    4284 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-6685f84c75-8wmg2_8b5a139b-f0ff-4d24-b17d-cd6698ff3de5/manager/1.log"
Oct 31 01:34:11.033291 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:34:11.033264    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:34:11.961611 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:34:11.961580    4284 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e999d104-c9ea-45a7-9930-7f829013c52e" path="/var/lib/kubelet/pods/e999d104-c9ea-45a7-9930-7f829013c52e/volumes"
Oct 31 01:34:12.311488 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:34:12.311395    4284 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-6685f84c75-8wmg2_8b5a139b-f0ff-4d24-b17d-cd6698ff3de5/manager/0.log"
Oct 31 01:34:12.457471 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:34:12.457438    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:34:17.012736 ip-10-0-132-173 kubenswrapper[4284]: E1031 01:34:17.012701    4284 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fd8sp" containerName="extractor"
Oct 31 01:34:23.789368 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:34:23.789333    4284 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_ai-gateway-controller-d6d549b95-zmcsl_3b3b88b3-5e11-44e1-9e0c-084e86e20d96/manager/0.log"
Oct 31 01:34:24.393004 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:34:24.392972    4284 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-module-controller-manager-7566cf5fc8-9l4r5_e452dfac-477e-46b3-8300-e90197abb7e9/manager/0.log"
Oct 31 01:34:25.183050 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:34:25.183018    4284 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-6685f84c75-8wmg2_8b5a139b-f0ff-4d24-b17d-cd6698ff3de5/manager/1.log"
Oct 31 01:34:26.316371 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:34:26.316341    4284 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-6685f84c75-8wmg2_8b5a139b-f0ff-4d24-b17d-cd6698ff3de5/manager/0.log"
Oct 31 01:34:26.919983 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:34:26.919952    4284 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-759ffb85c5-kdc86_d3cb3add-09fd-4bea-9441-d720542fbda1/maas-api/0.log"
Oct 31 01:34:27.184773 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:34:27.184691    4284 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-sub-4b97d7b7-lq6kx_e0d645bd-be9d-43da-b745-40ff32e3a51a/revoke-keys/0.log"
Oct 31 01:34:28.765446 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:34:28.765421    4284 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-sub-f88cd455-zwqnv_c756875d-8413-4d7d-ba88-c78cc982b3f1/revoke-keys/0.log"
Oct 31 01:34:28.987821 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:34:28.987783    4284 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-7cdbf6dc44-rw68z_6e4242f6-ec4d-4d19-a7e2-47611818369a/postgres/0.log"
Oct 31 01:34:34.113099 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:34:34.113063    4284 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-7ddd9cfdc8-bzn9w_746ed5e5-2723-4768-8824-63b135583812/discovery/0.log"
Oct 31 01:34:34.472817 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:34:34.472787    4284 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-67865c945c-ffm6r_241f1b93-92b4-4480-93db-6b4121d8b112/kube-auth-proxy/0.log"
Oct 31 01:34:35.453143 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:34:35.453114    4284 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7b98bb678c-9xcfc_1d75c55d-1352-44c8-8199-cbd0381ac2cf/payload-pre-processing/0.log"
Oct 31 01:34:35.672624 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:34:35.672592    4284 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-55fbbbf9dc-j9vgx_5fa890d5-a2f9-495a-972b-06b8db658c88/payload-processing/0.log"
Oct 31 01:34:36.875884 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:34:36.875853    4284 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-cbd4f5586-jqgc5_3e84fbe8-43ba-445f-90f9-8d6e899db62f/main/0.log"
Oct 31 01:34:37.147730 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:34:37.147664    4284 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-7b998b79d9-zdlfm_a691a8c5-a962-488d-9b76-41f83166cd8c/main/0.log"
Oct 31 01:34:37.689189 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:34:37.689157    4284 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-69b588897d-zk45d_bf572ad9-cd2a-43ad-a627-6a1f0dc0536f/main/0.log"
Oct 31 01:34:37.950611 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:34:37.950535    4284 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b82bqnm_543969c3-5246-4037-9101-7c566b535e9d/main/0.log"
Oct 31 01:34:38.475693 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:34:38.475661    4284 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-84b4ff656c-8mnf5_0f2982ea-8a19-4b39-b859-f52896b34b5b/main/0.log"
Oct 31 01:34:47.013629 ip-10-0-132-173 kubenswrapper[4284]: E1031 01:34:47.013545    4284 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fd8sp" containerName="extractor"
Oct 31 01:34:49.328065 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:34:49.328025    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:34:49.412043 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:34:49.412014    4284 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-must-gather-86prg/must-gather-94l6z"]
Oct 31 01:34:49.412365 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:34:49.412354    4284 cpu_manager.go:414] "containerMap: removing container" podUID="fc15a078-6a56-4f96-b6a7-ff31044d75c6" containerName="payload-pre-processing"
Oct 31 01:34:49.412417 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:34:49.412367    4284 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="fc15a078-6a56-4f96-b6a7-ff31044d75c6" containerName="payload-pre-processing"
Oct 31 01:34:49.412417 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:34:49.412390    4284 cpu_manager.go:414] "containerMap: removing container" podUID="c756875d-8413-4d7d-ba88-c78cc982b3f1" containerName="revoke-keys"
Oct 31 01:34:49.412417 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:34:49.412395    4284 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c756875d-8413-4d7d-ba88-c78cc982b3f1" containerName="revoke-keys"
Oct 31 01:34:49.412417 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:34:49.412402    4284 cpu_manager.go:414] "containerMap: removing container" podUID="e999d104-c9ea-45a7-9930-7f829013c52e" containerName="istio-proxy"
Oct 31 01:34:49.412417 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:34:49.412407    4284 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e999d104-c9ea-45a7-9930-7f829013c52e" containerName="istio-proxy"
Oct 31 01:34:49.412587 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:34:49.412421    4284 cpu_manager.go:414] "containerMap: removing container" podUID="e999d104-c9ea-45a7-9930-7f829013c52e" containerName="istio-proxy"
Oct 31 01:34:49.412587 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:34:49.412425    4284 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e999d104-c9ea-45a7-9930-7f829013c52e" containerName="istio-proxy"
Oct 31 01:34:49.412587 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:34:49.412434    4284 cpu_manager.go:414] "containerMap: removing container" podUID="e9f98efa-eb3e-4a3f-bf43-9b1603d0b725" containerName="payload-processing"
Oct 31 01:34:49.412587 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:34:49.412440    4284 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e9f98efa-eb3e-4a3f-bf43-9b1603d0b725" containerName="payload-processing"
Oct 31 01:34:49.412587 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:34:49.412450    4284 cpu_manager.go:414] "containerMap: removing container" podUID="c3fc41e7-426c-4bdb-b5ef-55dce305cfae" containerName="cleanup"
Oct 31 01:34:49.412587 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:34:49.412454    4284 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c3fc41e7-426c-4bdb-b5ef-55dce305cfae" containerName="cleanup"
Oct 31 01:34:49.412587 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:34:49.412460    4284 cpu_manager.go:414] "containerMap: removing container" podUID="eed2968a-68fc-44a9-beb1-669b189009de" containerName="istio-proxy"
Oct 31 01:34:49.412587 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:34:49.412465    4284 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="eed2968a-68fc-44a9-beb1-669b189009de" containerName="istio-proxy"
Oct 31 01:34:49.412587 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:34:49.412533    4284 memory_manager.go:365] "RemoveStaleState removing state" podUID="ccf1138e-2e92-4ab6-bfdd-9979f0b69893" containerName="istio-proxy"
Oct 31 01:34:49.412587 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:34:49.412542    4284 memory_manager.go:365] "RemoveStaleState removing state" podUID="c3fc41e7-426c-4bdb-b5ef-55dce305cfae" containerName="cleanup"
Oct 31 01:34:49.412587 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:34:49.412549    4284 memory_manager.go:365] "RemoveStaleState removing state" podUID="e6cf830a-f9d2-4d0c-90de-672a81d3a441" containerName="istio-proxy"
Oct 31 01:34:49.412587 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:34:49.412555    4284 memory_manager.go:365] "RemoveStaleState removing state" podUID="e999d104-c9ea-45a7-9930-7f829013c52e" containerName="istio-proxy"
Oct 31 01:34:49.412587 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:34:49.412561    4284 memory_manager.go:365] "RemoveStaleState removing state" podUID="e9f98efa-eb3e-4a3f-bf43-9b1603d0b725" containerName="payload-processing"
Oct 31 01:34:49.412587 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:34:49.412566    4284 memory_manager.go:365] "RemoveStaleState removing state" podUID="fc15a078-6a56-4f96-b6a7-ff31044d75c6" containerName="payload-pre-processing"
Oct 31 01:34:49.412587 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:34:49.412574    4284 memory_manager.go:365] "RemoveStaleState removing state" podUID="c756875d-8413-4d7d-ba88-c78cc982b3f1" containerName="revoke-keys"
Oct 31 01:34:49.412587 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:34:49.412579    4284 memory_manager.go:365] "RemoveStaleState removing state" podUID="e999d104-c9ea-45a7-9930-7f829013c52e" containerName="istio-proxy"
Oct 31 01:34:49.412587 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:34:49.412584    4284 memory_manager.go:365] "RemoveStaleState removing state" podUID="eed2968a-68fc-44a9-beb1-669b189009de" containerName="istio-proxy"
Oct 31 01:34:49.415651 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:34:49.415636    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-86prg/must-gather-94l6z"
Oct 31 01:34:49.465947 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:34:49.465917    4284 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-86prg\"/\"openshift-service-ca.crt\""
Oct 31 01:34:49.476078 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:34:49.476054    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:34:49.507664 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:34:49.507637    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5tlj2\" (UniqueName: \"kubernetes.io/projected/28e4a412-51b1-4088-b9d1-ee7883f4ca43-kube-api-access-5tlj2\") pod \"must-gather-94l6z\" (UID: \"28e4a412-51b1-4088-b9d1-ee7883f4ca43\") " pod="openshift-must-gather-86prg/must-gather-94l6z"
Oct 31 01:34:49.507797 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:34:49.507695    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/28e4a412-51b1-4088-b9d1-ee7883f4ca43-must-gather-output\") pod \"must-gather-94l6z\" (UID: \"28e4a412-51b1-4088-b9d1-ee7883f4ca43\") " pod="openshift-must-gather-86prg/must-gather-94l6z"
Oct 31 01:34:49.513597 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:34:49.513576    4284 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-86prg\"/\"kube-root-ca.crt\""
Oct 31 01:34:49.513716 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:34:49.513658    4284 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-86prg\"/\"default-dockercfg-hnnfk\""
Oct 31 01:34:49.515355 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:34:49.515334    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-86prg/must-gather-94l6z"]
Oct 31 01:34:49.609337 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:34:49.609260    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5tlj2\" (UniqueName: \"kubernetes.io/projected/28e4a412-51b1-4088-b9d1-ee7883f4ca43-kube-api-access-5tlj2\") pod \"must-gather-94l6z\" (UID: \"28e4a412-51b1-4088-b9d1-ee7883f4ca43\") " pod="openshift-must-gather-86prg/must-gather-94l6z"
Oct 31 01:34:49.609337 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:34:49.609316    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/28e4a412-51b1-4088-b9d1-ee7883f4ca43-must-gather-output\") pod \"must-gather-94l6z\" (UID: \"28e4a412-51b1-4088-b9d1-ee7883f4ca43\") " pod="openshift-must-gather-86prg/must-gather-94l6z"
Oct 31 01:34:49.609652 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:34:49.609635    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/28e4a412-51b1-4088-b9d1-ee7883f4ca43-must-gather-output\") pod \"must-gather-94l6z\" (UID: \"28e4a412-51b1-4088-b9d1-ee7883f4ca43\") " pod="openshift-must-gather-86prg/must-gather-94l6z"
Oct 31 01:34:49.668384 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:34:49.668357    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5tlj2\" (UniqueName: \"kubernetes.io/projected/28e4a412-51b1-4088-b9d1-ee7883f4ca43-kube-api-access-5tlj2\") pod \"must-gather-94l6z\" (UID: \"28e4a412-51b1-4088-b9d1-ee7883f4ca43\") " pod="openshift-must-gather-86prg/must-gather-94l6z"
Oct 31 01:34:49.741548 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:34:49.741524    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-86prg/must-gather-94l6z"
Oct 31 01:34:50.136756 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:34:50.136733    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-86prg/must-gather-94l6z"]
Oct 31 01:34:50.138503 ip-10-0-132-173 kubenswrapper[4284]: W1031 01:34:50.138468    4284 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod28e4a412_51b1_4088_b9d1_ee7883f4ca43.slice/crio-ed5de81cb40fa169f92161fb11109c6de7083f46086268c1fd561338dec3b906 WatchSource:0}: Error finding container ed5de81cb40fa169f92161fb11109c6de7083f46086268c1fd561338dec3b906: Status 404 returned error can't find the container with id ed5de81cb40fa169f92161fb11109c6de7083f46086268c1fd561338dec3b906
Oct 31 01:34:50.140292 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:34:50.140276    4284 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 31 01:34:50.313481 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:34:50.313444    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-86prg/must-gather-94l6z" event={"ID":"28e4a412-51b1-4088-b9d1-ee7883f4ca43","Type":"ContainerStarted","Data":"ed5de81cb40fa169f92161fb11109c6de7083f46086268c1fd561338dec3b906"}
Oct 31 01:34:57.352292 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:34:57.352259    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-86prg/must-gather-94l6z" event={"ID":"28e4a412-51b1-4088-b9d1-ee7883f4ca43","Type":"ContainerStarted","Data":"9d4f0c64addeef6819ebd6af455566b9652e1874d7596c401f75d8c4d2a15efc"}
Oct 31 01:34:58.359563 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:34:58.359512    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-86prg/must-gather-94l6z" event={"ID":"28e4a412-51b1-4088-b9d1-ee7883f4ca43","Type":"ContainerStarted","Data":"9e3d6c5ab6e8c2504aebe68b029545bb6b448910b9210f1ad9f516b7477c1ebf"}
Oct 31 01:34:58.470752 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:34:58.470699    4284 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-must-gather-86prg/must-gather-94l6z" podStartSLOduration=2.363972887 podStartE2EDuration="9.470684519s" totalImagesPullingTime="7.106711632s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 01:34:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 01:34:58.469854143 +0000 UTC m=+4012.614861061" watchObservedRunningTime="2026-10-31 01:34:58.470684519 +0000 UTC m=+4012.615691430"
Oct 31 01:35:00.522816 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:35:00.522779    4284 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-24sxq_97410648-b7ae-4cf4-accf-3164c398b525/global-pull-secret-syncer/0.log"
Oct 31 01:35:01.469551 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:35:01.469513    4284 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_ai-gateway-controller-d6d549b95-zmcsl_3b3b88b3-5e11-44e1-9e0c-084e86e20d96/manager/0.log"
Oct 31 01:35:02.012935 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:35:02.012899    4284 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-module-controller-manager-7566cf5fc8-9l4r5_e452dfac-477e-46b3-8300-e90197abb7e9/manager/0.log"
Oct 31 01:35:02.568333 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:35:02.568293    4284 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-6685f84c75-8wmg2_8b5a139b-f0ff-4d24-b17d-cd6698ff3de5/manager/0.log"
Oct 31 01:35:02.871074 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:35:02.870912    4284 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-6685f84c75-8wmg2_8b5a139b-f0ff-4d24-b17d-cd6698ff3de5/manager/1.log"
Oct 31 01:35:03.666197 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:35:03.666158    4284 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_ai-gateway-controller-d6d549b95-zmcsl_3b3b88b3-5e11-44e1-9e0c-084e86e20d96/manager/0.log"
Oct 31 01:35:03.719321 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:35:03.719280    4284 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log"
Oct 31 01:35:04.119924 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:35:04.119844    4284 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-module-controller-manager-7566cf5fc8-9l4r5_e452dfac-477e-46b3-8300-e90197abb7e9/manager/0.log"
Oct 31 01:35:05.194084 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:35:05.194046    4284 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-6685f84c75-8wmg2_8b5a139b-f0ff-4d24-b17d-cd6698ff3de5/manager/0.log"
Oct 31 01:35:05.392174 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:35:05.392135    4284 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-132-173.us-west-2.compute.internal_6c9ceace9ec30530fd2581e47cd10afe/haproxy/0.log"
Oct 31 01:35:05.492141 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:35:05.492040    4284 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-6685f84c75-8wmg2_8b5a139b-f0ff-4d24-b17d-cd6698ff3de5/manager/1.log"
Oct 31 01:35:06.764958 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:35:06.764921    4284 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_ai-gateway-controller-d6d549b95-zmcsl_3b3b88b3-5e11-44e1-9e0c-084e86e20d96/manager/0.log"
Oct 31 01:35:07.200024 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:35:07.199986    4284 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-module-controller-manager-7566cf5fc8-9l4r5_e452dfac-477e-46b3-8300-e90197abb7e9/manager/0.log"
Oct 31 01:35:08.021812 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:35:08.021778    4284 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-6685f84c75-8wmg2_8b5a139b-f0ff-4d24-b17d-cd6698ff3de5/manager/0.log"
Oct 31 01:35:08.341931 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:35:08.341846    4284 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-6685f84c75-8wmg2_8b5a139b-f0ff-4d24-b17d-cd6698ff3de5/manager/1.log"
Oct 31 01:35:09.049136 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:35:09.049100    4284 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-4b97d7b7-lq6kx"]
Oct 31 01:35:09.096358 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:35:09.096328    4284 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-4b97d7b7-lq6kx"]
Oct 31 01:35:09.962050 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:35:09.962016    4284 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e0d645bd-be9d-43da-b745-40ff32e3a51a" path="/var/lib/kubelet/pods/e0d645bd-be9d-43da-b745-40ff32e3a51a/volumes"
Oct 31 01:35:10.339001 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:35:10.338918    4284 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_ai-gateway-controller-d6d549b95-zmcsl_3b3b88b3-5e11-44e1-9e0c-084e86e20d96/manager/0.log"
Oct 31 01:35:10.794012 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:35:10.793982    4284 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-module-controller-manager-7566cf5fc8-9l4r5_e452dfac-477e-46b3-8300-e90197abb7e9/manager/0.log"
Oct 31 01:35:11.341440 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:35:11.341411    4284 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-6685f84c75-8wmg2_8b5a139b-f0ff-4d24-b17d-cd6698ff3de5/manager/0.log"
Oct 31 01:35:11.387235 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:35:11.385056    4284 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k2sk2_f93a119c-fb17-44bd-973c-699a3b87523b/konnectivity-agent/0.log" limit=1024
Oct 31 01:35:11.624127 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:35:11.624035    4284 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-must-gather-dczd8/perf-node-gather-daemonset-h79bp"]
Oct 31 01:35:11.633311 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:35:11.633279    4284 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-6685f84c75-8wmg2_8b5a139b-f0ff-4d24-b17d-cd6698ff3de5/manager/1.log"
Oct 31 01:35:11.633914 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:35:11.633895    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-dczd8/perf-node-gather-daemonset-h79bp"
Oct 31 01:35:11.684089 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:35:11.684056    4284 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-dczd8\"/\"kube-root-ca.crt\""
Oct 31 01:35:11.731640 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:35:11.731604    4284 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-dczd8\"/\"openshift-service-ca.crt\""
Oct 31 01:35:11.731640 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:35:11.731602    4284 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-dczd8\"/\"default-dockercfg-b7mvt\""
Oct 31 01:35:11.733552 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:35:11.733534    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-dczd8/perf-node-gather-daemonset-h79bp"]
Oct 31 01:35:11.821264 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:35:11.821230    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/7ce5cbb0-d7bf-456e-a767-8bba2eca1e22-lib-modules\") pod \"perf-node-gather-daemonset-h79bp\" (UID: \"7ce5cbb0-d7bf-456e-a767-8bba2eca1e22\") " pod="openshift-must-gather-dczd8/perf-node-gather-daemonset-h79bp"
Oct 31 01:35:11.821468 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:35:11.821299    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/7ce5cbb0-d7bf-456e-a767-8bba2eca1e22-podres\") pod \"perf-node-gather-daemonset-h79bp\" (UID: \"7ce5cbb0-d7bf-456e-a767-8bba2eca1e22\") " pod="openshift-must-gather-dczd8/perf-node-gather-daemonset-h79bp"
Oct 31 01:35:11.821468 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:35:11.821319    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7ce5cbb0-d7bf-456e-a767-8bba2eca1e22-sys\") pod \"perf-node-gather-daemonset-h79bp\" (UID: \"7ce5cbb0-d7bf-456e-a767-8bba2eca1e22\") " pod="openshift-must-gather-dczd8/perf-node-gather-daemonset-h79bp"
Oct 31 01:35:11.821468 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:35:11.821416    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6q5hx\" (UniqueName: \"kubernetes.io/projected/7ce5cbb0-d7bf-456e-a767-8bba2eca1e22-kube-api-access-6q5hx\") pod \"perf-node-gather-daemonset-h79bp\" (UID: \"7ce5cbb0-d7bf-456e-a767-8bba2eca1e22\") " pod="openshift-must-gather-dczd8/perf-node-gather-daemonset-h79bp"
Oct 31 01:35:11.821468 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:35:11.821459    4284 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/7ce5cbb0-d7bf-456e-a767-8bba2eca1e22-proc\") pod \"perf-node-gather-daemonset-h79bp\" (UID: \"7ce5cbb0-d7bf-456e-a767-8bba2eca1e22\") " pod="openshift-must-gather-dczd8/perf-node-gather-daemonset-h79bp"
Oct 31 01:35:11.922784 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:35:11.922752    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/7ce5cbb0-d7bf-456e-a767-8bba2eca1e22-podres\") pod \"perf-node-gather-daemonset-h79bp\" (UID: \"7ce5cbb0-d7bf-456e-a767-8bba2eca1e22\") " pod="openshift-must-gather-dczd8/perf-node-gather-daemonset-h79bp"
Oct 31 01:35:11.922986 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:35:11.922793    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7ce5cbb0-d7bf-456e-a767-8bba2eca1e22-sys\") pod \"perf-node-gather-daemonset-h79bp\" (UID: \"7ce5cbb0-d7bf-456e-a767-8bba2eca1e22\") " pod="openshift-must-gather-dczd8/perf-node-gather-daemonset-h79bp"
Oct 31 01:35:11.922986 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:35:11.922819    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6q5hx\" (UniqueName: \"kubernetes.io/projected/7ce5cbb0-d7bf-456e-a767-8bba2eca1e22-kube-api-access-6q5hx\") pod \"perf-node-gather-daemonset-h79bp\" (UID: \"7ce5cbb0-d7bf-456e-a767-8bba2eca1e22\") " pod="openshift-must-gather-dczd8/perf-node-gather-daemonset-h79bp"
Oct 31 01:35:11.922986 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:35:11.922854    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/7ce5cbb0-d7bf-456e-a767-8bba2eca1e22-proc\") pod \"perf-node-gather-daemonset-h79bp\" (UID: \"7ce5cbb0-d7bf-456e-a767-8bba2eca1e22\") " pod="openshift-must-gather-dczd8/perf-node-gather-daemonset-h79bp"
Oct 31 01:35:11.922986 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:35:11.922897    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7ce5cbb0-d7bf-456e-a767-8bba2eca1e22-sys\") pod \"perf-node-gather-daemonset-h79bp\" (UID: \"7ce5cbb0-d7bf-456e-a767-8bba2eca1e22\") " pod="openshift-must-gather-dczd8/perf-node-gather-daemonset-h79bp"
Oct 31 01:35:11.922986 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:35:11.922920    4284 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/7ce5cbb0-d7bf-456e-a767-8bba2eca1e22-lib-modules\") pod \"perf-node-gather-daemonset-h79bp\" (UID: \"7ce5cbb0-d7bf-456e-a767-8bba2eca1e22\") " pod="openshift-must-gather-dczd8/perf-node-gather-daemonset-h79bp"
Oct 31 01:35:11.922986 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:35:11.922897    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/7ce5cbb0-d7bf-456e-a767-8bba2eca1e22-podres\") pod \"perf-node-gather-daemonset-h79bp\" (UID: \"7ce5cbb0-d7bf-456e-a767-8bba2eca1e22\") " pod="openshift-must-gather-dczd8/perf-node-gather-daemonset-h79bp"
Oct 31 01:35:11.922986 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:35:11.922982    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/7ce5cbb0-d7bf-456e-a767-8bba2eca1e22-proc\") pod \"perf-node-gather-daemonset-h79bp\" (UID: \"7ce5cbb0-d7bf-456e-a767-8bba2eca1e22\") " pod="openshift-must-gather-dczd8/perf-node-gather-daemonset-h79bp"
Oct 31 01:35:11.923283 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:35:11.923028    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/7ce5cbb0-d7bf-456e-a767-8bba2eca1e22-lib-modules\") pod \"perf-node-gather-daemonset-h79bp\" (UID: \"7ce5cbb0-d7bf-456e-a767-8bba2eca1e22\") " pod="openshift-must-gather-dczd8/perf-node-gather-daemonset-h79bp"
Oct 31 01:35:11.982029 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:35:11.981990    4284 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6q5hx\" (UniqueName: \"kubernetes.io/projected/7ce5cbb0-d7bf-456e-a767-8bba2eca1e22-kube-api-access-6q5hx\") pod \"perf-node-gather-daemonset-h79bp\" (UID: \"7ce5cbb0-d7bf-456e-a767-8bba2eca1e22\") " pod="openshift-must-gather-dczd8/perf-node-gather-daemonset-h79bp"
Oct 31 01:35:12.102239 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:35:12.102185    4284 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_ai-gateway-controller-d6d549b95-zmcsl_3b3b88b3-5e11-44e1-9e0c-084e86e20d96/manager/0.log"
Oct 31 01:35:12.246427 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:35:12.246355    4284 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-dczd8/perf-node-gather-daemonset-h79bp"
Oct 31 01:35:12.561764 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:35:12.561670    4284 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-module-controller-manager-7566cf5fc8-9l4r5_e452dfac-477e-46b3-8300-e90197abb7e9/manager/0.log"
Oct 31 01:35:12.619612 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:35:12.619587    4284 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-dczd8/perf-node-gather-daemonset-h79bp"]
Oct 31 01:35:12.621699 ip-10-0-132-173 kubenswrapper[4284]: W1031 01:35:12.621674    4284 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod7ce5cbb0_d7bf_456e_a767_8bba2eca1e22.slice/crio-d929acc95cad3ee35bcb3545dbde8f5e97e7fd0bfc951a75c46ee72c567bfc0d WatchSource:0}: Error finding container d929acc95cad3ee35bcb3545dbde8f5e97e7fd0bfc951a75c46ee72c567bfc0d: Status 404 returned error can't find the container with id d929acc95cad3ee35bcb3545dbde8f5e97e7fd0bfc951a75c46ee72c567bfc0d
Oct 31 01:35:13.147384 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:35:13.147354    4284 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-6685f84c75-8wmg2_8b5a139b-f0ff-4d24-b17d-cd6698ff3de5/manager/0.log"
Oct 31 01:35:13.441785 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:35:13.441741    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-dczd8/perf-node-gather-daemonset-h79bp" event={"ID":"7ce5cbb0-d7bf-456e-a767-8bba2eca1e22","Type":"ContainerStarted","Data":"ffb24fd1d9c0423a208cfb83b1977781a3d0768028947803134bd705764da66c"}
Oct 31 01:35:13.442041 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:35:13.441846    4284 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-dczd8/perf-node-gather-daemonset-h79bp" event={"ID":"7ce5cbb0-d7bf-456e-a767-8bba2eca1e22","Type":"ContainerStarted","Data":"d929acc95cad3ee35bcb3545dbde8f5e97e7fd0bfc951a75c46ee72c567bfc0d"}
Oct 31 01:35:13.442041 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:35:13.441883    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-dczd8/perf-node-gather-daemonset-h79bp"
Oct 31 01:35:13.456563 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:35:13.456532    4284 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-6685f84c75-8wmg2_8b5a139b-f0ff-4d24-b17d-cd6698ff3de5/manager/1.log"
Oct 31 01:35:13.553601 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:35:13.553547    4284 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-must-gather-dczd8/perf-node-gather-daemonset-h79bp" podStartSLOduration=2.553534997 podStartE2EDuration="2.553534997s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-31 01:35:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 01:35:13.553271668 +0000 UTC m=+4027.698278583" watchObservedRunningTime="2026-10-31 01:35:13.553534997 +0000 UTC m=+4027.698541913"
Oct 31 01:35:15.468166 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:35:15.468108    4284 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_ai-gateway-controller-d6d549b95-zmcsl_3b3b88b3-5e11-44e1-9e0c-084e86e20d96/manager/0.log"
Oct 31 01:35:15.918097 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:35:15.918064    4284 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-module-controller-manager-7566cf5fc8-9l4r5_e452dfac-477e-46b3-8300-e90197abb7e9/manager/0.log"
Oct 31 01:35:16.456075 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:35:16.456037    4284 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-6685f84c75-8wmg2_8b5a139b-f0ff-4d24-b17d-cd6698ff3de5/manager/0.log"
Oct 31 01:35:16.744100 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:35:16.743983    4284 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-6685f84c75-8wmg2_8b5a139b-f0ff-4d24-b17d-cd6698ff3de5/manager/1.log"
Oct 31 01:35:17.018480 ip-10-0-132-173 kubenswrapper[4284]: E1031 01:35:17.018396    4284 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fd8sp" containerName="extractor"
Oct 31 01:35:18.470432 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:35:18.470393    4284 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-7ddd9cfdc8-bzn9w_746ed5e5-2723-4768-8824-63b135583812/discovery/0.log"
Oct 31 01:35:18.576501 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:35:18.576470    4284 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-67865c945c-ffm6r_241f1b93-92b4-4480-93db-6b4121d8b112/kube-auth-proxy/0.log"
Oct 31 01:35:19.428751 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:35:19.428727    4284 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7b98bb678c-9xcfc_1d75c55d-1352-44c8-8199-cbd0381ac2cf/payload-pre-processing/0.log"
Oct 31 01:35:19.459160 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:35:19.459138    4284 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-dczd8/perf-node-gather-daemonset-h79bp"
Oct 31 01:35:19.638003 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:35:19.637973    4284 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-55fbbbf9dc-j9vgx_5fa890d5-a2f9-495a-972b-06b8db658c88/payload-processing/0.log"
Oct 31 01:35:21.640382 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:35:21.640345    4284 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-7ddd9cfdc8-bzn9w_746ed5e5-2723-4768-8824-63b135583812/discovery/0.log"
Oct 31 01:35:21.750308 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:35:21.750267    4284 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-67865c945c-ffm6r_241f1b93-92b4-4480-93db-6b4121d8b112/kube-auth-proxy/0.log"
Oct 31 01:35:22.563862 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:35:22.563826    4284 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7b98bb678c-9xcfc_1d75c55d-1352-44c8-8199-cbd0381ac2cf/payload-pre-processing/0.log"
Oct 31 01:35:22.766775 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:35:22.766727    4284 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-55fbbbf9dc-j9vgx_5fa890d5-a2f9-495a-972b-06b8db658c88/payload-processing/0.log"
Oct 31 01:35:25.344200 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:35:25.344166    4284 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-np54c_27f80c70-260d-492b-b0c5-8dd358f8a8f1/node-exporter/0.log"
Oct 31 01:35:25.758804 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:35:25.758760    4284 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-np54c_27f80c70-260d-492b-b0c5-8dd358f8a8f1/kube-rbac-proxy/0.log"
Oct 31 01:35:26.190433 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:35:26.190403    4284 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-np54c_27f80c70-260d-492b-b0c5-8dd358f8a8f1/init-textfile/0.log"
Oct 31 01:35:30.807864 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:35:30.807826    4284 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-9cd89b74c-4zn4l_98471199-826e-416a-8052-d88790dc7893/prometheus-operator/0.log"
Oct 31 01:35:31.211029 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:35:31.210982    4284 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-9cd89b74c-4zn4l_98471199-826e-416a-8052-d88790dc7893/kube-rbac-proxy/0.log"
Oct 31 01:35:31.621539 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:35:31.621456    4284 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-6489d8bf89-g6vkf_4a4ec287-fc28-413d-83fa-d9a05ddedba3/prometheus-operator-admission-webhook/0.log"
Oct 31 01:35:35.838462 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:35:35.838427    4284 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-7ddd9cfdc8-bzn9w_746ed5e5-2723-4768-8824-63b135583812/discovery/0.log"
Oct 31 01:35:35.938906 ip-10-0-132-173 kubenswrapper[4284]: I1031 01:35:35.938877    4284 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-67865c945c-ffm6r_241f1b93-92b4-4480-93db-6b4121d8b112/kube-auth-proxy/0.log"